Troubleshooting when Approval Server fails to process records from Application Pending form

1.

Determine on which Remedy Action Request (AR) System Plug-in Server the  plug-in is running.

1. In the ar.cfg file or on the Centralized Configuration form, search for the ARSYS.ARDBC.PREVIEW plug-in alias for Remedy AR System Server.

2. Check the <Server Name>:<Port> with which the plug-in alias is associated. In the following example, REM01 is the name of AR System server and 9999 is the port number.

   Server-Plugin-Alias: ARSYS.ARDBC.PREVIEW ARSYS.ARDBC.PREVIEW REM01:9999

   The preceding example indicates that the plug-in server is running on port 9999 on AR System server REM01. Port 9999 indicates that the  plug-in is running on the default Java Plug-in Server of AR System.

3. If the ARSYS.ARDBC.PREVIEW plug-in alias is not available in the ar.cfg file, retrieve the alias either from a working server or environment or from the backup copy of the ar.cfg file and then add it to the ar.cfg file.

1. Run the netstat nao | findstr <port> command where <port> is the number from step 1 on the server that is found in the ar.cfg file. You can see an output similar to the following image, if the plug-in server is running:
   
   The preceding output indicates that the plug-in server is running.
2. Ensure that the connectivity to the plug-in server is not blocked by a firewall or by other monitoring programs.
3. If you do not see an entry similar to the one shown in the preceding image:
   1. Stop all AR System services and then start them by providing a gap of at least 15 seconds for all connections to close properly.
   2. Check the arerror.log, armonitor.log, and pluginserver.log files for any error messages and remediate those.
4. If you cannot see the corresponding plug-in server running, follow step 3 to determine the plug-in server that corresponds to this port number.
5. Check the armonitor.cfg file to ensure that the plug-in server is available.
6. Use a working system to compare the armonitor.cfg files.

Alternatively, you can use the Plug-in Server Test Utility to test if the plug-in server is running and if it can accept remote procedure calls.
For more information, see How can I test to see if a Plugin Server is running?.

Troubleshoot when  takes too long to process approvals.

 should receive regular commands from the Application Signaler. If you rely on your Approval-Polling-Interval to process normal work, there might be an issue with the Application Signaler. 

Enable  logs.

1. Enable  logs. See How to enable the Approval log and set the logging in DEBUG mode.

2. Enable API logs, Filter logs, and SQL logs. See How to enable a combined log in AR Server when being requested by BMC Support?.

3. Run  long enough to capture at least a few cycles of Approval-Polling-Interval of .

9.

Determine if  is checking for work (by analyzing  logs).

If  is accurately checking for work, you should see a message similar to the following:

This message should appear at regular intervals that correspond to the Approval-Polling-Interval value.

If  is checking for work, but is not finding any work to do, you should see a message similar to the following:

It is expected to see this message during a non-peak operation and it indicates that  could check for work, but did not find any work to do.

If  is checking for work, and has found some work to do, you should see a message similar to the following:

It is expected to see this message during a peak operation and it indicates that  could check for work, and found some work to do. Usually, this message is followed by several lines of messages detailing the record that  is working on and the function it is performing. These lines might be different based on the workflow and records that are being processed.

If you do not see the preceding messages, ensure that  is running and is configured correctly.

1. Verify that  is checking for work, as described in step 9.

2. Review the API log for an entry that looks similar to the following message:
   +GLEWFARGetListEntryWithFields -- schema Application Pending from Approval Server (protocol 26) at IP address 192.168.x.x using RPC // :q:0.0s

   This message indicates that Get List Entry With Fields (which is an SQL Select statement) appears in the SQL schema Application Pending that was initiated from . You can also use the API protocol to determine which version of approval server made the request. This message also indicates the server from which the request came and the time to process the request.

   After you see the API call being made, there should be a corresponding SQL statement in the same thread ID (TID) made by the Remedy Application Service user. The SQL statement should be a SELECT statement, where FROM is the T Table that corresponds to Application Pending, and the WHERE qualifier corresponds to the Command field on the Application Pending form. To know these values, you should inspect your individual system. See the following guidelines:

   1. To know the T Table that corresponds to your Application Pending form, run the following SQL statement on the AR System Database: SELECT schemaId FROM arschema WHERE name='Application Pending'
   2. To know the Column ID that corresponds to your Command field on the Application Pending form, open the Application Pending form in . Then, examine the database properties of the Command field. The ID found should match the column in the database.
      
      A proper SQL statement that looks for work should be similar to the following statement, where T40 equals Application Pending and T40.C501 equals Command:
      
      SELECT T40.CX, T40.CX,....... FROM T40 WHERE (T40.C501 IN ('ABCXYZ...…….')
      
      There might be additional items in the preceding statement that help in qualifying the results further. Ensure that you review these additional details so that they do not disqualify the relevant data.
      
      If you do not see an SQL statement as indicated in this step, ensure that  is running and is configured correctly.

If you still experience issues after following the preceding steps, or if you cannot complete any of the steps, run the Log Zipper utility. When you submit a case on Approval Engine, provide the configuration files and Remedy logs that you collect by running the Log Zipper utility to BMC Support.

Collect and send logs and detailed information when creating a case with BMC Support:

1. Provide the following information as part of your case:
   
   • The request example for the approval that has issues
   • The time when the issue was captured in the log files
   • Any error messages that you get in the log files

2. Attach a ZIP file to your case.
   You can attach a ZIP file of size up to 2 GB. You can also upload the file on FTP.
   For more information, see Steps to send logs, files, screenshots, etc to BMC Support for a Remedy Product related case.