Troubleshooting when Approval Server fails to process records from Application Pending form

1.

Determine on which Action Request (AR) System Plug-in Server the Approval Server plug-in is running.

1. In the ar.cfg file or on the Centralized Configuration form, search for the ARSYS.ARDBC.PREVIEW plug-in alias for AR System server.

2. Check the <Server Name>:<Port> with which the plug-in alias is associated.
   In the following example, REM01 is the name of AR System server and 9999 is the port number.

   Server-Plugin-Alias: ARSYS.ARDBC.PREVIEW ARSYS.ARDBC.PREVIEW REM01:9999

   The example indicates that the plug-in server is running on port 9999 on AR System server REM01. Port 9999 indicates that the Approval Server plug-in is running on the default Java Plug-in Server of AR System.

3. If the ARSYS.ARDBC.PREVIEW plug-in alias is not available in the ar.cfg file, retrieve the alias either from a working server or environment or from the backup copy of the ar.cfg file and then add it to the ar.cfg file.

1. Run the netstat nao | findstr <port> command; where <port> is the number from step 1 on the server that is found in the ar.cfg file.
   You can see an output similar to the following image, if the plug-in server is running:
   
   The output indicates that the plug-in server is running.
2. Make sure that the connectivity to the plug-in server is not blocked by a firewall or by other monitoring programs.
3. If you do not see an entry similar to the one shown in the preceding image:
   1. Stop all AR System services and then start them after 15 seconds for all connections to close properly.
   2. Check the arerror.log, armonitor.log, and pluginserver.log files for any error messages and remediate those.
4. If the corresponding plug-in server is not running, follow step 3 to determine the plug-in server that corresponds to this port number.
5. Check the armonitor.cfg file to make sure that the plug-in server is available.
6. Use a working system to compare the armonitor.cfg files.

Alternatively, you can use the Plug-in Server Test Utility to test if the plug-in server is running and if it can accept remote procedure calls.
For more information, see How can I test to see if a Plugin Server is running?.

Troubleshoot when Approval Server takes too long to process approvals.

Approval Server should receive regular commands from the Application Signaler. If you rely on your Approval-Polling-Interval to process normal work, there might be an issue with the Application Signaler.  

Enable Approval Server logs.

1. Enable Approval Server logs.
   See How to enable the Approval log and set the logging in DEBUG mode.

2. Enable API logs, Filter logs, and SQL logs.
   See How to enable a combined log in AR Server when being requested by BMC Support?.

3. Run Approval Server long enough to capture at least a few cycles of Approval-Polling-Interval of Approval Server.

5.

Determine if Approval Server is checking for work (by analyzing Approval Server logs).

If Approval Server is accurately checking for work, you should see a message similar to the following:

This message should appear at regular intervals that correspond to the Approval-Polling-Interval value.

If Approval Server is checking for work, but is not finding any work to do, you should see a message similar to the following:

It is expected to see this message during a non-peak operation and it indicates that Approval Server could check for work, but did not find any work to do.

If Approval Server is checking for work, and has found some work to do, you should see a message similar to the following:

It is expected to see this message during a peak operation and it indicates that Approval Server could check for work, and found some work to do. Usually, this message is followed by several lines of messages detailing the record that Approval Server is working on and the function it is performing. These lines might be different based on the workflow and records that are being processed.

If you do not see any of the messages, make sure that Approval Server is running and is configured correctly.

1. Verify that Approval Server is checking for work, as described in step 5.

2. Review the API log for an entry that looks similar to the following message:
   +GLEWFARGetListEntryWithFields -- schema Application Pending from Approval Server (protocol 26) at IP address 192.168.x.x using RPC // :q:0.0s

   This message indicates that Get List Entry With Fields (which is an SQL Select statement) appears in the SQL schema Application Pending that was initiated from Approval Server. You can also use the API protocol to determine which version of approval server made the request. This message also indicates the server from which the request came and the time to process the request.

   After you see the API call being made, there should be a corresponding SQL statement in the same thread ID (TID) made by the Application Service user. The SQL statement should be a SELECT statement, where FROM is the T Table that corresponds to Application Pending, and the WHERE qualifier corresponds to the Command field on the Application Pending form. To know these values, you should inspect your individual system. See the following guidelines:

   1. To know the T Table that corresponds to your Application Pending form, run the following SQL statement on the AR System Database:
      SELECT schemaId FROM arschema WHERE name='Application Pending'
   2. To know the Column ID that corresponds to your Command field on the Application Pending form, open the Application Pending form in Developer Studio. Then, examine the database properties of the Command field. The ID found should match the column in the database.
      
      A proper SQL statement that looks for work should be similar to the following statement, where T40 equals Application Pending and T40.C501 equals Command:
      
      SELECT T40.CX, T40.CX,....... FROM T40 WHERE (T40.C501 IN ('ABCXYZ...…….')
      
      There might be additional items in the preceding statement that help in qualifying the results further. Ensure that you review these additional details so that they do not disqualify the relevant data.
      
      If you do not see an SQL statement as indicated in this step, ensure that Approval Server is running and is configured correctly.

Collect and send logs with detailed information when creating a case with BMC Support:

1. Provide the following information as part of your case:
   
   • The request example for the approval that has issues
   • The time when the issue was captured in the log files
   • Any error messages that you get in the log files
2. Attach a ZIP file to your case.
   You can attach a ZIP file of size up to 2 GB. You can also upload the file on FTP.