This documentation supports the 18.08 version of Remedy Action Request System.

To view the latest version, select the version from the Product version menu.

Troubleshooting when Remedy Approval Server fails to process records from Application Pending form

You might notice that Remedy Approval Server does not process records that are available on the Application Pending form. The Application Pending form contains multiple records that are assigned to approval server. Over a period of time, no records are processed. Thus, the number of pending records increases. This results in approval requests not getting processed. See the guidelines described here to resolve this issue. Alternatively, create a BMC Support case.

Symptoms

You notice that approval server does not process records that are available on the Application Pending form.

Scope

  • One or more users or applications might experience the symptoms.
  • Only approval server is known to have issues, and the guidelines are described considering other plug-ins to be functional.
  • In a Server Group environment, this issue can affect any number of servers in the group.

Resolution

Best practice

When approvals are not processed as expected, we recommend that you first restore approval server by restarting it. For more information, see Troubleshooting when approvals are not processed or other unexpected behaviors occur.
When approval server fails to process records that are available on the Application Pending form, perform the following steps to troubleshoot the issue:

StepTaskDescription

1.

Determine on which Remedy Action Request (AR) System Plug-in Server the approval server plug-in is running.
  1. In the ar.cfg file or on the Centralized Configuration form, search for the ARSYS.ARDBC.PREVIEW plug-in alias for Remedy AR System Server.
  2. Check the <Server Name>:<Port> with which the plug-in alias is associated. In the following example, REM01 is the name of AR System server and 9999 is the port number.

    Server-Plugin-Alias: ARSYS.ARDBC.PREVIEW ARSYS.ARDBC.PREVIEW REM01:9999

    The preceding example indicates that the plug-in server is running on port 9999 on AR System server REM01. Port 9999 indicates that the approval server plug-in is running on the default Java Plug-in Server of AR System.

  3. If the ARSYS.ARDBC.PREVIEW plug-in alias is not available in the ar.cfg file, retrieve the alias either from a working server or environment or from the backup copy of the ar.cfg file and then add it to the ar.cfg file.
2.Determine if the plug-in server is running.
  1. Run the netstat nao | findstr <port> command where <port> is the number from step 1 on the server that is found in the ar.cfg file. You can see an output similar to the following image, if the plug-in server is running:

    The preceding output indicates that the plug-in server is running.
  2. Ensure that the connectivity to the plug-in server is not blocked by a firewall or by other monitoring programs.
  3. If you do not see an entry similar to the one shown in the preceding image:
    1. Stop all AR System services and then start them by providing a gap of at least 15 seconds for all connections to close properly.
    2. Check the arerror.log, armonitor.log, and pluginserver.log files for any error messages and remediate those.
  4. If you cannot see the corresponding plug-in server running, follow step 3 to determine the plug-in server that corresponds to this port number.
  5. Check the armonitor.cfg file to ensure that the plug-in server is available.
  6. Use a working system to compare the armonitor.cfg files.

Alternatively, you can use the Plug-in Server Test Utility to test if the plug-in server is running and if it can accept remote procedure calls.
For more information, see How can I test to see if plug-in server is running?

3.

Determine if the plug-in server is running the PREVIEW plug-in.

  1. Log on to Remedy Mid Tier.
  2. Select Applications > AR System Administration > AR System Administration Console.
    The AR System Administration Console page is displayed.
  3. Select System > General > Plugin Server Configuration.
    The Plugin Server Configuration page is displayed.
  4. Click the Plugin Server Instance list and then click the server name and port that correspond to what you found in step 1.
  5. Click the Plugin Configuration tab.
  6. In the Plugins section, click ARSYS.ARDBC.PREVIEW.
  7. Verify that the Plugin Class Name is com.bmc.arsys.approval.main.ApprovalPlugin.

Note:

If you do not find the plug-in or the plug-in class name, the plug-in server is not running the required plug-in. You can apply the plug-in by retrieving it either from a working server or environment or from a backup copy of the pluginserver_config.xml file and then add the entry.

4.

Determine if the plug-in has a plug-in file name.
  1. Perform sub-steps 1 through 6 as described in step 3.
  2. Verify that a Plugin File Name is present that maps to a file on the server found in step 3.
    For example, you should find the C:\BMC\ARSystem\approval\bin\arasj91_build005.jar file on the server.
  3. If you do not find the file, get it from one of the following locations:
    • AR System installer
    • The latest hotfix
    • Another working server or environment
    • One of your backups

5.

Determine if the plug-in contains path elements.
  1. Perform sub-steps 1 through 6 as described in step 3.
  2. In the Path Elements section, verify that the ARSYS.ARDBC.PREVIEW.pathelement.type.path setting name is present.
    The value of this setting name changes as per your environment. For example, a new installation has the FRESH value, but an upgraded server has the UPGRADE value.
  3. Verify that the value for the ARSYS.ARDBC.PREVIEW.pathelement.type.location setting name includes the Plugin File Name as indicated in step 4.
    This plug-in also contains several other location settings.
  4. Ensure that all setting names containing locations are physically present on the server.
  5. If the plug-in server binaries are not present on the system, add or restore them from one of the following locations:

    • AR System installer
    • The latest hotfix
    • Another working server or environment
    • One of your backups
6.Determine if the plug-in that is currently running is using the correct version.
  1. Ensure that the files as indicated in step 4 and step 5 correlate with your version of AR System. You can cross-reference the version with the Server Version on the Platform tab of the Server Information form.
    As a rule of thumb, the server version and plug-in binary files should match. See the following table:

    AR Server versionName of the binary file
    9.1.07 (19.02)<name>91_build006
    9.1.06 (18.08)<name>91_build005
    9.1.05 (18.05)<name>91_build004
    9.1.04<name>91_build003
    ......
    9.1.00<name>91
  2. Do not run a different version of the binary files as compared to your AR server version, because this can cause system errors, workflow errors, and even data loss.

  3. If the plug-in server binaries are of the wrong version, add or restore them from one of the following locations:

    • AR System installer
    • The latest hotfix
    • Another working server or environment
    • One of your backups
7.Troubleshoot when approval server takes too long to process approvals.Approval server should receive regular commands from the Application Signaler. If you rely on your Approval-Polling-Interval to process normal work, there might be an issue with the Application Signaler.  In this case, see Troubleshooting performance issues while processing approvals.
8.Enable approval server logs.
  1. Enable approval server logs. See KA 000031898.
  2. Enable Java Plug-in Server logs. See KA 000031552.
  3. Enable API logs, Filter logs, and SQL logs. See KA 000093289.
  4. Run approval server long enough to capture at least a few cycles of Approval-Polling-Interval of approval server.

9.

Determine if approval server is checking for work (by analyzing approval server logs).

If approval server is accurately checking for work, you should see a message similar to the following:

Approval Scheduler is checking for pending work...

This message should appear at regular intervals that correspond to the Approval-Polling-Interval value.

If approval server is checking for work, but is not finding any work to do, you should see a message similar to the following:

No pending work for Approval Server

It is expected to see this message during a non-peak operation and it indicates that approval server could check for work, but did not find any work to do.

If approval server is checking for work, and has found some work to do, you should see a message similar to the following:

Approval Engine is working with Thread-1

It is expected to see this message during a peak operation and it indicates that approval server could check for work, and found some work to do. Usually, this message is followed by several lines of messages detailing the record that approval server is working on and the function it is performing. These lines might be different based on the workflow and records that are being processed.

If you do not see the preceding messages, ensure that approval server is running and is configured correctly.

10.Troubleshoot when you see the No pending work for Approval Server message, but there is some work available in the Application Pending form (by analyzing the API log, Filter log, and SQL log).
  1. Verify that approval server is checking for work, as described in step 9.
  2. Review the API log for an entry that looks similar to the following message:
    +GLEWFARGetListEntryWithFields -- schema Application Pending from Approval Server (protocol 26) at IP address 192.168.x.x using RPC // :q:0.0s

    This message indicates that Get List Entry With Fields (which is an SQL Select statement) appears in the SQL schema Application Pending that was initiated from approval server. You can also use the API protocol to determine which version of approval server made the request. This message also indicates the server from which the request came and the time to process the request.

    After you see the API call being made, there should be a corresponding SQL statement in the same thread ID (TID) made by the Remedy Application Service user. The SQL statement should be a SELECT statement, where FROM is the T Table that corresponds to Application Pending, and the WHERE qualifier corresponds to the Command field on the Application Pending form. To know these values, you should inspect your individual system. See the following guidelines:
    1. To know the T Table that corresponds to your Application Pending form, run the following SQL statement on the AR System Database: SELECT schemaId FROM arschema WHERE name='Application Pending'
    2. To know the Column ID that corresponds to your Command field on the Application Pending form, open the Application Pending form in Remedy Developer Studio. Then, examine the database properties of the Command field. The ID found should match the column in the database.

      A proper SQL statement that looks for work should be similar to the following statement, where T40 equals Application Pending and T40.C501 equals Command:

      SELECT T40.CX, T40.CX,....... FROM T40 WHERE (T40.C501 IN ('ABCXYZ...…….')

      There might be additional items in the preceding statement that help in qualifying the results further. Ensure that you review these additional details so that they do not disqualify the relevant data.

      If you do not see an SQL statement as indicated in this step, ensure that approval server is running and is configured correctly.
11.Analyze logs.Review the log files to identify any error messages or behaviors. See the table to troubleshoot issues based on the symptoms and error messages.
12.Run the Log Zipper utility.If you still experience issues after following the preceding steps, or if you cannot complete any of the steps, run the Log Zipper utility. When you submit a case on Approval Engine, provide the configuration files and Remedy logs that you collect by running the Log Zipper utility to BMC Support.
13.Create a BMC Support case.

Collect and send logs and detailed information when creating a case with BMC Support:

  1. Provide the following information as part of your case:
    • The request example for the approval that has issues
    • The time when the issue was captured in the log files
    • Any error messages that you get in the log files
  2. Attach a ZIP file to your case.
    You can attach a ZIP file of size up to 2 GB. You can also upload the file on FTP.
    For more information, see How to use BMC's Managed File Transfer (MFT) to send log files to BMC Support to comply with GDPR policies.

After you determine a specific symptom or error message, use the following table to identify the solution:

SymptomLocationActionReference

You get the following error message:

The specified plug-in does not exist: ARSYS.ARDBC.PREVIEW (ARERR 8755).

arerror.log file

Add the ARSYS.ARDBC.PREVIEW plug-in to the ar.cfg file.

See KA 000064940.

Related topic

Contacting Support

Was this page helpful? Yes No Submitting... Thank you

Comments