This documentation applies to the 8.0 version of Remedy Action Request System, which is in "End of Version Support." You will not be able to leave comments.

To view the latest version, select the version from the Product version menu.

Troubleshooting out-of-memory exceptions in the BMC Remedy Mid Tier

You can perform the following checks while troubleshooting out-of-memory exceptions in mid tier:

Checks to be performed in your Tomcat servlet container

  1. Make sure that multiple contexts for arsys (mid tier) are not present. Having multiple contexts for the mid tier application can lead to cache corruption, abnormally large cache folder size and out-of-memory errors. This issue is typically observed when Tomcat is used as a servlet container.
  2. Tomcat instance (or servlet engine instance) used for hosting mid tier should not be used for hosting any application other than mid tier. If any other application is hosted in the same Tomcat instance (or servlet engine instance), this can have an adverse effect on heap requirements and scalability of mid tier.

Heap settings

Following are the recommended JVM parameter settings when the servlet engine hosts only the mid tier web application:

  • Initial memory pool — 1536 MB (-Xms parameter for Java)
  • Maximum memory pool — 2048 MB (-Xmx parameter for Java)
  • PermGen allocation — 256 MB (-XX:PermSize parameter for Java)

The maximum memory pool size of 2048 MB is the minimum recommended value. Depending on the deployment scenario, this may need to be revised. For example, in a deployment scenario with heavy searches of up to 5000 objects being performed by users, maximum memory pool may need to be revised to 2048 MB or more.

When mid tier is deployed in a virtual environment, setting -Xms and -Xmx to the same number is recommended as acquiring additional memory after mid tier startup may not be possible in virtual environment.

Application level checks

  1. In Midtier/WEB-INF/lib, make sure that multiple copies of same jar file are not present. For example, when you apply any hot-fix, the Midtier.jar is copied to WEB-INF/lib and the older Midtier.jar is backed up to Midtier_backup.jar. Due to this Tomcat loads the redundant libraries/jar and this could result in unexpected behavior.
    To resolve this, it is recommended to rename Midtier.jar to a non-Jar file such as Midtier.jar.bak or remove it from the lib folder completely. Verify that the mid tier does not get deployed twice due to adding it to the webapps folder of the servlet container.
  2. Check the MidtierFolder/WEB-INF/classes/config.properties
    The value of arsystem.pooling_max_connections_per_server should be set to 80 which is its default value.
    If the value of the arsystem.pooling_max_connections_per_server parameter is set at a higher number, it might have a undesirable effect on heap consumption.

AR System server level checks

In the AR System server config file ar.cfg, check the values for the following parameters:

  1. Allow-Unqual-Queries
    Specifies whether unqualified searches can be performed on the AR System server. Valid values are T (allows unqualified searches) or F (disallows unqualified searches). The default value is T. The recommended value is F (disallow). Unqualified searches are ARGetListEntry or ARGetListEntryWithFields calls, in which the qualifier parameter is NULL or has an operation value of 0 (AR_COND_OP_NONE). Such searches might cause performance problems because they return all requests for a form. This is especially problematic for large forms. It is recommended to turn-off unqualified searches otherwise the volume of search results could be large resulting in out-of-memory errors on the mid tier. For information about Allow-Unqual-Queries, see ar.cfg or ar.conf options: A-B.
  2. Max-Entries-Per-Query
    The maximum number of requests returned by a search. The default value is no server-defined maximum (entry is not defined). The recommended value is 2000 (recommendation may need to be changed on a case by case basis). Because users can also specify the maximum number of requests returned through Search Preferences in the AR System User Preference form, the actual maximum is the lower of these values. It is recommended to set a value for this parameter as unqualified searches can yield a high number of results resulting in unpredictable system behavior. Increasing it would affect the heap size requirement of mid tier and cause an out-of-memory exception.
    For information about Max-Entries-Per-Query, see ar.cfg or ar.conf options E-M.

Analyze HeapDumps generated at the time of the out-of-memory error

If out-of-memory errors continue, then study the HeapDump (created when the JVM begins to run out of memory). To do so, add the following Java parameters to the Java options and then use the mid tier. A HeapDump is created when an out-of-memory error occurs.

  • -XX:-HeapDumpOnOutOfMemoryError
  • -XX:HeapDumpPath=Path
    Path should be replaced with Correct Folder path. Please ensure sufficient disk space for the heap dump.

The first parameter requires a dash after the colon as shown. The other parameter does not require the dash.

This version of the documentation is no longer supported. However, the documentation is available for your convenience. You will not be able to leave comments.

Comments