This documentation supports the 9.1 version of Remedy Action Request System.

To view the latest version, select the version from the Product version menu.

Troubleshooting overall mid tier performance issues

Actions processed by the mid tier and the resource available on the mid tier might impact mid tier performance. This section helps to obtain the appropriate logging and troubleshooting steps to either resolve the problem or create a BMC Support case.

Symptom

  • The overall mid tier performance is slow.
  • Mid tier stops responding.
  • Performance issue is observed on a specific mid tier.
  • Web server process continues to run, however the mid tier does not respond to the requests. 

Scope

All users connected to the specific mid tier experience the problem.

Resolution

Perform the following steps to troubleshoot the overall mid tier performance issues:

Step

Task

Description

1Review the configuration.

Verify the mid tier and AR server configuration.  For more information, see Health check from Remedy midtier perspective .

2

Determine the frequency of the performance issue.


Analyze when the performance issue occurs? At the start-up or after running for a predictable interval?


3

Enable logs and reproduce the issue.

To understand the behavior of the Java Virtual Machine (JVM) supporting the mid tier, perform the following steps:

  1. Gather thread dumps when the performance is slow or when the mid tier is not responding.
    Use the following options to take the thread dump:
    • Use the jVisualVM to gather thread dumps or heap dumps. For more information, see the knowledge article Remedy - Mid Tier - How to enable the Java JVisualVM process to collect logging
    • Use the Thread Dump feature on the Log Settings page of Remedy Mid Tier Configuration Tool. 
      This will create five separate thread dumps at a set interval.  The mid tier thread dumps are stored in the midtier/logs folder on the specific mid tier.

    • For mid tier versions earlier than 18.05, refer to the knowledge article Collect thread dumps in midtier without using JMX, jvisualvm or other third party tools
  2. Fetch detailed mid tier logs by using SERVLET and PERFORMANCE log categories. For more information, see Enabling logs for Remedy Mid Tier troubleshooting.
    Detailed mid tier logging gives a perspective of the mid tier to the AR System server processing. You can track the HTTP request to the mid tier log and determine if the delay is due to mid tier.

  3. Fetch Webserver logs. For more information, see Gathering additional logging information.
4.Disable the logging

Perform the steps listed in Enabling detailed Mid Tier logging and Enabling server-side AR System logs to disable the logging.

Remember to disable the logs from each mid tier and AR System server, on which logging was enabled.

5

Collect the logs

Copy the logs to another location where you can review them so that they do not get overwritten.

The logs may be large depending on volume of activity during the logging period.

Collecting mid tier logs and thread dumps

Zip up all logs in the log location specified on the Log Settings page of Remedy Mid Tier Configuration Tool. 
For Microsoft Windows, the default location is as follows:

<Installation directory>\BMC Software\ARsystem\midtier\logs 

For Linux, the default location is as follows:

/opt/bmc/ARsystem/midtier/logs

For more information, see mid tier and webserver configuration files.

7

Create a BMC Support Case

Collect and send logs and detailed information when creating a case with BMC Support:

  1. Provide the following information as part of your case:
    • Name of the user who performed the search
    • The search criteria used
    • Time when the problem occurred
    • Any error messages received
  2. Collect the logs.
    • Zip each mid tier log set.
    • Zip each AR System server log set.
    • Zip the network capture file. (*.har or *.saz file)
  3. Attach the zip file to your case. You can attach zip files upto 2 GB. You can also upload the files on FTP.
    For more information, see How to use BMC's Managed File Transfer (MFT) to send log files to BMC Support to comply with GDPR policies.  
8Analyze the logs

You can review the logs for error messages or behaviors.

Use the table below to troubleshoot based on your symptoms and error messages

After you determine a specific symptom or error message, use the following table to identify the solution:

SymptomWhereActionReference

Mid tier 9.x might stop responding due to logs

Mid tier version 9.1.03 or laterDisable the Internal log category on the Log settings page of Remedy Mid Tier Configuration Tool.

See the knowledge article Mid Tier 9x can become unresponsive due to logs


Sync cache feature causes server lockup or performance issuesMid tier 9.1.02 or earlierUpgrade to mid tier 9.1.03 or later.

See the knowledge article With Midtier 9.1.02, the Sync Cache feature may cause server lockup or performance problems


Related topic

Troubleshooting Remedy Mid Tier

Was this page helpful? Yes No Submitting... Thank you

Comments