Monitoring and diagnosing issues in the TrueSight Server Automation environment

You can also use the following tools for monitoring and diagnosing issues in the TrueSight Server Automation environment:

Application Server Diagnostics tool

The Application Server Diagnostics tool provides tests for monitoring the status of your TrueSight Server Automation environment and for working with Customer Support to identify and resolve issues. These predefined tests collect data about the status of the TrueSight Server Automation environment while it is running, compare the data to expected behavior, and analyze it to determine test success or failure.

Note

To use the Application Server Diagnostics tool, your role must be granted the following authorizations:

  • To access the Application Server Diagnostics View requires the BL_Administration.Read authorization.
  • To run the diagnostic tests requires the BL_Administration.Write authorization.
  1. In the TrueSight Server Automation Console, select Configuration > Application Server Diagnostics View.

  2. In the Application Server Diagnostics view, select the Application Servers from which you want to collect data. Accept the default selection (the Application Server to which you are connected) or click Browse and select one or more Application Servers. 
    The Select Application Servers dialog box lists Application Servers configured to use the same database and file server; however, they do not need to be running on the same host. The Application Server Diagnostics area lists the tests to be run.

  3. Accept the tests listed in the Application Server Diagnostics area or refine the list in one of the following ways:
    • Select one or more tests from the Application Server Diagnostics area (use Shift + click or Ctrl + click to select multiple tests).
    • Select a group of tests from the Diagnostic Group drop-down menu.Tests are grouped by the type of evaluation they do. For example, selecting the Configuration test group runs both the AppServer Test and the Service Deployment Test, both of which test the Application Server configuration.
    • Selecting a test group lists those tests in the Application Server Diagnostics area. You can select one or more tests from the list.
  4. Do one of the following:
    • Click Run All Tests to run all tests listed in the Application Server Diagnostics area.
    • Click Run Selected Tests to run only the tests you select in that area.
  5. In the Application Server Diagnostics area, the Status column shows an icon that indicates the success or failure of each test. Select a test and click the View Results icon to show detailed test results.
  6. In the Diagnostic Results dialog box, select the Application Server for which you want to display test results. Then click the Test Output, Log, or Failure Advice tab.
  7. When you are finished viewing test results, click Close.

Database Diagnostics tool

The Database Diagnostics tool provides tests to monitor the status of the database and to work with Customer Support to identify and resolve issues. These predefined tests collect data about the configuration of the TrueSight Server Automation database and provide feedback.

Note

The warning messages displayed for the DB diagnostics are, in most cases, an indication that the system needs to be tuned with the values recommended for optimum performance of the product. Consult with your DBA to see whether the recommendations can be applied. For more information, see Database-performance-recommendations.

To run the Database Diagnostics tool, complete the following steps:

  1. On the TrueSight Server Automation Application Server, open a shell (in Linux) or a command prompt (in Microsoft Windows).

  2. From the NSH/bin folder, execute either dbdiagnostic.exe or dbdiagnostic.sh with any of the parameters shown in the following table.
    For the diagId argument used by some of the parameters, run dbdiagnostics list to determine the ID for the specific diagnostic test.

     DBdiagnostic command parameters

 IDs for the diagnostics

Each diagnostic test has an associated ID. Note that these IDs are not fixed and can be different in different environments.

The following table shows example IDs for each of the diagnostic tests available through the dbdiagnostics tool. To run a diagnostic test, first obtain the list of IDs by running dbdiagnostics list and then use the ID of the particular diagnostic that you want to run as input to the utility.

Note

The Top_N_tables_chk and JRE_Row_Count_Chk diagnostic tests apply to both Oracle and Microsoft SQL Server databases. The remaining tests apply to Oracle databases only.

Sample syntax

The following example shows the command format to run the Oracle Check Block Size diagnostic:

dbdiagnostics runDiag diagId=1000000

The following example shows the command format to run the DBMS_STATS_CHK diagnostic:

dbdiagnostics runDiag diagId=1000006

To view the results of the diagnostic, run the command using the getResLastExec parameter, which displays the results of the last execution for this diagnostic (the following figure):

dbdiagnostics getResLastExec diagId=1000000

or (for the example of the DBMS_STATS_CHK diagnostic):

dbdiagnostics getResLastExec diagId=1000006

 Oracle Check Block Size diagnostic sample output

diagId=1000000diagId=1000000
execDiagId=2000000
execStartTime=2010-03-22 12:47:02.0
messageLevel=INFO
message=ORACLE CHECK BLOCK SIZE: Block size on the Database is 8192, which is large enough.
messageTime=2010-03-22 12:47:03.0

 DBMS_STATS_CHK diagnostic sample output

diagId=1000006
execDiagId=2000040
execStartTime=2012-09-21 10:00:30.0
messageLevel=INFO
message=DBMS_STATS_CHK: DBMS_STATS on the Database ran 221 days ago, which is NOT OK. The Expected running of DBMS_STATS is once in 15 days. Please run BL_GATHER_SCHEMA_STATS PROC for this schema.
messageTime=2012-09-21 10:00:31.0 

Reviewing the Application Server log file for diagnostic information

You can review the log file on the Application Server for additional diagnostic information.

By default, the Application Server log file, appserver.log, is located in /opt/bmc/bladeLogic/NSH/br (on UNIX) or installationDirectory\BladeLogic\NSH\br (on Windows).

This log file contains various memory usage statistics, including statistics for the following items:

  • JVM memory
  • File descriptors
  • Work item threads
  • NSH proxy threads
  • Client connections
  • Database connections — from the client-connection pool, job-connection pool, and general-connection pool
Example

Below is an example of the Memory Monitor entries in the log file:

[20 Oct 2014 15:18:52,140] [Scheduled-System-Tasks-Thread-18] [INFO] [System:System:]
[Memory Monitor]
Total JVM (B): 328269824
Free JVM (B): 156032944
Used JVM (B): 172236880
VSize (B): 613498880
RSS (B): 469307392
Used File Descriptors: 2403
Used Work Item Threads: 0/50
Used NSH Proxy Threads: 0/15
Used Client Connections: 2/200
DB Client-Connection-Pool: 1/1/0/100/75/25
DB Job-Connection-Pool: 0/0/0/100/75/25
DB General-Connection-Pool: 1/1/0/100/75/25
Host/Appserver/Version: LEX-ENG-01/lex-eng-01/8.9.01.00


Note that the numbers of used threads and used client connections are presented as number of used out of total number of items available. For database connections from the pools, the numbers are current number of connections in primary pool (high water mark) / available connections in primary pool / used connections in primary pool / maximum connections for primary + secondary pools / maximum connections in primary pool / maximum connections in secondary pool.

The last line, with basic information about the Application Server (host name, deployment name, and version), is available as of version 8.9.01 of TrueSight Server Automation.


If you are running a multiple Application Server (MAS) environment, see Considerations for troubleshooting jobs in a MAS environment.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*