Monitoring and diagnosing issues in the TrueSight Server Automation environment
You can also use the following tools for monitoring and diagnosing issues in the TrueSight Server Automation environment:
Application Server Diagnostics tool
The Application Server Diagnostics tool provides tests for monitoring the status of your TrueSight Server Automation environment and for working with Customer Support to identify and resolve issues. These predefined tests collect data about the status of the TrueSight Server Automation environment while it is running, compare the data to expected behavior, and analyze it to determine test success or failure.
To use the Application Server Diagnostics tool, your role must be granted the following authorizations:
- To access the Application Server Diagnostics View requires the BL_Administration.Read authorization.
- To run the diagnostic tests requires the BL_Administration.Write authorization.
- In the TrueSight Server Automation Console, select Configuration > Application Server Diagnostics View.
In the Application Server Diagnostics view, select the Application Servers from which you want to collect data. Accept the default selection (the Application Server to which you are connected) or click Browse and select one or more Application Servers.
The Select Application Servers dialog box lists Application Servers configured to use the same database and file server; however, they do not need to be running on the same host. The Application Server Diagnostics area lists the tests to be run.
Tests the Application Server configuration connectivity with other Application Servers.
BlExec Job Diagnostic
Tests the job execution framework, including parallel execution, using a job created for test purposes.
Tests the Application Server connectivity with the database and executes test queries.
Checks if the bladelogic.keystore files are properly synchronized between the various Application Servers.
File Manager Diagnostic
Tests the Application Server connectivity with the File Server.
Pseudo Job Diagnostic
Tests the job framework, using a job created for test purposes.
Service Deployment Diagnostic
Tests the Application Server deployment to determine if the Application Server has been properly configured
- Accept the tests listed in the Application Server Diagnostics area or refine the list in one of the following ways:
- Select one or more tests from the Application Server Diagnostics area (use Shift + click or Ctrl + click to select multiple tests).
- Select a group of tests from the Diagnostic Group drop-down menu.Tests are grouped by the type of evaluation they do. For example, selecting the Configuration test group runs both the AppServer Test and the Service Deployment Test, both of which test the Application Server configuration.
- Selecting a test group lists those tests in the Application Server Diagnostics area. You can select one or more tests from the list.
- Do one of the following:
- Click Run All Tests to run all tests listed in the Application Server Diagnostics area.
- Click Run Selected Tests to run only the tests you select in that area.
- In the Application Server Diagnostics area, the Status column shows an icon that indicates the success or failure of each test. Select a test and click the View Results icon to show detailed test results.
- In the Diagnostic Results dialog box, select the Application Server for which you want to display test results. Then click the Test Output, Log, or Failure Advice tab.
- When you are finished viewing test results, click Close.
Database Diagnostics tool
The Database Diagnostics tool provides tests to monitor the status of the database and to work with Customer Support to identify and resolve issues. These predefined tests collect data about the configuration of the TrueSight Server Automation database and provide feedback.
The warning messages displayed for the DB diagnostics are, in most cases, an indication that the system needs to be tuned with the values recommended for optimum performance of the product. Consult with your DBA to see whether the recommendations can be applied. For more information, see Database performance recommendations.
To run the Database Diagnostics tool, complete the following steps:
- On the TrueSight Server Automation Application Server, open a shell (in Linux) or a command prompt (in Microsoft Windows).
From the NSH/bin folder, execute either dbdiagnostic.exe or dbdiagnostic.sh with any of the parameters shown in the following table.
For the diagId argument used by some of the parameters, run
dbdiagnostics listto determine the ID for the specific diagnostic test.
DBdiagnostic command parameters
Displays help for the command.
Lists only the ID, name, and description for all diagnostics.
Lists all diagnostics with full details (for example, the parameters and their children).
diagId= diagId Displays information for a specific diagnostic, such as statistics on the last execution of the diagnostic, the status of the run, and the parameters used for the run.
diagId= diagId Displays the parameters for a diagnostic.
diagId= diagId afterDate= MM-dd-yy (you can enter a 2 or 4-digit year) Displays all of the results for diagnostics recorded on or after the specified date starting at 00:00:00 AM.
diagId= diagId Displays the results for the last execution for a specific diagnostic.
diagId= diagId Delete the results for a specific diagnostic.
Deletes all results for all diagnostics.
diagId= <diagId> optName1 = val1 optName2 = val2 Runs a specific diagnostic using optional parameters. To view a list of parameters for a diagnostic, run the diagnostic using the getDiagParams parameter followed by the diagId.
IDs for the diagnostics
Each diagnostic test has an associated ID. Note that these IDs are not fixed and can be different in different environments.
The following table shows example IDs for each of the diagnostic tests available through the dbdiagnostics tool. To run a diagnostic test, first obtain the list of IDs by running
dbdiagnostics list and then use the ID of the particular diagnostic that you want to run as input to the utility.
The Top_N_tables_chk and JRE_Row_Count_Chk diagnostic tests apply to both Oracle and Microsoft SQL Server databases. The remaining tests apply to Oracle databases only.
Diagnostic name and description
ORACLE CHECK BLOCK SIZE Checks the Oracle block size and provides advice.
ORACLE CHECK NUMBER PROCESSES ALLOWED Checks the number of Oracle processes and provides advice.
ORACLE OPTIMIZER SETTINGS CHK Checks the Oracle optimizer settings.
TOP_N_TABLES_CHK Checks the data volumes/sizes of the top N tables.
JRE_ROW_COUNT_CHK Checks the job_run table and returns the record with the largest number of events.
1000005 or 1000006
DBMS_STATS_CHK Checks to see if the Schema statistics are current (based on a user-supplied expiration), and recommends remediation if the statistics are not current.
The following example shows the command format to run the Oracle Check Block Size diagnostic:
dbdiagnostics runDiag diagId=1000000
The following example shows the command format to run the DBMS_STATS_CHK diagnostic:
dbdiagnostics runDiag diagId=1000006
To view the results of the diagnostic, run the command using the getResLastExec parameter, which displays the results of the last execution for this diagnostic (the following figure):
dbdiagnostics getResLastExec diagId=1000000
or (for the example of the DBMS_STATS_CHK diagnostic):
dbdiagnostics getResLastExec diagId=1000006
Oracle Check Block Size diagnostic sample output
diagId=1000000diagId=1000000 execDiagId=2000000 execStartTime=2010-03-22 12:47:02.0 messageLevel=INFO message=ORACLE CHECK BLOCK SIZE: Block size on the Database is 8192, which is large enough. messageTime=2010-03-22 12:47:03.0
DBMS_STATS_CHK diagnostic sample output
diagId=1000006 execDiagId=2000040 execStartTime=2012-09-21 10:00:30.0 messageLevel=INFO message=DBMS_STATS_CHK: DBMS_STATS on the Database ran 221 days ago, which is NOT OK. The Expected running of DBMS_STATS is once in 15 days. Please run BL_GATHER_SCHEMA_STATS PROC for this schema. messageTime=2012-09-21 10:00:31.0
Reviewing the Application Server log file for diagnostic information
You can review the log file on the Application Server for additional diagnostic information.
By default, the Application Server log file, appserver.log, is located in /opt/bmc/bladeLogic/NSH/br (on UNIX) or installationDirectory\BladeLogic\NSH\br (on Windows).
This log file contains various memory usage statistics, including statistics for the following items:
- JVM memory
- File descriptors
- Work item threads
- NSH proxy threads
- Client connections
- Database connections — from the client-connection pool, job-connection pool, and general-connection pool
If you are running a multiple Application Server (MAS) environment, see Considerations for troubleshooting jobs in a MAS environment.