Monitoring and diagnosing issues in the BMC Server Automation environment
You can also use the following tools for monitoring and diagnosing issues in the BMC Server Automation environment:
Application Server Diagnostics tool
The Application Server Diagnostics tool provides tests for monitoring the status of your BMC Server Automation environment and for working with Customer Support to identify and resolve issues. These predefined tests collect data about the status of the BMC Server Automation environment while it is running, compare the data to expected behavior, and analyze it to determine test success or failure.
To use the Application Server Diagnostics tool, your role must be granted the following authorizations:
- To access the Application Server Diagnostics View requires the BL_Administration.Read authorization.
- To run the diagnostic tests requires the BL_Administration.Write authorization.
- In the BMC Server Automation Console, select Configuration > Application Server Diagnostics View.
In the Application Server Diagnostics view, select the Application Servers from which you want to collect data. Accept the default selection (the Application Server to which you are connected) or click Browse and select one or more Application Servers.
The Select Application Servers dialog box lists Application Servers configured to use the same database and file server; however, they do not need to be running on the same host. The Application Server Diagnostics area lists the tests to be run.
Tests the Application Server configuration connectivity with other Application Servers.
BlExec Job Diagnostic
Tests the job execution framework, including parallel execution, using a job created for test purposes.
Tests the Application Server connectivity with the database and executes test queries.
Checks if the bladelogic.keystore files are properly synchronized between the various deployments (each deployment has its own keystore file).
File Manager Diagnostic
Tests the Application Server connectivity with the File Server.
Pseudo Job Diagnostic
Tests the job framework, using a job created for test purposes.
Service Deployment Diagnostic
Tests the Application Server deployment to determine if the Application Server has been properly configured
- Accept the tests listed in the Application Server Diagnostics area or refine the list in one of the following ways:
- Select one or more tests from the Application Server Diagnostics area (use Shift + click or Ctrl + click to select multiple tests).
- Select a group of tests from the Diagnostic Group drop-down menu.Tests are grouped by the type of evaluation they do. For example, selecting the Configuration test group runs both the AppServer Test and the Service Deployment Test, both of which test the Application Server configuration.
- Selecting a test group lists those tests in the Application Server Diagnostics area. You can select one or more tests from the list.
- Do one of the following:
- Click Run All Tests to run all tests listed in the Application Server Diagnostics area.
- Click Run Selected Tests to run only the tests you select in that area.
- In the Application Server Diagnostics area, the Status column shows an icon that indicates the success or failure of each test. Select a test and click the View Results icon to show detailed test results.
- In the Diagnostic Results dialog box, select the Application Server for which you want to display test results. Then click the Test Output, Log, or Failure Advice tab.
- When you are finished viewing test results, click Close.
Database Diagnostics tool
The Database Diagnostics tool provides tests to monitor the status of the database and to work with Customer Support to identify and resolve issues. These predefined tests collect data about the configuration of the BMC Server Automation database and provide feedback.
The warning messages displayed for the DB diagnostics are, in most cases, an indication that the system needs to be tuned with the values recommended for optimum performance of the product, which are listed in the BMC Server Automation Installation Guide. Consult with your DBA to see whether the recommendations can be applied.
To run the Database Diagnostics tool, complete the following steps:
- On the BMC Server Automation Application Server, open a shell (in Linux) or a command prompt (in Microsoft Windows).
From the NSH/bin folder, execute either dbdiagnostic.exe or dbdiagnostic.sh with any of the parameters shown in the following table.
For the diagId argument used by some of the parameters, run
dbdiagnostics listto determine the ID for the specific diagnostic test.
DBdiagnostic command parameters
Displays help for the command.
Lists only the ID, name, and description for all diagnostics.
Lists all diagnostics with full details (for example, the parameters and their children).
diagId= diagId Displays information for a specific diagnostic, such as statistics on the last execution of the diagnostic, the status of the run, and the parameters used for the run.
diagId= diagId Displays the parameters for a diagnostic.
diagId= diagId afterDate= MM-dd-yy (you can enter a 2 or 4-digit year) Displays all of the results for diagnostics recorded on or after the specified date starting at 00:00:00 AM.
diagId= diagId Displays the results for the last execution for a specific diagnostic.
diagId= diagId Delete the results for a specific diagnostic.
Deletes all results for all diagnostics.
diagId= <diagId> optName1 = val1 optName2 = val2 Runs a specific diagnostic using optional parameters. To view a list of parameters for a diagnostic, run the diagnostic using the getDiagParams parameter followed by the diagId.
IDs for the diagnostics
Each diagnostic test has an associated ID. Note that these IDs are not fixed and can be different in different environments.
The following table shows example IDs for each of the diagnostic tests available through the dbdiagnostics tool. To run a diagnostic test, first obtain the list of IDs by running
dbdiagnostics list and then use the ID of the particular diagnostic that you want to run as input to the utility.
The Top_N_tables_chk and JRE_Row_Count_Chk diagnostic tests apply to both Oracle and Microsoft SQL Server databases. The remaining tests apply to Oracle databases only.
Diagnostic name and description
ORACLE CHECK BLOCK SIZE Checks the Oracle block size and provides advice.
ORACLE CHECK NUMBER PROCESSES ALLOWED Checks the number of Oracle processes and provides advice.
ORACLE OPTIMIZER SETTINGS CHK Checks the Oracle optimizer settings.
TOP_N_TABLES_CHK Checks the data volumes/sizes of the top N tables.
JRE_ROW_COUNT_CHK Checks the job_run table and returns the record with the largest number of events.
DBMS_STATS_CHK Checks to see if the Schema statistics are current (based on a user-supplied expiration), and recommends remediation if the statistics are not current. To use this diagnostic to verify that Schema statistics are current, see the "Before you install" chapter of the BMC Server Automation Installation Guide.
The following example shows the command format to run the Oracle Check Block Size diagnostic.
dbdiagnostics runDiag diagId=1000000
To view the results of the diagnostic, run the command using the getResLastExec parameter, which displays the results of the last execution for this diagnostic (the following figure).
dbdiagnostics getResLastExec diagId=1000000
Oracle Check Block Size diagnostic sample output
diagId=1000000diagId=1000000 execDiagId=2000000 execStartTime=2010-03-22 12:47:02.0 messageLevel=INFO message=ORACLE CHECK BLOCK SIZE: Block size on the Database is 8192, which is large enough. messageTime=2010-03-22 12:47:03.0
Running a job in debug mode
If you experience issues with job execution, use the DEBUG_MODE_ENABLED job property to provide additional diagnostic information to the job log. The additional level of logging provides you or BMC Customer Support representatives with more detailed information when diagnosing issues with job execution.
The generated debug logs are stored on the Application Servers involved in running the job in the following directory and with the following file name convention:
<installDirectory>/NSH/tmp/debug/<deploymentName>/<jobName + date>.log
To set the DEBUG_MODE_ENABLED job property, complete the following steps:
- In the Jobs folder, select a job.
- Do one of the following:
- Right-click the job and select Set Property. The Set Job Properties window is displayed. Select DEBUG_MODE_ENABLED from the Name drop-down list.
- In the Properties tab, select DEBUG_MODE_ENABLED from the Extended properties list.
- Set the property value to True.
The default value for the DEBUG_MODE_ENABLED property is False.
Be sure to set the DEBUG_MODE_ENABLED property back to False when not diagnosing an issue, as running the job in Debug mode has a negative impact on performance.
To view the job log
- Open the Jobs folder, navigate to a job or Execution Task, right-click the job or Execution Task, and select Show Results to display its job runs.
- Select a run of a job, right-click, and select Show Log. A window displays log messages generated by the job.
- To filter messages so the job log only shows servers with specific job results, use the Run Details drop-down to select Errors, Warnings, Success, or All.
- To display messages in a dialog box that enables you to scroll through messages one by one, double-click a message. The Log Item Details dialog box opens. Click the Up arrow or the Down arrow to scroll through messages one by one. Click Close to close the dialog box.
- Click Close to close the log messages window.
You can also review the log file on the Application Server for additional diagnostic information. By default, the Application Server log file is located in:
If you are running a multiple Application Server (MAS) environment, see Considerations for troubleshooting jobs in a MAS environment.