Monitoring and diagnosing issues in the BMC Server Automation environment

You can also use the following tools for monitoring and diagnosing issues in the BMC Server Automation environment:

Application Server Diagnostics tool

The Application Server Diagnostics tool provides tests for monitoring the status of your BMC Server Automation environment and for working with Customer Support to identify and resolve issues. These predefined tests collect data about the status of the BMC Server Automation environment while it is running, compare the data to expected behavior, and analyze it to determine test success or failure.

1. In the BMC Server Automation Console, select Configuration > Application Server Diagnostics View.

2. In the Application Server Diagnostics view, select the Application Servers from which you want to collect data. Accept the default selection (the Application Server to which you are connected) or click Browse and select one or more Application Servers. 
   The Select Application Servers dialog box lists Application Servers configured to use the same database and file server; however, they do not need to be running on the same host. The Application Server Diagnostics area lists the tests to be run.

   Test	Description
   AppServer	Tests the Application Server configuration connectivity with other Application Servers.
   BlExec Job Diagnostic	Tests the job execution framework, including parallel execution, using a job created for test purposes.
   Database Diagnostic	Tests the Application Server connectivity with the database and executes test queries.
   Environment KeyStore	Checks if the bladelogic.keystore files are properly synchronized between the various deployments (each deployment has its own keystore file).
   File Manager Diagnostic	Tests the Application Server connectivity with the File Server.
   Pseudo Job Diagnostic	Tests the job framework, using a job created for test purposes.
   Service Deployment Diagnostic	Tests the Application Server deployment to determine if the Application Server has been properly configured

3. Accept the tests listed in the Application Server Diagnostics area or refine the list in one of the following ways:
   • Select one or more tests from the Application Server Diagnostics area (use Shift + click or Ctrl + click to select multiple tests).
   • Select a group of tests from the Diagnostic Group drop-down menu.Tests are grouped by the type of evaluation they do. For example, selecting the Configuration test group runs both the AppServer Test and the Service Deployment Test, both of which test the Application Server configuration.
   • Selecting a test group lists those tests in the Application Server Diagnostics area. You can select one or more tests from the list.
4. Do one of the following:
   • Click Run All Tests to run all tests listed in the Application Server Diagnostics area.
   • Click Run Selected Tests to run only the tests you select in that area.
5. In the Application Server Diagnostics area, the Status column shows an icon that indicates the success or failure of each test. Select a test and click the View Results icon to show detailed test results.
6. In the Diagnostic Results dialog box, select the Application Server for which you want to display test results. Then click the Test Output, Log, or Failure Advice tab.
7. When you are finished viewing test results, click Close.

Database Diagnostics tool

The Database Diagnostics tool provides tests to monitor the status of the database and to work with Customer Support to identify and resolve issues. These predefined tests collect data about the configuration of the BMC Server Automation database and provide feedback.

To run the Database Diagnostics tool, complete the following steps:

1. On the BMC Server Automation Application Server, open a shell (in Linux) or a command prompt (in Microsoft Windows).

2. From the NSH/bin folder, execute either dbdiagnostic.exe or dbdiagnostic.sh with any of the parameters shown in the following table.
   For the diagId argument used by some of the parameters, run dbdiagnostics listto determine the ID for the specific diagnostic test.

   
   DBdiagnostic command parameters

   Parameter	Description
   help	Displays help for the command.
   list	Lists only the ID, name, and description for all diagnostics.
   listFull	Lists all diagnostics with full details (for example, the parameters and their children).
   getDiag	diagId= diagId Displays information for a specific diagnostic, such as statistics on the last execution of the diagnostic, the status of the run, and the parameters used for the run.
   getDiagParams	diagId= diagId Displays the parameters for a diagnostic.
   getResAfterDate	diagId= diagId afterDate= MM-dd-yy (you can enter a 2 or 4-digit year) Displays all of the results for diagnostics recorded on or after the specified date starting at 00:00:00 AM.
   getResLastExec	diagId= diagId Displays the results for the last execution for a specific diagnostic.
   delRes	diagId= diagId Delete the results for a specific diagnostic.
   delAllRes	Deletes all results for all diagnostics.
   runDiag	diagId= <diagId> optName1 = val1 optName2 = val2 Runs a specific diagnostic using optional parameters. To view a list of parameters for a diagnostic, run the diagnostic using the getDiagParams parameter followed by the diagId.

IDs for the diagnostics

Each diagnostic test has an associated ID. Note that these IDs are not fixed and can be different in different environments.

The following table shows example IDs for each of the diagnostic tests available through the dbdiagnostics tool. To run a diagnostic test, first obtain the list of IDs by running dbdiagnostics list and then use the ID of the particular diagnostic that you want to run as input to the utility.

Sample syntax

The following example shows the command format to run the Oracle Check Block Size diagnostic. 

dbdiagnostics runDiag diagId=1000000

To view the results of the diagnostic, run the command using the getResLastExec parameter, which displays the results of the last execution for this diagnostic (the following figure).

dbdiagnostics getResLastExec diagId=1000000

Oracle Check Block Size diagnostic sample output

diagId=1000000diagId=1000000
execDiagId=2000000
execStartTime=2010-03-22 12:47:02.0
messageLevel=INFO
message=ORACLE CHECK BLOCK SIZE: Block size on the Database is 8192, which is large enough.
messageTime=2010-03-22 12:47:03.0

Running a job in debug mode

If you experience issues with job execution, use the DEBUG_MODE_ENABLED job property to provide additional diagnostic information to the job log. The additional level of logging provides you or BMC Customer Support representatives with more detailed information when diagnosing issues with job execution.

The generated debug logs are stored on the Application Servers involved in running the job in the following directory and with the following file name convention:
<installDirectory>/NSH/tmp/debug/<deploymentName>/<jobName + date>.log

To set the DEBUG_MODE_ENABLED job property, complete the following steps:

1. In the Jobs folder, select a job.
2. Do one of the following:
   • Right-click the job and select Set Property. The Set Job Properties window is displayed. Select DEBUG_MODE_ENABLED from the Name drop-down list.
   • In the Properties tab, select DEBUG_MODE_ENABLED from the Extended properties list.
3. Set the property value to True.
   The default value for the DEBUG_MODE_ENABLED property is False.

4. Click OK.

   Note

   Be sure to set the DEBUG_MODE_ENABLED property back to False when not diagnosing an issue, as running the job in Debug mode has a negative impact on performance.

To view the job log

1. Open the Jobs folder, navigate to a job or Execution Task, right-click the job or Execution Task, and select Show Results to display its job runs.
2. Select a run of a job, right-click, and select Show Log. A window displays log messages generated by the job.
3. To filter messages so the job log only shows servers with specific job results, use the Run Details drop-down to select Errors, Warnings, Success, or All.
4. To display messages in a dialog box that enables you to scroll through messages one by one, double-click a message. The Log Item Details dialog box opens. Click the Up arrow or the Down arrow to scroll through messages one by one. Click Close to close the dialog box.
5. Click Close to close the log messages window.

You can also review the log file on the Application Server for additional diagnostic information. By default, the Application Server log file is located in:
(UNIX)

/opt/bmc/bladeLogic/NSH/br/appserver.log

(Windows)

installationDirectory\BladeLogic\NSH\br\appserver.log

If you are running a multiple Application Server (MAS) environment, see Considerations for troubleshooting jobs in a MAS environment.