Ops dashboard

The Ops dashboard of TrueSight Server Automation provides a consolidated status of job servers and jobs in your environment. You can use this dashboard to understand the overall status of jobs and other parameters related to jobs, such as work item threads and light weight work item threads. 

The Ops dashboard has the following characteristics:

  • Runs on the HTTPS port of the Maintenance Server
  • Uses the REST APIs to retrieve data that is stored in the SQLite database (flat file database) of the Maintenance Server. For information about the REST APIs used, see Rest APIs used by the Ops dashboard.

You can access this dashboard directly or using the TrueSight Server Automation console. 

Work item threads

Work items are separately-schedulable units of work performed as part of the execution of a job. It is in the execution of work items that the job carries out its responsibilities. Each job server maintains a pool of threads, known as work item threads for the execution of work items.

Light weight work item threads

Some work items are designated as lightweight work items because their execution consumes significantly fewer server resources than the execution of normal work items. In addition to the work item thread pool, each job server maintains a separate thread pool for lightweight work items, which is called the lightweight work item thread pool.

For more information, see Job-execution-framework.

Prerequisites

Do the following to view the Ops dashboard:

  1. Deploy the Maintenance Server. For details, see Deploying-a-Maintenance-Server.

  2. Enable the display of the Ops dashboard and start data collection for the dashboard.

    Do the following:
    1. Log in to the system where the Maintenance Server is deployed.
    2. (Linux) Change to the following directory:
      /opt/bmc/bladelogic/NSH/br/
    3. Start the blasadmin utility.
      ./blasadmin
    4. Run the following command to view the name of the Maintenance Server:
      list deployments
    5. Change to the Maintenance Server.
      switch <maint_server>
    6. Run this command to view all the parameters of the Maintenance Server:
      show maintenance all
    7. Change the value of the EnableOpsDashboard parameter to true. The default value is false.
      set maintenance enableopsdashboard true

    8. Restart the Maintenance Server.

      Do the following:

      1. In the 

        TrueSight Server Automation

         console, select Configuration > Infrastructure Management.

      2. Expand the Application Servers node.
      3. Right-click the Maintenance Server and select Restart.
      4. Click OK.

Analyzing jobs using the Ops dashboard

To use the Ops dashboard for analyzing jobs, do the following:

  1. Access the Ops dashboard using one of the following ways:

    • Enter the following URL in the address bar of the browser, and log in as a BLAdmin user: 
      https://<ip_address>:<maintenance_server_https_port>/ops/
      For example, https://<ip_address>:12343/ops/
    • Log in to the Health dashboard, and click Ops dashboard.

    Important

    You can access the dashboard only when you log in as a member of the BLAdmins role.

  2. Review the information on the dashboard to know the job status.

The Ops dashboard provides the following information.

The TrueSight Server Automation tab displays the following information about jobs:

  • Average percentage utilization in the last 2 minutes
  • Last 24 hours history

Last 2 Minutes Average Utilization in %

This section displays the average utilization percentage of jobs, work item threads, and light work item threads in the last two minutes.

last_2_mins_jobs_running.png

Displays the following details of running jobs:

  • Last 2 mins Job Runs: The number of jobs that were run during the last two minutes.
  • Max Concurrent Job Runs: The maximum number of jobs that ran simultaneously on the Job Servers during the last two minutes.
  • % Dial Indicator: The utilization of Job Servers (in percentage) during the last two minutes. The dial position indicates whether the capacity utilization of Job Servers is within normal limits. If the dial position starts moving towards the red zone, it indicates that the Job Server capacity is about to be used up. In this scenario, you might need to provision more Job Servers or balance the load between the existing Job Servers.

last_2_mins_workitem_threads.png

Displays the following details of work item threads:

  • Last 2 mints WITs: The number of work item threads that were used for jobs during the last two minutes.
  • Max WITs Available: The maximum work item threads that were available for jobs during the last two minutes.
  • % Dial Indicator: The percentage of work item threads that were used during the last two minutes. The dial position indicates whether the utilization of work item threads is within normal limits. If the dial position starts moving towards the red zone, it indicates that the maximum allocation limit of work items is approaching.

last_2_mins_lightweight_workitem_threads.png

Displays the following details of lightweight work item threads:

  • Last 2 mins LWWITs: The number of lightweight work item threads that were used for jobs during the last two minutes.
  • Max LWWITs Available: The maximum lightweight work item threads that were available for jobs during the last two minutes.
  • % Dial Indicator: The percentage of lightweight work item threads that were used during the last two minutes. The dial position indicates whether the utilization of work item threads is within normal limits. If the dial position starts moving towards the red zone, it indicates that the maximum allocation limit of lightweight work items is approaching.

last_2_mins_server_utilization.png

Displays a list of Job Servers with these details:

  • Allocated work item threads out of available maximum work item threads
  • Number of jobs that are run

Last 24 Hours History

This section displays the following job-related data for the last 24 hours:

  • Running jobs and jobs that are waiting to be run
  • Work item threads and lightweight work item threads

Waiting to run Vs Active Running Jobs Count

last_24_hours_wait_run_job_counts.png

The Waiting to run Vs Active Running Jobs Count chart shows the following data for the last 24 hours:

  • Date-wise count of the actively running jobs and jobs that were waiting to be run.
  • Threshold value that indicates the maximum number of jobs that are processed by the Application Servers.
  • Click Threshold to show or hide the threshold line on the graph.
  • Select a value on the X axis and drag it to select a specific duration for an enlarged view. Click zoom_out.pngto restore the original view.

Running Jobs History

last_24_hours_run_job_history.png

The Running Jobs History chart shows the following data for the last 24 hours:

  • Date-wise count of jobs that were running on each Job Server
  • Threshold value that indicates the maximum number of jobs that are processed by the Application Server
  • Click Threshold to show or hide the threshold line on the graph.
  • Select a value on the X axis and drag it to select a specific duration for an enlarged view. Click zoom_out.pngto restore the original view.

last_24_hours_workitem_threads.png

The Work Item Threads chart shows the following historical data for the last 24 hours:

  • Date-wise count of work item threads that were used for each Application Server deployment
  • Threshold value that indicates the count of maximum available work item threads
  • Click Threshold to show or hide the threshold line on the graph.
  • Select a value on the X axis and drag it to select a specific duration for an enlarged view. Click zoom_out.pngto restore the original view.

last_24_hours_lightweight_workitem_threads.png

The Light Weight Work Item Threads chart shows the following historical data for the last 24 hours:

  • Date-wise count of lightweight work item threads that were used for each Application Server deployment.
  • Threshold value that indicates the count of maximum available lightweight work item threads.
  • Click Threshold to show or hide the threshold line on the graph.
  • Select a value on the X axis and drag it to select a specific duration for an enlarged view. Click zoom_out.pngto restore the original view.

JVM utilization metrics

last_24_hours_jvm_utilization_metrics.png

The JVM Utilization Metrics chart shows a comparative analysis of JVM utilization for all the Application Server deployments in the last 24 hours.

  • Select a value on the X axis and drag it to select a specific duration for an enlarged view. Click zoom_out.pngto restore the original view.
  • Hover your mouse over jvm_utilization_mouse_over.pngto view the following details:

    jvm_utilization_tooltip.png

The Deployments tab displays the infrastructure details of each Application Server that is deployed in your environment.

Click an Application Server from the list to view its details, such as OS name, version, JRE name, version, free and used memory.

The Dashboard Configurations tab displays the auto refresh interval. Data displayed on the dashboard is refreshed every 30 seconds. This refresh interval cannot be modified.

You can configure the following parameters using the blasadmin settings: data collection interval and data retention period

Do the following:

  1. Log in to the system where the Maintenance Server is deployed.
  2. (Linux) Change to the following directory:
    /opt/bmc/bladelogic/NSH/br/
  3. Start the blasadmin utility.
    ./blasadmin
  4. Run the following command to view the Maintenance Server name:
    list deployments
  5. Change to the Maintenance Server.
    switch <maint_server>
  6. Run this command to view all the parameters of the Maintenance Server:
    show maintenance all

  7. Change the value of the following parameters:

    Parameter

    Description

    OpsDataCollectionInterval

    The interval at which data is collected. The default interval is 30 seconds. Example: To change the interval to 15 minutes, specify the value as PT15M.

    OpsDataRetentionPeriod

    The duration up to which the collected data is stored in the database. The default value is 10, which indicates that data before 10 days from the current date is removed from the database.

  8. Restart the Maintenance Server.
    Do the following:
    1. In the TrueSight Server Automation console, select Configuration > Infrastructure Management.
    2. Expand the Application Servers node.
    3. Right-click the Maintenance Server and select Restart.
    4. Click OK.


REST APIs used by the Ops dashboard 

The Ops dashboard uses the following REST APIs to retrieve data. Make sure that you acquire the session before accessing any of these APIs. You can access these APIs from the following Swagger URL:

<ip_address_of_maintenance_server_host>:<maint_server_https_port>/rest/swagger-ui.html#/metrics

Module: Metric APIs

Method

API

GET

/api/v1/metrics/{param}

GET

/api/v1/metrics/{param}/{deploymentID}

GET

/api/v1/metrics/24 hours/{param}

GET

/api/v1/metrics/all

GET

/api/v1/metrics/deployment/{deploymentID}

GET

/api/v1/metrics/last1krecs

GET

/api/v1/metrics/listalldeployments

GET

/api/v1/metrics/listallparameters

GET

/api/v1/metrics/listDbservicesDetails

GET

/api/v1/metrics/listInfraDetails

GET

/api/v1/metrics/listJobManagerServiceDetails

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*