Moviri - K8s Heapster Extractor

"Moviri Integrator for BMC Helix Continuous Optimization – k8s Heapster" is an additional component of BMC Helix Continuous Optimization product. It allows extracting data from the Kubernetes cluster management system, a leading solution to manage cloud-native containerized environments.  Relevant capacity metrics imported into BMC Helix Continuous Optimization, which provides advanced analytics over the extracted data in the form of an interactive dashboard, the Kubernetes View.

The integration supports the extraction of both performance and configuration data across the different components of the Kubernetes system and can configurable via parameters that allow entity filtering and many other settings. Furthermore, the connector can replicate relationships and logical dependencies among entities such as clusters, nodes, namespaces, deployments, and pods.

This documentation is targeted at BMC Helix Continuous Optimization administrators, in charge of configuring and monitoring the integration between BMC Helix Continuous Optimization and Kubernetes.

The "Moviri Integrator for Kubernetes - Heapster" does not support the following features:

  • "Kubernetes - Pod Workload" entity type
  • Metrics imported at the Container level (BYCONT_*)
  • Metrics imported at the Container Image level (BYCONT_IMAGE_*)
  • High Mark metrics (for CPU and Memory metrics)

To enable these features, please consider to migrate to the "Moviri Integrator for Kubernetes - Prometheus"


Step I. Complete the Pre-Configuration Tasks

Step II. Configure the ETL

Step III. Run the ETL

Step IV. Verify Data Collection


Step I. Complete the Pre-Configuration Tasks


Step II. Configure the ETL

  1. In the console, navigate to Administration > ETL & System Tasks, and select ETL tasks.
  2. On the ETL tasks page, click Add > Add ETL. The Add ETL page displays the configuration properties. You must configure properties in the following tabs: Run configuration, Entity catalog, Connection, and Kubernetes Extraction filters.
  3. On the Run Configuration tab, select Moviri - k8s Heapster Extractor from the ETL Module list. The name of the ETL is displayed in the ETL task name field. You edit this field to customize the name.
    Screen Shot 2019-07-02 at 2.45.19 PM.png
  4. Click the Entity catalog tab, and select one of the following options:
    • Shared Entity Catalog:
      • From the Sharing with Entity Catalog list, select the entity catalog name that is shared between ETLs.
    • Private Entity Catalog: Select if this is the only ETL that extracts data from the k8s Prometheus resources.

  5. Click the Connection tab, and configure the following properties:

    The [confluence_table-plus] macro is a standalone macro and it cannot be used inline. Click on this message for details.
    The ETL tasks page shows the details of the newly configured k8s Heapster ETL:
    Screen Shot 2019-07-02 at 3.25.39 PM.png

Step III. Run the ETL


After you configure the ETL, you can run it to collect data. You can run the ETL in the following modes:

A. Simulation mode: Only validates connection to the data source, does not collect data. Use this mode when you want to run the ETL for the first time or after you make any changes to the ETL configuration.

B. Production mode: Collects data from the data source.

Running the ETL in simulation mode

To run the ETL in the simulation mode:

  1. In the console, navigate to Administration > ETL & System Tasks, and select ETL tasks.
  2. On the ETL tasks page, click the ETL. The ETL details are displayed.

    Screen Shot 2019-07-02 at 3.25.39 PM.png

  3. In the Run configurations table, click Editimage2019-6-3_11-28-46.png to modify the ETL configuration settings.
  4. On the Run configuration tab, ensure that the Execute in simulation mode option is set to Yes, and click Save.
  5. Click Run active configuration. A confirmation message about the ETL run job submission is displayed.
  6. On the ETL tasks page, check the ETL run status in the Last exit column.
    OK Indicates that the ETL ran without any error. You are ready to run the ETL in the production mode.
  7.  If the ETL run status is Warning, Error, or Failed:
    1. On the ETL tasks page, click image2019-6-3_11-29-45.png in the last column of the ETL name row.
    2. Check the log and reconfigure the ETL if required.
    3. Run the ETL again.
    4. Repeat these steps until the ETL run status changes to OK.

Running the ETL in the production mode

You can run the ETL manually when required or schedule it to run at a specified time. As of version 20.02.00.005, the integration delays the hierarchy to delay for six hours helping to reduce the load on the loader step.

Running the ETL manually

  1. On the ETL tasks page, click the ETL. The ETL details are displayed.
  2. In the Run configurations table, click Edit image2019-6-3_11-30-9.png to modify the ETL configuration settings. The Edit run configuration page is displayed.
  3. On the Run configuration tab, select No for the Execute in simulation mode option, and click Save.
  4. To run the ETL immediately, click Run active configuration. A confirmation message about the ETL run job submission is displayed.
    When the ETL is run, it collects data from the source and transfers it to the database.

Step IV. Verify Data Collection

Verify that the ETL ran successfully and check whether the k8s Heapster data is refreshed in the Workspace.

To verify whether the ETL ran successfully:

  1. In the console, click Administration > ETL and System Tasks > ETL tasks.
  2. In the Last exec time column corresponding to the ETL name, verify that the current date and time are displayed.

To verify that the k8s Heapster data is refreshed:

  1. In the console, click Workspace.
  2. Expand (Domain name) > Systems > k8s Heapster > Instances.
  3. In the left pane, verify that the hierarchy displays the new and updated Heapster instances.
  4. Click a k8s Heapster entity, and click the Metrics tab in the right pane.
  5. Check if the Last Activity column in the Configuration metrics and Performance metrics tables displays the current date.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*