k8s (Kubernetes) Heapster version_11.5.00

"Moviri Integrator for TrueSight Capacity Optimization – k8s Heapster" is an additional component of BMC TrueSight Capacity Optimization product. It allows extracting data from the Kubernetes cluster management system, a leading solution to manage cloud-native containerized environments.  Relevant capacity metrics imported into BMC TrueSight Capacity Optimization, which provides advanced analytics over the extracted data in the form of an interactive dashboard, the Kubernetes View.

The integration supports the extraction of both performance and configuration data across the different components of the Kubernetes system and can configurable via parameters that allow entity filtering and many other settings. Furthermore, the connector can replicate relationships and logical dependencies among entities such as clusters, nodes, namespaces, deployments, and pods. The documentation is targeted at BMC TrueSight Capacity Optimization administrators, in charge of configuring and monitoring the integration between BMC TrueSight Capacity Optimization and Kubernetes.

• • Requirements
  • Installation
  • Data Source Check and Configuration
  • Connector Configuration
  • Supported Entities
  • Supported Metrics
  • Hierarchy
  • Kubernetes Labels to TSCO Tags

Requirements

Supported versions of the data source software

• Kubernetes versions 1.5 to 1.12
• Rancher 1.6 to 2.3 (when managing a Kubernetes cluster version 1.5 to 1.7)
• Openshift 3.9 to 3.12

Please note that the integration requires running Heapster on the Kubernetes cluster.

Supported configurations of the data source software

The Kubernetes connector requires the Heapster monitoring component of Kubernetes to be continuously and correctly monitoring the various entities supported by the integration, full list available below. Missing these requirements can result in data loss.

Also, the connector requires access to the Kubernetes API. Kubernetes API access is accessible by default with a fresh installation. 

Supported versions of TrueSight Capacity Optimization

• Supports TrueSight Capacity Optimization versions: 10.7.01 with Patch v2.0.X
• Supports TrueSight Capacity Optimization versions: 11.0 to 11.3 with Patch v2.1.X
• Supports TrueSight Capacity Optimization versions: 11.5.01 with Patch v2.2.X

Installation

Downloading the additional package

ETL Module is made available in the form of an additional component, which you may download from BMC electronic distribution site (EPD) or retrieve from your content media.

Installing the additional package

To install the connector in the form of a TrueSight Capacity Optimization additional package, refer to Performing system maintenance tasks instructions.

Data Source Check and Configuration

Preparing to connect to the data source software

The Kubernetes connector adopts a specific integration strategy to be able to extract all of the key metrics of Kubernetes environments for capacity management in a scalable fashion. In particular, the connector integrates with two data sources:

• Kubernetes API: to extract entity catalogs, relationships, and relevant configuration properties 
• Heapster: to extract key performance metrics related to the managed entities (e.g., nodes, pods, and namespaces)

The next sections outline the configuration required for the two data sources.

Kubernetes API

Kubernetes makes use of service accounts for its authentication and authorization. Upon creation, each service account gets a unique token used for authentication and authorized with read-only privileges for querying a set of specific API endpoints. 

Please use the following example to create the service account in a Kubernetes cluster using the kubectl CLI program.

Creating a Service Account

First, create the service account to be used by the Kubernetes connector using Kubectl:

Then, describe the service account to discover which shows the associated secret:

Now, describe the secret to get the corresponding token:

The token data ("eyJhb ... YygBQ") is used by the Kubernetes integration as a bearer token to authenticate against the API. Make a note of the token for ETL creation later on.

Grant the service account read-only privileges

The following section outlines a suggested example configuration on the Kubernetes cluster to allow API access to the service account used by the integrator. We provide example configurations for the two most common authorization schemes used in Kubernetes clusters, namely RBAC (Role-Based Access Control) and ABAC (Attribute-Based Access Control). To identify which authorization scheme your Kubernetes cluster has, please refer to the official project documentation: https://kubernetes.io/docs/admin/authorization/

RBAC authorization

RBAC is the authorization mode enabled by default from Kubernetes 1.6 onward. In order to utilize this feature, you must create a new cluster role. The new cluster role allows needs read-only privileges to a set of API operations and entities for the integration to work correctly.

Here is an example policy file that creates a cluster role for the integration:

Now, create the cluster role and associate it to the connector service account:

ABAC authorization

ABAC authorization grants access rights to users and service accounts via configurable policies inside a YAML file. The Kubernetes API server then uses such file via the startup parameter --authorization-policy-file.

The following policy line needs to be appended to the policy file to allow read-only access for the service account:

Restart the API server to pick up the new policy changes.

Test the configuration

After having performed the above configuration, it is useful to verify that the service account can successfully connect to the Kubernetes API and execute the intended operations. 

To verify, execute the curl Linux command from one of the TrueSight Capacity Optimization servers:

You should get a JSON document describing the nodes comprising the Kubernetes cluster.

Heapster

The Kubernetes integrator acts like a 'sink' for Heapster to receive cluster performance metrics at a custom resolution. Heapster supports a variety of sinks, but the integration uses the OpenTSDB protocol to receive Heapster messages.

The integration architecture for data collection of performance metrics works as follow:

• On Kubernetes: the Heapster monitoring component sends data to the Kubernetes integrator via OpenTSDB
• On TrueSight Capacity Optimization: the integration listens on a designated TCP port on the TrueSight Capacity Optimization server and stores the data in a database 

The Integration requires that Heapster be configured with an additional sink of type OpenTSDB. To perform this operation, add the following option to the Heapster startup command:

where:

• SERVER_HOSTNAME is the hostname (or IP address) of the TrueSight Capacity Optimization application server (or ETL engine) where the Kubernetes connector runs on.
• PORT is the TCP port on which the integration listens on. The port must be set to the same value as the "TSCO Listening Port" in the ETL configuration page.
• KUBERNETES_CLUSTER_NAME is a string representing the cluster name.

In most Kubernetes clusters, Heapster runs as a deployment within the kube-system namespace. In this scenario, the above configuration change can be carried out by changing the relevant section of the Heapster deployment YAML manifest. After configuring the change, the Heapster pod needs to be restarted for the changes to take effect. Please refer to the documentation of your Kubernetes vendor for more details on the actual Heapster configuration.

Connector Configuration

The following table shows specific properties of the connector, all the other generic properties are documented here. 

The following image shows the list of options in the ETL configuration menu, with also the advanced entries.

Supported Entities

The following entities are supported:

• Kubernetes Cluster
• Kubernetes Node
• Kubernetes Namespace (ns)
• Kubernetes Deployment
• Kubernetes Pod
• Kubernetes Persistent Volume (pv) 

Supported Metrics

The following sections describe the metrics that are managed by the Kubernetes connector.

Data Source Metrics

This section describes the metrics that are imported by the Kubernetes connector from the data source and assigns to corresponding BMC TrueSight Capacity Optimization (TSCO) entities and metrics, with minimal transformations.

Derived Metrics

This section describes the metrics that are derived by the Kubernetes connector from the data source metrics to support a wide range of capacity management use cases and analyses.

 For more details about the Kubernetes entities and resource management concepts, please refer to the official project documentation:

• Compute Resource Management: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container
• Resource Quotas: https://kubernetes.io/docs/concepts/policy/resource-quotas

Hierarchy

The connector can replicate relationships and logical dependencies among these entities as they are found configured within the Kubernetes cluster.

The integration uses the following hierarchy:

• A Kubernetes Cluster is attached to the root of the hierarchy
• Each Kubernetes Cluster contains Nodes, Namespaces, and Persistent Volumes
• Each Kubernetes Namespace contains Deployments and (standalone) Pods
• Each Kubernetes Deployment contains Pods

The following image shows a sample hierarchy.

Kubernetes Labels to TSCO Tags

The connector can import labels, which are key/value pairs attached to entities configured within the Kubernetes cluster. The labels will only be imported in TSCO versions 11 and beyond.

The keys in labels map to the TSCO Tag type, and the value from Kubernetes API label maps to the TSCO Tag value. Tag type has an underscore (_) as a word delimiter.

When specify labels want to import in configuration, specify the keys of the labels. They can be in the original format appears on Kubernetes API, or they can be using underscore (_) as a delimiter. For example, node_role_kubernetes_io_compute and node-role.kubernetes.io/compute are equivalent and will be imported as node_role_kubernetes_io_compute.

In particular, the entities we import labels are:

• Kubernetes - Pod
• Kubernetes - Node
• Kubernetes - Namespace
• Kubernetes - Deployment
• Kubernetes - Persistent Volumes

The following picture shows a sample of tags: