Collecting data for additional performance and instance type configuration metrics

You can collect additional performance metrics and instance type configuration metrics according to your requirements. The AWS API ETL does not collect these metrics by default. To collect these metrics, you must define them in JSON files and configure the ETL to use these JSON files.

Collecting additional CloudWatch metrics

You can collect additional performance metrics from CloudWatch. These metrics are not available for collection by default. To collect these metrics, you must add them in a JSON file. When you configure the ETL to use this JSON file and run the ETL, the database is populated with these additional metrics.

To collect additional performance metrics:

1. Download the attached JSON file.

2. Open the JSON file, and add the details of additional performance metrics that you want to collect. You can use the following JSON code as an example:

   Sample code for JSON file

   Collecting performance information

   {
      "systemTypeMap" :
   //Amazon Web Services
   // entity type mapping
   {
   "gm:aws": "EC2 Instance",
          "asg:aws": "Auto Scaling group",
           "volume:aws": "EBS Volume"
   },
   "typeConfiguration": [
   {
   "entypenm" : "gm:aws|asg:aws",
   //Metrics for "Instance" or "Autoscale group"
   "mapping"  : [
   //Mapping definition
   {
   "targetMetric": "NET_IN_BYTE_RATE",
   //  metric
   "formula"    : "(NetworkIn)/period",
   //Formula to compute the above metric using the CloudWatch metric
   "sourceMeters":[{
   //Method used for collecting the value from CloudWatch
   "awsMetric"    : "NetworkIn",
   //Amazon Web Services CloudWatch metric
   "statistic"    : "Sum"
   //This can be Average, Sum, SampleCount, Minimum, or Maximum
   }]
   },
   
   {
   //Additional metric mappings
   }
   }
   {
   "entypenm" : "volume:aws",
   //Metrics for "EBS Volume"
   "mapping"  : [
   {
   "targetMetric": "ST_VOLUME_TRANSFER_READ_BYTE_RATE",
   "formula"    :"(VolumeReadBytes)/duration",
   "sourceMeters":[{
   "awsMetric"    : "VolumeReadBytes",
   "statistic"    : "Sum",
   }]
   },
   {
                  "targetMetric":"BYFS_USED_SPACE_PCT",
                  "formula":"(DiskSpaceUtilization)/100",
                  "sourceMeters":[{
                  "awsMetric":"DiskSpaceUtilization",
   "statistic":"Average",
   "dimesionForSubObject":"Filesystem",
   "namespace": "System/Linux"
                   }]
   },
   {
   //Additional metric mappings
   }
   }
   
   ]
   }
   
3. Save the file. For example, aws-metric-conf.json.
4. Upload the file to a directory on the ETL engine server.
5. In BMC Helix Continuous Optimization, complete these steps:
   
   1. Click Administration > ETL and System Tasks > ETL Tasks.
   2. On the ETL tasks page, click the AWS API ETL that you want to configure. The ETL details are displayed.
   3. Click Edit. The ETL configuration settings are displayed.
   4. Click Advanced > Amazon Web Services Connection.
   5. In the Additional CloudWatch metrics JSON file path field, specify the path where the JSON file is saved.
      The path must include the file name also. For example, /opt/bmc/bco/aws-metric-conf.json
   6. Click Save.
6. Run the ETL or after the next scheduled ETL run, verify that the additional metrics are collected. For information about running the ETL and verifying data collection, see Amazon-Web-Services-AWS-API-Extractor.

Collecting additional instance type configuration metrics

You can define additional configuration metrics of an instance that you want to collect in a JSON file. When you configure the ETL to use this JSON file and run it, the database is populated with these additional configuration metrics.

To collect additional configuration metrics:

1. Download the sample attached file.
2. Open the file, and complete these steps:
   

   1. Add the details of additional configuration metrics.
      The following JSON file code shows some sample configuration metrics.

      Sample JSON format

      Collecting configuration information

      {
      "version": "20180130",
      "instanceTypeConfiguration" :{
      "m3.medium" :{
      "CPU_NUM": "1",
      "TOTAL_REAL_MEM": "4026531840",
      "DISK_SIZE" : "4000000000"
      },   
      "AWS_INSTANCE_TYPE_NAME" :{                  
      "CO_METRIC_NAME_1" : "VALUE",
      "CO_METRIC_NAME_2" :{
      "CO_METRIC_SUBRESOURCE_2" : "VALUE"
      }
      }
      }
      }

   2. Update the version parameter to add the current date in the yyyymmdd format.

      Information

      Info

      The ETL uses this parameter to verify that the file is the most recent.

   3. Save the file on your computer. For example, aws-instance-type.json.
3. Upload the file to a directory on the ETL Engine Server.
4. In BMC Helix Continuous Optimization, complete these steps:
   
   1. Click Administration > ETL and System Tasks > ETL Tasks.
   2. On the ETL tasks page, click the AWS API ETL that you want to configure. The ETL details are displayed.
   3. Click Edit . The ETL configuration settings are displayed.
   4. Click Advanced > Amazon Web Services Connection.
   5. In the Instance type definition JSON file path field, type the path where the JSON file is saved.
   6. Click Save.
5. Run the ETL or after the next scheduled ETL run, verify that the additional metrics are collected. For information about running the ETL and verifying data collection, see Amazon-Web-Services-AWS-API-Extractor.

Related topics

Collecting-EC2-instance-metrics-by-using-the-CloudWatch-agent

Entities-lookup-information-and-metrics-for-AWS-API-ETL