Amazon Web Services - AWS Cost and Usage Extractor

Before you begin

Before you configure this ETL, the owner of the consolidated AWS Billing account must complete the following prerequisites:

1. Set up an S3 bucket to store daily billing reports from AWS. For more information about setting up the S3 bucket,  see Creating a bucket.

2. Configure an AWS IAM Account with specific privileges to access billing reports from the S3 bucket. For more information, Configuring-an-AWS-IAM-account-with-privileges-to-access-S3

3. Schedule the AWS Cost and Usage report to be generated daily. Ensure that you configure the usage report to include business service tag information for the billed items. 
   For more information, see Turn on daily reports

   After you schedule the generation of cost and usage report, it becomes available for collection from the next day.

   When creating the report, configure these parameters in the Billing and Cost Management console:

   1. Select Daily as the Time unit value.
   2. Select the Resource IDs check box to associate the resources with business services.
      

   
   

   
   

4. Ensure that you configure the Amazon Web Services - AWS API Extractor to collect tagging related data. The ETL module then creates a hierarchy of business service entities and related resources. 

   The Amazon Web Services - AWS Cost and Usage Extractor can organize the cost and usage data that it collects, and present it per business service to the TrueSight Cloud Cost Control. To learn more about the collection of business service data by ETLs, see Collecting-business-service-data.

Integration steps

To integrate TrueSight Capacity Optimization with the extractor, perform the following task:

1. In the TrueSight Capacity Optimization console, navigate to Administration > ETL & SYSTEM TASKS > ETL tasks.
2. In the ETL tasks page, under the Last run tab, click Add > Add ETL.
   The Add ETL page is displayed.

3. In the Run configuration tab, for the ETL module property, select Amazon Web Services - AWS Cost and Usage Extractor.
   Specify values for the properties under each expandable tab. For details about the common properties, see ETL-common-configuration-properties. 

   Note

   By default, the most common, basic properties that you can set for an ETL are displayed in the Add ETL page. You can accept these default selections.

4. In the Entity catalog tab, ensure that you select the same entity catalog that is used by the Amazon-Web-Services-AWS-API-Extractor ETL module.

5. In the Amazon Web Services Connection tab, specify values for the following properties:

   The [confluence_table-plus] macro is a standalone macro and it cannot be used inline.
   
   

6. Click Save.
   You return to the Last run tab under the ETL tasks page.
7. In simulation mode, validate the results: In the ETL tasks table under ETL tasks > Last run, locate your ETL (ETL task name), click Run  to run the ETL.
   After you run the ETL, the Last exit column in the ETL tasks table will display one of the following values:
   
   1. OK: The ETL executed without any error in simulation mode.
   2. WARNING: The ETL execution returned some warnings in simulation mode. Check the ETL log.
   3. ERROR: The ETL execution returned errors and was unsuccessful. Edit the active Run configuration and try again.
8. After you verify that the ETL is running correctly, switch the ETL to production mode:
   
   1. In the ETL tasks table under ETL tasks > Last run, click the ETL name under the Name column.
   2. In the Run configurations table in the ETL details page, click Edit  to edit the active run configuration.
   3. In the Edit run configuration page, navigate to the Run configuration expandable tab and set Execute in simulation mode to No.
   4. Click Save.
9. Locate the ETL in the ETL tasks table and either schedule an ETL run or click Run  to run it now.
   After you run the ETL, or schedule the ETL for a run, it will extract the data from the source and transfer it to the TrueSight Capacity Optimization database.