Deploying and configuring the Microsoft Azure - Azure API extractor

This topic explains how to configure the Microsoft Azure - Azure API extractor ETL from the TrueSight Capacity Optimization console for extracting data from the Microsoft Azure cloud deployments.

This topic covers the following information:

Related topics

Microsoft Azure - Azure API Extractor

Entities, lookup information, metrics, and API calls for Azure classic model

.Entities, lookup information, metrics, and API calls for Azure Resource Manager model v10.7.01

Working with ETLs

Prerequisites

Depending on the Azure model, complete the following prerequisite steps:

Azure Resource Manager Model
Azure Classic Model

Azure Resource Manager Model

Prerequisite step	Reference topics
Get your Azure subscription ID. The subscription ID is a GUID that uniquely identifies your subscription to use Azure services.	Steps to obtain Azure subscription ID 1. Log on to the Azure portal. 2. In the left navigation panel, click Subscriptions. The list of your subscriptions is displayed along with the subscription ID.
If you want to retrieve data for multiple subscriptions through a single ETL, create a Subscription list file that contains all the subscription details. The Subscription list file contains the Subscription ID for every subscription, each on a new line. The file can be in .txt or .csv format. You need a separate subscription file per tenant.	Steps to create a Subscription list file Open a new .txt or .csv file. Add the subscription ID and press Enter: Repeat step 2 for every subscription. For example: <subscription ID1> <subscription ID2>
Ensure that you have the required permissions to create an application in Azure Active Directory (AAD).	Check Azure Active Directory permissions
Create an AAD application to gain access to Azure resources on behalf of the ETL.	Create an Azure Active Directory application
Get the Application ID and generate an authentication key for this application.	Get application ID and authentication key
Get the Tenant ID, which is the ID of the AAD directory in which you created the application. About Tenants A Tenant is representative of an organization within Azure Active Directory. It is a dedicated instance of the Azure AD service. An AAD tenant is required for defining an application and for assigning permissions so the application can make use of other Azure services' REST APIs.	Get tenant ID
Assign API access to your application.	Steps to assign API access to your application Log on to the Microsoft Azure Resource Manager portal. The Dashboard opens. In the left pane, select Azure Active Directory. The Overview page is displayed. In the left pane of Azure Active Directory, select App Registrations, and in the right pane, select the application that you created in AAD. Click Required Permissions > +Add. In the Add API access page, complete the following steps: In the Select an API field, click Windows Azure Service Management API and click Select. In the Select permissions field, click Access Azure Service Management as organization users, and then click Select. Note: If you select the DELEGATE PERMISSIONS check box before selecting the permission, the Select button is not enabled. Click Done.
Assign reader role to the application.	In the left pane of the Azure portal menu, select Subscriptions. Select your subscription. Select the Access Control (IAM) tab, and assign the Reader role to the application. For details, see Assign application to role .

Azure Classic Model

Obtain a subscription ID: The subscription ID is a GUID that uniquely identifies your subscription to use Azure services.
Upload the management certificate to the Microsoft Azure Classic portal: A management certificate is an SSL certificate that is a small data file that digitally binds a cryptographic key to an organization's details. This certificate is associated with your subscription and is required to authenticate your API calls. Create the management certificate and upload it in the Azure portal.
Steps to create and use a management certificate
Creating and using a management certificate
Use Keytool to create and export the management certificate.

About Keytool

Keytool is a key and certificate management utility that allows users to administer their own public or private key pairs and associated certificates. The certificates are used in self-authentication (where the users authenticate themselves to other users or services) or data integrity and authentication services, using digital signatures.

The keytool.exe file is located in the same folder as the JRE. For example, on a Windows computer, the keytool.exe file is stored in C:\Program Files\Java\jre6\bin.
1. Open the command console of your operating system and navigate to the directory where keytool.exe is located and run the following command to create a keystore file:
```
keytool -genkey -keyalg RSA -alias <alias-name> -keystore <keystore-file-name>.jks 
-storepass <keystore-file-password> -keysize 2048
```
  <alias-name> is the alias name for the certificate.
  <keystore-file-name> is the name of the keystore file that you want to create.
  <keystore-file-password> is the password for the keystore file.
  The following example shows the creation of a keystore called MyAzureKeyStore.jks with P@ssword as the password. By default, the keystore file is stored in the directory from where the utility is run. For example, for JRE7, the file is stored in <C:\Program Files\Java\jre7\bin>.
```
keytool -genkeypair -alias mydomain -keyalg RSA -keystore MyAzureKeyStore.jks 
-storepass P@ssword -keysize 2048 
```
2. Provide additional information such as your name, about your organization, domain name of the server.
3. Export the management certificate by running the following command:
```
keytool -export -alias <alias-name> -storepass <keystore-file-password> -file <certificate-name> -keystore <ketstore-file-name>
```
  <certificate-name> is the name of the certificate that you want to export.
  
  See the following example of an MyAzureSMAPI.cer certificate. This certificate is created in the E:\ directory of your computer.
```
keytool -export -file E:\MyAzureSMAPI.cer -keystore MyAzureKeyStore.jks -alias mydomain
```
4. Log on to the Microsoft Azure portal , under the SETTINGS tab, click MANAGEMENT CERTIFICATES and upload the management certificate file.

Configuring the ETL module

To integrate TrueSight Capacity Optimization with the extractor, complete the following steps:

In the TrueSight Capacity Optimization console, navigate to Administration > ETL & SYSTEM TASKS > ETL tasks.
In the ETL tasks page, under the Last run tab, click Add > Add ETL.
The Add ETL page is displayed.
In the Run configuration tab, for the ETL module property, select Microsoft Azure - Azure API Extractor.

Specify values for the properties under each expandable tab. For details about the common properties, see ETL common configuration properties. The following table lists the properties that are specific to this ETL module under the Microsoft Azure configuration tab:

Note

By default, the most common, basic properties that you can set for an ETL are displayed in the Add ETL page. You can accept these default selections.

Basic properties

Property	Resource Manager Deployment	Classic Deployment	Description
Subscription access mode			Select the subscription depending on whether you want to retrieve data from a single subscription or many subscriptions. Single: Specify the following property values: In the Subscription ID property, specify the ID of the subscription for which you want to retrieve infrastructure data. Multiple: In the Multiple subscription file path box, specify the path of the file (.txt or .csv) that contains the subscription ID of all the subscriptions for which you want to retrieve data.
Keystore file path	-		Specify the complete path to the keystore file that is stored on the computer where TrueSight Capacity Optimization is installed. For example, /data1/bmc/BCO/secure/cotruststore.ts The keystore file contains the certificate entry for Azure authentication.
Keystore file password	-		Enter the password that is required to access the keystore file.
Tenant ID		-	Specify the Directory ID from your Active Directory properties.
Application ID		-	Specify the Application ID from App registrations in Azure Active Directory.
Authentication key		-	Specify the key you generated when you created the web application in Azure Active Directory.
Business Service hierarchy		-	Select Create Business Service hierarchy based on specified tag key and specify a tag key for importing business services. The default tag key is Service. Based on the tag key, the ETL creates business service entities, and maps resources to each business service. For example, if you have VMs tagged as follows: AS1: {user=John, Purpose=Dev, Service=Data Solutions} vl-pub-bco-qa35: {user=Adam, Purpose=Production, Service=Data Solutions} vl-pun-bco-qa20: {user=Jane, Purpose=QA, Service=Data Solutions} Then the ETL displays data in a hierarchy as follows: If you do not want to create business service hierarchy, select Do not create Business Service hierarchy.
Is target Azure Government Cloud		-	If you are using the Azure Government Cloud, specify Yes to extract data from the Government Cloud entities.
Use Proxy			If you have configured a proxy server to route the internet traffic to and from your Azure environment, you can configure the ETL to connect with your environment via the proxy server. If you want to use a proxy, provide the Proxy server host and port. If the Proxy server requires authentication, provide the Proxy server user name and password. By default, No is selected.

Advanced properties

Property	Classic Deployment	Resource Manager Deployment	Description
Instance type definition JSON file path			The path where you saved the JSON file that has the instance type configuration metrics. For more information, see Deploying and configuring the Microsoft Azure - Azure API extractor v11.0#Collecting data for additional instance type configuration metrics.

Click Save.
You return to the Last run tab under the ETL tasks page.
In simulation mode, validate the results: In the ETL tasks table under ETL tasks > Last run, locate your ETL (ETL task name), click Run to run the ETL.
After you run the ETL, the Last exit column in the ETL tasks table will display one of the following values:
- OK: The ETL executed without any error in simulation mode.
- WARNING: The ETL execution returned some warnings in simulation mode. Check the ETL log.
- ERROR: The ETL execution returned errors and was unsuccessful. Edit the active Run configuration and try again.
After you verify that the ETL is running correctly, switch the ETL to production mode:
1. In the ETL tasks table under ETL tasks > Last run, click the ETL name under the Name column.
2. In the Run configurations table in the ETL details page, click Edit to edit the active run configuration.
3. In the Edit run configuration page, navigate to the Run configuration expandable tab and set Execute in simulation mode to No.
4. Click Save.
Locate the ETL in the ETL tasks table and either schedule an ETL run or click Run to run it now.
After you run the ETL, or schedule the ETL for a run, it will extract the data from the source and transfer it to the TrueSight Capacity Optimization database.
In the left pane of Workspace, the following hierarchy of entities is displayed:

Resource Manager deployment mode	Classic deployment mode

Collecting data for additional instance type configuration metrics

An out-of-the-box JSON file contains the mapping for configuration metrics collected by the extractor. This file is stored on the ETL Engine server. You can configure this file to modify the existing instance type configuration or to add new configuration metrics. For example, you can add the BY_BENCHMARK_VALUE metric in the following format:

{
 	"instanceTypeConfiguration": {
		"standard_a10":{
    		"REQUESTED_INSTANCE_TYPE": "Standard_A10",
        	"CPU_MODEL":"Sparc",         
			"BYBENCHMARK_VALUE":{
             	"SPECINT2006":"2006",    
				"SPECINT2008":"2008"
			}
		},
		"standard_a11":{
			"REQUESTED_INSTANCE_TYPE": "Standard_A11"
		}
	}
}

To upload and then use a JSON file for collecting additional instance type configuration metrics:

Download the attached JSON file and update the file for existing and additional metrics.
Name the JSON file appropriately, such as msazure-metric-conf.json, and save it to your local machine.
Upload the file to a folder on the ETL Engine server.
To use the file, include it when you create the ETL:
1. In the Add ETL page, after you select Microsoft Azure - Azure extractor as the ETL, click Advanced and navigate to the Microsoft Azure configuration tab.
2. Type the file path in Instance type definition JSON file path.
3. Click Save.
  For detailed procedure, see Integration steps