Control-M for Azure Data Factory

Azure Data Factory is a cloud-based ETL and data integration service that allows you to create data-driven workflows to automate the movement and transformation of data. 

Control-M for Azure Data Factory enables you to do the following:

• Connect to any Azure Data Factory pipeline from a single computer with secure login, which eliminates the need to provide authentication.
• Integrate Azure Data Factory jobs with other Control-M jobs into a single scheduling environment.
• Monitor the pipeline status and view the results in the Monitoring domain.
• Attach an SLA job to your entire Azure data service.
• Introduce all Control-M capabilities to Azure Data Factory, including advanced scheduling criteria, complex dependencies, quantitative and control resources, and variables.
• Run 50 Azure Data Factory jobs simultaneously per Control-M/Agent.

Control-M can connect to Azure Data Factory using one of the following authentication methods:

• Managed Identity: Enables you to access other Azure AD-protected resources. The identity is managed by the Azure platform and does not require you to provide additional credentials within Control-M. Use this option if the Agent is installed on an Azure virtual machine that has an assigned Managed Identity with the required permissions.
• Service Principal: An Azure service principal, also known as App Registration, is an identity created for use with applications, hosted services, and automated tools to access Azure resources. This access is restricted by the roles assigned to the service principal, which gives the Azure Administrator control over which resources can be accessed and at which level. You can use this option if the Agent is installed on-premises, on an Azure virtual machine, or any other cloud vendor.
  

Control-M for Azure Data Factory Compatibility

The following table lists the prerequisites that are required to use the Azure Data Factory plug-in, each with its minimum required version.

Control-M for Azure Data Factory is supported on Control-M Web and Control-M Automation API, but not on Control-M client.

To download the required installation files for each prerequisite, see Obtaining Control-M Installation Files.

Setting up Control-M for Azure Data Factory

This procedure describes how to deploy the Azure Data Factory plug-in, create a connection profile, and define an Azure Data Factory job in Control-M Web and Automation API.

NOTE: Integration plug-ins released by BMC require an Application Integrator installation at your site. However, these plug-ins are not editable and you cannot import them into Application Integrator. To deploy these integrations to your Control-M environment, you import them directly into Control-M using Control-M Automation API.

Before you Begin

Verify that Automation API is installed, as described in Automation API Installation.

Begin

1. Create a temporary directory to save the downloaded files.

2. Click http://www.bmc.com/available/epd and follow the instructions on the EPD site to download the Azure Data Factory plug-in from the Control-M Integrations page, or go directly to the Control-M for Azure Data Factory download page.

3. Deploy the Azure Data Factory job via Automation API, as described in deploy jobtype.
4. Create an Azure Data Factory connection profile in Control-M Web or Automation API, as follows:
   • Web: Creating a Centralized Connection Profile with Azure Data Factory Connection Profile Parameters
   • Automation API:  ConnectionProfile:ADF
5. Define an Azure Data Factory job in Control-M Web or Automation API, as follows:
   • Web: Create a Job and then define specific ADF parameters in Azure Data Factory Job parameters.
   • Automation API: Job:ADF