Configuring the Azure DevOps pipeline

To configure the pipeline in Azure DevOps

1. On the Azure DevOps dashboard, click Pipelines to expand it.
2. In the upper right-hand corner of the Pipelines area, click New Pipeline.
3. From the list, select your repository type.
4. From the list, select your repository name.
5. On Configure your pipeline tab, perform one of the following actions:
   • To create a new pipeline:
     1. Click Starter pipeline.
     2. Click Rename and rename the pipeline.
     3. In the upper-right corner, click Variables and define the pipeline variables.
     4. Click either Save or Save and run.
   •  To open an existing pipeline:
     1. Click Existing Azure Pipelines YAML file.
     2. In the Select an existing YAML file dialog box, from the menu, select the branch and path of the existing pipeline, and click Continue.
     3. Click Rename and rename the pipeline.
     4. In the upper-right corner, click Variables and define the pipeline variables.
     5. Click either Run or Save.

Customizing your pipeline content for BMC AMI DevOps

1. Define the pipeline’s trigger, resources, container, and pool as configured in your environment.

   Example

   
   

   trigger:
   - none
   resources:
     containers:
     - container: my_container
       image:attach:xwiki:Mainframe.Data-for-Db2.BMC-AMI-DevOps-for-Db2.AMA13100.Using.Using-BMC-AMI-DevOps-with-Universal-Connector.Configuring-the-pipeline.WebHome@filename <Image location>/devopsuc:13.01.00.0001-GA
       options: -u root
   pool:
       name: pool1

   In this example, the image specified is your path location for the BMC AMI DevOps Universal Connector image, and you must use the -u root option to pass into the container host.

   Important

   For Linux-based containers, Azure DevOps requires the default user (USER) to have access to the groupadd and other privilege commands without using the sudo command.

   
   

   The Universal Connector container image uses a default non-root user without access to groupadd or other privilege commands required by Azure DevOps. Therefore, in the pipeline, configure the options with -u root root user, so that Azure DevOps can create and run the container with the root user. 

   Azure DevOps performs the following functions during the pipeline Initialize containers step:

   1. Fetches the agent process user name, user id, group name, and group id.
   2. Creates a new user with same user id, similar user name, same group id, and similar group name.

   After the pipeline Checkout step is complete, Azure DevOps uses the newly created user to run the pipeline-configured jobs and the steps in the container. 

   We recommend installing the self-hosted agent with a non-root user id, because Azure DevOps uses the same user id to create the new user that runs the pipeline jobs and steps inside the container.

2. You can specify whether to use variable groups, which are defined in Azure DevOps.

   Example

   variables:
   - group: Login
   - group: FileTransmission
   - group: SchemaMigration

3. Define the pipeline step scripts as follows:

   • Linux commands
   • BMC AMI DevOps application steps

   1. To display text in logs, use the Linux echo command.
      

      Example

      - script: |
          echo "user id: $(USER_ID)"
          echo "Replacing User Id and Password in the Azure/configFiles/config_LCBaseline.yml file"

   2. To replace variable values, use the Linux sed -i command. This applies to group and pipeline variables defined in Azure DevOps. You must provide the absolute path of the config YAML file in which the variable is used.
      

      Example

      - script: |
          sed -i "s/USER_ID/$(USER_ID)/" Azure/SMPE/JL/configFiles/JL_config_LCBaseline.yml
          sed -i "s/PASSWORD/$(PASSWORD)/" Azure/SMPE/JL/configFiles/JL_config_LCBaseline.yml
          sed -i "s|CERT_PATH_AUTH|$(clientCertificate.secureFilePath)|" Azure/configFiles/config_LCBaseline.yml
          sed -i "s/CERT_PASS_AUTH/$(cert_pass)/" Azure/configFiles/config_LCBaseline.yml
          sed -i "s/PLUGINFILE/$(PLUGINFILE)/" Azure/SMPE/JL/configFiles/JL_config_LCBaseline.yml
          sed -i "s/JOB_ID/$(JOB_ID)/" Azure/SMPE/JL/configFiles/JL_config_LCBaseline.yml
          sed -i "s/PIPELINENAME_VALUE/$(Build.DefinitionName)/" Azure/SMPE/JL/configFiles/JL_config_LCBaseline.yml
          sed -i "s/RUNNUMBER_VALUE/$(Build.BuildNumber)/" Azure/SMPE/JL/configFiles/JL_config_LCBaseline.yml
          sed -i "s/RUNBY_VALUE/$(Build.RequestedFor)/" Azure/SMPE/JL/configFiles/JL_config_LCBaseline.yml
          sed -i "s/RUNMODE_VALUE/$(Build.Reason)/" Azure/SMPE/JL/configFiles/JL_config_LCBaseline.yml

   3. To concatenate the config YAML file in log and preview the contents, use the Linux cat command.

      Example

      - script: |
          cat Azure/configFiles/config_LCBaseline.yml 

   1. To run the steps from the config YAML file, specify the step stepName and the config YAML file with its repository path. If config YAML file resides in the repository root path, then the path is not required. You use the step command to run the application step from the Azure DevOps Pipeline workflow configuration file. The arguments for this command are application step name, config YAML file path, and debug flag.
      To use the debug parser option, specify true after the step's config file.
      

      Example

      - script: |
          step authentication JL_config_LCBaseline.yml

   2. To print the step name in the pipeline log, specify displayName: and the 'stepName' enclosed in single quotation marks.
      

      Example

      displayName: 'Authentication'

   3. To retain and access job output from your pipeline run, use the Azure DevOps task, PublishPipelineArtifact@1, in your pipeline yaml file.

      • Set the targetPath to the workspace relative path (.) to get the files from the pipeline workspace.
      • Set the artifact to create the artifact folder.
      • Set condition to succeededOrFailed(), so that it will retain the artifacts archive no matter the previous step result.
        After the pipeline runs, in the beginning of the pipeline job console log, it produces an artifact link to the folder and saves the output files from the job and repository files.

      You can define the file, .artifactignore, in your repository to list files and folders that you want to ignore during the publish task. For more information on .artifactignore file and PublishPipelineArtifact@1 task, see the Azure DevOps documentation.

      Example

      - task: PublishPipelineArtifact@1
        inputs:
          targetPath: .
          artifact: 'report'
        condition: succeededOrFailed()

      
      The example for a sample pipeline step script is as follows:
      

      Example

      steps:
      - script: |
          echo "Replace variables defined in Config_Schema_Standards_Schema_Type_DDL.yml file"
          # Linux command sed is used to replace the config file variables with Azure DevOps pipeline variables
          sed -i "s/USER_ID_AUTH/$(user_id)/" Config_Schema_Standards_Schema_Type_DDL.yml
          sed -i "s/USER_PASS_AUTH/$(user_pass)/" Config_Schema_Standards_Schema_Type_DDL.yml
          # For certBased Authentication, uncomment the following 2 sed commands
          # sed -i "s|CERT_PATH_AUTH|$(clientCertificate.secureFilePath)|" Config_Schema_Standards_Schema_Type_DDL.yml
          # sed -i "s/CERT_PASS_AUTH/$(cert_pass)/" Config_Schema_Standards_Schema_Type_DDL.yml
          # Use Azure DevOps environment variables to fetch pipeline name, run number, run by and run mode of the executing pipeline
          sed -i "s/PIPELINENAME_VALUE/$(Build.DefinitionName)/" Config_Schema_Standards_Schema_Type_DDL.yml
          sed -i "s/RUNNUMBER_VALUE/$(Build.BuildNumber)/" Config_Schema_Standards_Schema_Type_DDL.yml
          sed -i "s/RUNBY_VALUE/$(Build.RequestedFor)/" Config_Schema_Standards_Schema_Type_DDL.yml
          sed -i "s/RUNMODE_VALUE/$(Build.Reason)/" Config_Schema_Standards_Schema_Type_DDL.yml
          echo "-----   Config_Schema_Standards_Schema_Type_DDL.yml file is updated -----"
        displayName: 'Initialization'
      - script: |
          step authentication Config_Schema_Standards_Schema_Type_DDL.yml
        displayName: 'Authentication'
      - script: |
          step schema_standards_ddl Config_Schema_Standards_Schema_Type_DDL.yml
        displayName: 'Schema Standards – Analyze DDL'

      The detailed explanation of the syntax used in the above example is as follows:

      • The example for the step command for an application step with one mandatory argument is as follows:
        

        Example

        step <arg>

        In this example, the argument <arg> is a required application step name.

      • The example for the step command with the default configuration file config.yml that is maintained under the root directory is as follows:
        

        Example

        # Execute application step - authentication
        - script: |
            step authentication
          displayName: 'Authentication'
        
        # Execute application step - schemaStandards (schema_standards_ddl)
        - script: |
            step schema_standards_ddl
          displayName: 'Schema Standards – Analyze DDL'

      • The example for step command for an application step with more than one argument is as follows:
        

        Example

        step <arg1> <arg2> <arg3>

        In this example, the arguments are defined as follows:

        • <arg1> — Required application step name.
        • <arg2> — Optional configuration YAML file path and file name.
        • <arg3> — Optional Debug flag set as Boolean. The default value is false.
         

      • The example for the step command with user-defined configuration YAML file that is maintained under the root directory is as follows:
        

        Example

        # Execute application step - authentication
        - script: |
            step authentication Config_Schema_Standards_Schema_Type_DDL.yml
          displayName: 'Authentication'
        
        # Execute application step - schemaStandards (schema_standards_ddl)
        - script: |
            step schema_standards_ddl Config_Schema_Standards_Schema_Type_DDL.yml
          displayName: 'Schema Standards – Analyze DDL'

      • The example for the step command with user-defined configuration YAML file that is maintained under a directory or sub-directory is as follows:
        

        Example

        # Execute application step - authentication
        - script: |
            step authentication /directory/sub-directory(s)/Config_Schema_Standards_Schema_Type_DDL.yml
          displayName: 'Authentication'
        # Execute application step - schemaStandards (schema_standards_ddl)
        - script: |
            step schema_standards_ddl /directory/sub-directory(s)/Config_Schema_Standards_Schema_Type_DDL.yml
          displayName: 'Schema Standards – Analyze DDL'

Defining a variable in the pipeline

You can define variables in the pipeline as pipeline variables or a variable group

For more information, see Variables-for-Universal-Connector.