Integrating Git to Code Pipeline for GitHub Actions

This topic describes in detail, all the tasks that you need to perform to synchronize Git and Code Pipeline when you are using the GitHub Actions orchestrators.

Task 1: Setting up the environment

This step provides you with instructions to install the required plugins and other configuration.

  1. Install EGit. For more information on the installation process, see Egit download manual.

  2. Configure GitHub Self-hosted runner.
    You can install the runner on a virtual machine or on your own machine. There are 3 levels of a self-hosted runner.

    • Repository-level runners are dedicated to a single repository.
    • Organization-level runners can process jobs for multiple repositories in an organization.
    • Enterprise-level runners can be assigned to multiple organizations in an enterprise account.

    To configure the GitHub Self-hosted runner, perform the following steps:

    1. Create the necessary environment before you run the steps for creating a self-hosted runner.


        • For an organization-level self-hosted runner, create an organization if one does not already exist.
        • For a repository-level self-hosted runner, create a new repository through GitHub, or you may simply use an existing repository if you already have one created.

      Important

      Create private repositories when working with self-hosted runners. You do not want public access to your privately self-hosted machines.

    2. In GitHub, navigate to the settings page that will allow you to create a self-hosted runner. You can find this page by going into the settings of your organization/repository and clicking on the Actions menu option within the settings.
    3. Go to the bottom of the page and find the Self-hosted runners section.
    4. Select the Add new option > New runner for an Organization > New runner for a Repository.
      GitHubWorkflowDetailedSteps_1.png

    5. You will be redirected to a new page that mentions all the steps for creating a runner. Before you perform any of the steps, make sure you select the Operating System of the machine on which you will host the runner. The following steps described below are for Windows VM, but the steps are similar to any other OS.

      Note

      The token within the Configure section of the instructions page expires and you will need to refresh it to retrieve a new token.

    6. Run the commands provided on the page using the appropriate shell for your OS.


    7. Start the PowerShell the run as administrator option within Windows. You might need to download PowerShell for Windows from Github (run the .msi in Github from https://github.com/PowerShell/PowerShell/releases/tag/v7.1.3) or bash shell for Linux. When prompted if you want the runner to run as a service, enter Yes, so that you do not have to manually start each time. Alternately, go to the actions-runner location and run the run.cmd. 

      Tip

      You can also provide labels for your runner while running the GitHub instructions in the shell window. In the beginning of the runner installation, a step will ask you the following:

      • This runner will have the following labels...
      • Enter any additional labels

      You can enter any labels you want to associate with this runner. Labels can be used to run specific runners by their label during an action. You can also change the labels after installing the runner. To change them through GitHub, see the Using labels with self-hosted runners article on GitHub docs.

    8. After running the steps within your shell, the shell should look like:
      GitHubWorkflowDetailedSteps_2.png
    9. To confirm if the runner is installed successfully for a Windows VM once the process is completed,  go to Start Menu >Your services> search GitHub Actions Runner. If you see the runner, it means the runner has been configured and is running.
      GitHubWorkflowDetailedSteps_3.png
  3. Complete Extra Setup for Code Pipeline Synchronization

    • Setup if using the ispw-sync GitHub Docker action:

      To use ispw-sync Docker container action, self-hosted runner must use a Linux operating system and have Docker installed. GitHub only supports Linux self-hosted runner for Docker container action.  A WorkbenchCLI Docker image is distributed from Docker hub and you do not need to install WorkbenchCLI on the self-hosted runner.

      If an Ubuntu based self-hosted runner is preferred, run the following commands. For more information, see Install Docker Engine on Ubuntu article on docker docs. 

      $ sudo apt-get remove docker docker-engine docker.io containerd runc

      $ sudo apt-get update

      $ sudo apt-get install \
          apt-transport-https \
          ca-certificates \
          curl \
          gnupg \
          lsb-release

      $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

      $ echo \
        "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
              $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

      $ sudo apt-get update

      $ sudo apt-get install docker-ce docker-ce-cli containerd.io

      As the action-runner cannot run under root, the action-runner will have the following error to trigger any docker action:

      Got permission denied while trying to connect to the Docker daemon socket

      To resolve the issue, modify the permission and restart Docker daemon:

      $ sudo usermod -aG docker ${USER}
      $ sudo rm -rf ~/.docker
      $ sudo systemctl restart docker
      $ sudo chmod 666 /var/run/docker.sock

      Important

      You can setup Docker on Windows WSL2 (Windows Subsystem for Linux, version 2). In this case, ispw-sync will work on Windows self-hosted runner too. For more information, see Microsoft online documentation about Getting started with Docker container. 

    • Setup if using the ispw-sync-local GitHub action:
      • For Windows based self-hosted runner, install the WorkbenchCLI windows distribution.
      • For Linux based self-hosted runner, install the WorkbenchCLI Linux distribution.

Tip

Remember the root path to WorkbenchCLI, and specify the CLI location in the ispw-sync-local action.

Task 2: Verifying if the Code Pipeline mainframe PLAY application is available

The Code Pipeline PLAY application is installed as a part of the Training Application delivered in the Code Pipeline SAMPLIB in the Installation Verification Process (IVP).

Perform the following steps to install the PLAY application instance based on your host connection. If you are unable to locate the PLAY application, refer Installation Verification Procedures in the Code Pipeline Installation and Configuration guide.

  1. In Workbench for Eclipse, open the Code Pipeline perspective.
  2. From the Window menu, select Open Perspective>Other. The Open Perspective dialog box appears.

  3. Select Code Pipeline and click OK. The Code Pipeline Repository Explorer view appears.

    Important

    The Code Pipeline Repository Explorer view is visible by default. If it is not visible:

    1. From the Window menu, select Show View>Other. The Show View dialog box appears.
    2. Open Code Pipeline, select the Code Pipeline Repository Explorer view, and click OK.
  4. In the Code Pipeline Repository Explorer select the following:
    • From the Stream list, select PLAY.
    • From the Application list, select PLAY.
    • From the Sub-Application list, select PLAY.
    • From the Level list, select DEV1.
    • From the Level option list, select First found in level and above.

  5. Click Apply. A filtered list of the components within the PLAY application appears.

    Tip

    If you are not logged onto a host connection where the Code Pipeline PLAY application is installed, you will be prompted when you select any list in the Code Pipeline Repository Explorer view.

Task 3: Setting up a GitHub repository with the Code Pipeline source and YAML configuration file

To create a new GitHub repository named GitPlay, perform the following steps:

  1. Go to the GitHub.
  2. To create a personal repository, select Your repositories from profile, then select New.
    GitHubWorkflowDetailedSteps_4.png
  3. Make sure Add a README file is checked, so the repository is fully initialized.
    GitHubWorkflowDetailedSteps_5.png

Tip

The repository can also be created under an organization. Refer to GitHub online help.

Task 4: Cloning the GitPlay repository from GitHub and importing as an Eclipse project

  1. In Workbench for Eclipse, open the Git perspective.
  2. Clone the GitPlay repository.
    GitHubWorkflowDetailedSteps_6.png
  3. Import the repository as standard Eclipse project once the repository is cloned successfully,
    GitHubWorkflowDetailedSteps_7.png

Task 5: Configuring Code Pipeline and importing the Code Pipeline source

  1. In Workbench for Eclipse’s Project Explorer view, right-click the GitPlay project node and select Configure>Configure to Use 

    Code Pipeline

    . The Configure 

    Code Pipeline

    and GIT Mapping wizard appears.

  2. From the Host list, select the host where the PLAY application is located, or click Configure to configure a host. We recommend using the fully qualified domain name when configuring the host.
  3. In the Runtime Configuration field, enter the runtime configuration if not using the default Code Pipelineinstance.
  4. Click Next. The next page of the wizard appears.
  5. From the Stream list, select PLAY.
  6. From the Application list, select PLAY.
  7. From the Sub-Application list, select PLAY.
  8. From the Level list, select the level used when doing a build.
  9. From the Download source from path list, choose the level from which to download source for the project.
  10. In the YAML mapping file location field, enter or browse to the project folder in which to create the YAML mapping file. By default, the location is the root of project.
  11. Click Finish. The source is downloaded. The ispwconfig.yml file is created and added to the root of the GitPlay project in the Project Explorer view. The Console view shows the activity, and the Project Explorer view shows the downloaded files.
    GitHubWorkflowDetailedSteps_8_rebranded.png

  12. To examine the contents of the ispwconfig.yml file, right-click the file and select Open.

    !!com.compuware.ispw.cli.model.IspwRoot
       ispwApplication:
       application: PLAY
      host: somehost.example.com
       pathMappings:
      - path: \CLST
       types:
      - fileExtension: clst
       ispwType: CLST
         - path: \COB
       types:
      - fileExtension: cob
       ispwType: COB
         - path: \COPY
       types:
      - fileExtension: copy
       ispwType: COPY
         - path: \JOB
       types:
      - fileExtension: job
       ispwType: JOB
          port: 12345
          runtimeConfig: TPZP
          stream: PLAY
  13. For information on the available Code Pipelineproperty settings and path mappings, see Git-to-Code-Pipeline-integration-the-Code-Pipeline-YAML-configuration-file.

Task 6: Committing and Push the GitPlay project to the Master branch in the GitPlay repository

  1. In Workbench for Eclipse, open the Git perspective.
  2. If the Git Staging view is not visible, from the Window menu, select Show View>Other. The Show View dialog box appears. Select Git>Git Staging, and click Open.
  3. If there is no repository selected in the Git Staging view, click GitHub_icon_gitplay.pngand select GitPlay to switch to the GitPlay repository.
  4. In the Unstaged Changes box, select all of the components and click GitHub_icon_GitStagingAddAll.pngto add them to the Staged Changes box.
    ISPWDetailedSteps_11.png
  5. In the Commit Message box, enter a commit message, such as Commit to master.
  6. Click Commit and Push.
  7. If the Push Branch master dialog box appears, in the URI field, specify the URL of the GitPlay repository. The repository URL can be found in the Git server Repository Settings of the GitPlay repository under the Name field. Complete the rest of the fields if not automatically populated. Click Next to select the remote repository. The repository name should contain the branch name. Click Next again to confirm the expected push result, then click Finish.
    ISPWDetailedSteps_12.png
  8. In the Push Results dialog, click Close.
    ISPWDetailedSteps_13.png

Task 7: Creating a GitHub workflow

  1. In Project Explorer View, select Filters and Customization..., then uncheck .*resources to show hidden files and folders for the GitPlay project.
    GitHubWorkflowDetailedSteps_10_rebranded.jpg

    GitHubWorkflowDetailedSteps_11_rebranded.JPG
  2. Expand GitPlay project, create folder .github/workflows.
  3. Create a GitHub workflow file, for example, ispw-sync-build-deploy.yml (Download a template), the workflow is broken up into the following stages:
    GitHubWorkflowDetailedSteps_12_rebranded.png
    • Checkout - checks out the source code.
    • Synchronize changeset to Code Pipeline - uses GitHub action ispw-sync to perform GitHub to Code Pipeline synchronization.
    • Build Code Pipeline tasks - uses GitHub action ispw-build to perform a build.

    • Deploy Code Pipeline tasks - uses gitHub action ispw-deploy to perform a deploy.
      GitHubWorkflowDetailedSteps_13.png

      GitHubWorkflowDetailedSteps_14.png

      Tip

      To better understand how variable substitutions work in the workflow file, see Environment variables.

  4. Or if you prefer to use GitHub action ispw-sync-local, you may specify the sync step as below, the workflow is broken up into the following stages (Download a template - ispw-sync-local-generate-deploy.yml):

    • Checkout - checks out the source code.
    • Synchronize changeset to Code Pipeline - uses GitHub action ispw-sync-local to perform GitHub to Code Pipeline synchronization.
    • Generate Code Pipeline tasks - uses GitHub action ispw-generate to perform a generate.
    • Deploy Code Pipeline tasks - uses GitHub action ispw-deploy to perform a deploy.
      GitHubWorkflowDetailedSteps_15.png

      GitHubWorkflowDetailedSteps_16.png
  5. Create secrets to hide usernames, passwords, or tokens that will be used in the workflows.
    1. Navigate to your GitHub repository where your workflows are stored.
    2. Click on the Settings tab that is located on the top left just under the repository path.
    3. Click on the Secrets section.
    4. From the following screen, you can either create organization level secrets (secrets that can be used across all repositories within your organization group) or repository level secrets.

    5. Some secrets examples you may create based on the ispw-sync-build-deploy.yml (Download a template) are as follows:

      • secrets.TSOUSER
      • secrets.TSOPASS
      • secrets.GITUID
      • secrets.GITHUB_TOKEN
      • secrets.CES_TOKEN

Task 8: Making a change and building

To make a change to Cobol component TPROG15.cob, perform the following steps: 

  1. In Workbench for Eclipse’s Project Explorer view, expand GitPlay>COB.
  2. Right-click TPROG15.cob and select Open. The source opens in the editor.
  3. On line 8, append the TEST to ...PROGRAM.
    GitHubWorkflowDetailedSteps_17_rebranded.png
  4. From the File menu, select Save.
  5. From the File menu, select Close.

Task 9: Performing the build action to verify the source generates successfully along with any impacted components

  1. In Workbench for Eclipse’s Project Explorer view, Right click GitPlay project, select Properties, add 

    Code Pipeline

     Nature, then click Apply and Close.
    GitHubWorkflowDetailedSteps_18_rebranded.png
    In Workbench for Eclipse’s Project Explorer view, right-click the GitPlay project node and select Properties>

    Code Pipeline

    . The 

    Code Pipeline

     page of the Properties dialog box appears.

  2. In the YAML mapping file location field, enter or browse to the YAML mapping file.
  3. In the Level field, enter DEV1 as the life cycle level for the project that will be used to load the file into Code Pipeline at that level. The level value is required to perform any Code Pipeline action.

  4. Alternatively, in the Assignment description field, enter the assignment name to be used for the generate/build process; click Insert Variable, if desired, to choose variables to build the assignment name.

    Important

    If no valid value exists for the chosen variable, the Description field in the Containers view will treat the variable as empty, Descriptions longer than 50 characters will be truncated. The default assignment description is ${user} ${branch_name}.

  5. Click Apply and Close.

  6. In the Project Explorer view’s COB folder, right-click TPROG15.cob and select Code Pipeline>Build. The Console view shows the progress of the build.

    Important

    If an Code Pipeline pop-up appears, click YES to continue.

    GitHubWorkflowDetailedSteps_19.png

  7. In the Console view, note the assignment where TPROG15 was loaded. In the example above, TPROG15 was loaded into assignment PLAY004840 at the DEV1 level.
  8. Delete the task from container PLAY004840 and close the container.

Task 10: Submitting the change to GitHub

To commit and push the changes to GitHub and trigger the GitHub Workflow, perform the following:

  1. To Commit and Push TPROG15, in the Unstaged Changes box, select TPROG15 and click GitHub_icon_GitStagingAdd.pngto add it to the Staged Changes box.
    GitHubWorkflowDetailedSteps_20.png
  2. In the Commit Message box, enter a commit message, such as TPROG15 commit.
  3. The Push Results dialog box appears.
  4. Click Close.

Task 11: Determining if synchronization process completed successfully

Once the TPROG15 is pushed to GitHub, the workflow on GitHub is immediately triggered (for demo purpose, ispw-sync-local-generate-deploy workflow is disabled, workflow ispw-sync-build-deploy is enabled).

  1. Visit GitHub repository GitPlay | Actions.
    GitHubWorkflowDetailedSteps_21.png

  2. Once the job completes, click TPROG15 commit to view workflow run detail.
    GitHubWorkflowDetailedSteps_22.png
    The following is an example of what the sync step will appear after a successful Git to Code Pipeline synchronization:

    Finished synchronizing changes from Git to ISPW assignment PLAY005107 at level DEV1

     COB/TPROG15.cob                                     -----  Success                               
    1 total changes detected during synchronization: Success 1, Failure 0, Skipped 0

    The following is an example of what the build step will appear after a successful build:

    Build parameters are being retrieved from the build_automatically input.
    Starting the build process for task 7E5655E16646

    ISPW: Set S000248884 - The generate request completed successfully for TPROG15 in PLAY005107. Job ID and name: J0156513 ISPWTPZG
    ISPW: Set S000248884 - The build process completed successfully

    The build request completed successfully.

    The following is an example of what the deploy step will appear after a successful build:

    Deploy parameters are being retrieved from the deploy_automatically input.

    Starting to submit the deploy request for task 7E5655E16646
    ISPW: received set ID: S000248885

    ISPW: received URL: http://10.100.12.250:48226/ispw/cw09-47623/sets/S000248885

    The deploy request has been submitted.

    Tip

    Git commit information can be viewed in Workbench within the Code Pipeline Assignment view and the Code Pipeline Release view.

Task 12: Verifying the updates occurred to the Mainframe

  1. In Workbench for Eclipse’s 

    Code Pipeline

    Containers view, find the assignment where TPROG15 was loaded and double-click the assignment. The 

    Code Pipeline

    Tasks view appears.

  2. Double-click TPROG15. The source is opened in the editor.

    Important

    If a copybook download message appears, click No to not download copybooks.

  3. Verify that line 8 shows TEST appended.
    image-2023-3-20_7-43-31-1.png



 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*