BMC AMI DevX DevOps Pipeline
Getting Started
Summary
The purpose of this Test Drive is to highlight the integration between BMC AMI DevX Code Pipeline, BMC AMI DevX Workbench for Eclipse, and Jenkins, enabling automated deployments through DevOps Continuous Integration. Learn about the modern Agile/DevOps features of DevX Code Pipeline as well as how developers can use its integration with Jenkins to:
- Automate the download of source and its associated unit tests for testing using BMC Jenkins plugins
- Drive Continuous Integration processes with webhooks and Jenkins
- Access and execute BMC AMI DevX Code Debug assets for automated unit testing
- Use test metrics in SonarQube Quality Gates and automatically regress failed code in DevX Code Pipeline to enable remediation
- Automate orchestration of release deployment using a Release Pipeline in Jenkins and DevX Code Pipeline APIs
The following diagram illustrates a subset of the DevOps toolchain which this pipeline script accomplishes.
DevOps Toolchain
Use Case
You will check out 3 programs to a DevX Code Pipeline assignment container, make a slight change to one of the subroutines, and regenerate (recompile) all the modules. Once this is complete, the user will create a DevX Code Pipeline release container and transfer the 3 programs from the assignment container to this release container to be promoted to the next stage.
Next, the user promotes the programs in the release container to the QA3 level. This action will trigger the Jenkins pipeline job behind the scenes via BMC webhooks to download the program source and associated unit tests, execute the unit tests including code coverage data, and feed the execution results to SonarQube for analysis.
If the "Quality Gate" passes, the programs are promoted to QA3 level and a Release Pipeline is automatically created in Jenkins. The user, acting as release manager, can then orchestrate the release to Production. This orchestration, using the DevX Code Pipeline APIs, automatically promotes the programs to DevX Code Pipeline levels defined in the DevX Code Pipeline workflow, all the way to PRD level.
BMC AMI DevX Code Pipeline
BMC AMI DevX Code Pipeline enables developers to quickly and safely build, test and deploy mainframe code. This Test Drive script will take you through changing the source code, generating the code, and promoting the code which will trigger program analysis in SonarQube.
This Test Drive script will take you through the following activities:
- Checking out the COBOL programs
- Changing the code
- Generating the code (aka Compiles)
- Promoting the code into Test and Production environments
- Feeding data into SonarQube from the COBOL Source, Unit tests, and Code Coverage from BMC AMI DevX Code Debug
- Orchestrating the release in production environment using Jenkins Release Pipeline
Instructions:
- This guide contains many screenshots to provide a visual reference
- Every action you must take is in "Do This..." info box
- Please note each place that you must enter your own specific ID or number
- You must complete each step before proceeding to the next to successfully follow the guide
To get started in the DevX Workbench
During this Test Drive you will encounter terms specific to DevX Code Pipeline. Here is a glossary for your convenience.
Term | Description |
Application | Business Area |
Stream | Development Life-Cycle used by one or more Applications |
Life-Cycle | Transitional Code States (DEV, QA, PROD) in the Development Life-Cycle |
Level | Instance of a Code State (DEV, QA, PROD) |
Component | SCM managed object such as a COBOL source member |
Component Version | An immutable version of a Component |
Task | An instance of a unit of work. For example, a change to a Component |
Part | A generated object such as a load module, or a Listing or a DBRM |
Warehouse | Storage location for inactive historical or overlaid Component Versions (source and parts) |
Assignment Container | A package of one or more, usually related, Tasks. A development container that is used to organize and manage development work |
Release Container | A package of Tasks from one or more Assignment Containers. An operational container that is used to organize and manage operational releases |
Set Container | A package of Tasks at an instance in time. A special DevX Code Pipeline container that is used internally to organize and manage work requests |
Add Task | Make a statement of intent, often because of some impact analysis |
Checkout Task | Copy source to development area |
Edit Task | Make a change to the source in the development area |
Generate Task | Execute Compile and Link processes |
Promote Task | Move Task forward to the next Level in the Life-Cycle |
Regress Task | Move Task backwards to the previous Level in the Life-Cycle |
Delete Task | Remove a Task from the development area |
Fallback Task | Restore previous version of Task |
Deploy Task | Implement Task in one or more Run Time environments |
Substitute your values in the screenshots
In this Test Drive the screen shots provided have used specific values that may differ from your assigned values. These were provided in your email notice.
Variable | Your Assigned Values |
Application | Txxx* |
SubAppl | Txxx* |
Assignment Prefix | Txxx* |
* For the value of xxx, substitute the last three digits of your Test Drive username.
- Open the Host Explorer perspective,
- Click on the arrow next to Hosts,
- Right-click on "TestDrive" and select Login,
- Logon with the user credentials that were provided to you for Test Drive and click OK,
- Select "Code Pipeline" from the BMC menu.
If at any time during the execution of this script the BMC AMI Common Enterprise Services (CES) Login popup is shown, enter your test drive ID and password under User ID and Password, check the Save credentials box and then depress the ENTER key or click OK.
Code Changes
Your development Task in this Test Drive is to add 3 COBOL programs—CWXTCOB, CWXTDATE, and CWXTSUBC—to an assignment container, make a change to either COBOL program CWXTDATE or CWXTSUBC, and generate and promote all 3 to the QA3 Level. To start this process, you will find and checkout the 3 COBOL programs into an DevX Code Pipeline Assignment from which you can perform all this work.
In the "Repository Explorer"
- Select the Stream CWEZ,
- Select the Application to your assigned application value (Txxx, where xxx is your unique number),
(Leave SubAppl to the default value. It will default to the name of the Application)
- Select the Level to "PRD" and ensure the Level option is set to "Selected level only",
- Enter "CWXT*" in the "Name" field to filter the relevant components,
- Filter the view even further by setting the Type to COB, this will allow you to focus only on the COB components available in your stream/application at the PRD level. Click Apply.
The Repository View lists the COB Components for your application.
Select CWXTCOB, CWXTDATE, and CWXTSUBC, then right-click and select "Add to Assignment with Check Out…"
From the Add Task dialog box, select NEW to the right of Assignment. This will allow you to create a new Assignment Container to put your COBOL programs into.
For Stream, use CWEZ. For Application, SubAppl and Assignment prefix, use your assigned values (see "Substitute your values in the screenshots"). Select a Default Path of DEV3 and provide a description for your Assignment.
Click OK.
A message will be displayed providing you with your new Assignment name.
Make note of the Assignment name. Click OK.
Your new Assignment has been created and is ready to have Tasks added to it. An Assignment Container is the only Container Type in DevX Code Pipeline that Tasks can be added to. The Assignment Container is where a developer will spend much of their time.
Select your new assignment name from the drop-down list. Select DEV3 for the Path.
Select "Use production versions" option.
Click Finish.
Messages will be displayed to inform you that the tasks have been added to your Assignment and checked out to the DEV3 level. You have added your first Tasks to your Assignment.
Click OK.
The "Assignment" view will be open. This will list the tasks added to the container
Every operation that needs to be performed on these Tasks and other Components which are impacted by changing these Tasks (edit, impact analysis, compile, promote, deployment, etc.) can be performed from this Assignment view.
Note that the COBOL Tasks in the Assignment are checked out and sitting in the DEV3 level. The checkout date/time and userid is reflected in the Tasks information.
At checkout, a destination level for the checkout was chosen – DEV3.
To see the DevX Code Pipeline Life-Cycle path structure that was constructed for the Application, Click once on the Task and then Click on the View Life-cycle icon.
A picture of the Lifecycle is presented. You will notice the levels containing Versions of the highlighted Tasks are colored indicating the existence of a version of that Component at those levels.
From this picture it is very easy to visually see where all versions exist. By choosing the DEV3 Level, you have defined the Path to Production DEV3-QA3-STG-PRD. Other versions of these Components may also exist at other levels and may be passing through the other three paths – FIX, DEV1, or DEV2.
Four paths were created for the application – one for emergencies starting at level FIX and three for development starting at DEV1, DEV2, and DEV3. The application level structure is customizable when defining the applications to DevX Code Pipeline. Any number of paths can be defined with a minimum of three Levels.
Now that you have Versions of the COBOL programs the next step would be to make your changes.
To do this right-click on CWXTDATE and select "Edit" or simply double-click on it.
The version's source will be presented for editing.
Edit a comment line (ensure you do not "break" the COBOL program).
Once you have made a change, close the Editor view. Click Yes to save the changes.
Note the Operation, date/time and User ID fields have been updated to reflect the change in the Task status
Generate (AKA Compile)
The COBOL programs will be generated/compiled, and your Assignment Task List will be updated to reflect the new status.
- The Operation will be Generate
- The User ID, Date/Time will be updated
- The Message will be updated to reflect the successful completion of the generate
Select all 3 COBOL programs and right click and select generate. The programs will be compiled. There are some deliberate errors in these programs so that you can view them later in SonarQube.
Upon successful Generate, the Operation field in the Tasks view will say "Generate"
Promotion
At this point you have
- Created an Assignment
- Checked out 3 COBOL programs to the Assignment
- Edited 1 of the COBOL programs
- Generated all 3 COBOL programs
Now you are ready to promote your changes to the QA3 level. Once everything has been generated successfully you will now promote the 3 components together to the next level, QA3.
Create a new Release container
- Click on Containers view
- Click on "New Release" button on the far right of the "containers" view
In the "Add New Release" popup dialog, enter the values for the following fields:
- Release ID <enter your Test Drive username + "R". Ex. CWEZXXXR0>
- Description <enter a description>
- Click OK.
You will see a dialog box indicating that your release container was successfully created. Click OK.
The newly created Release container shows up in the Containers list:
Transfer Task from Assignment to Release Container
You need to transfer the 3 tasks from Assignment container to the newly created Release container.
Open the Assignment container (created earlier), select the 3 tasks, right-click, select "Transfer Task" option
In the "Transfer Task" popup window, select the newly created Release container. Click OK.
Click on the Containers view, then click on the link for your release container. Now, all 3 tasks should show up in the Release container.
Select all 3 components, right-click and select promote.
This action will create a Set Container. Sets are a special category of containers within DevX Code Pipeline that are used to conduct operations such as promotion and deploy. They are temporary and created by DevX Code Pipeline as needed for the work. This allows you to promote subsets of Tasks within an Assignment container without the need to act on all the Tasks at once.
Once you click Promote, the Task you selected was placed in a SET container for the promotion.
The DevX Code Pipeline Set processor will:
- Perform the promotions of all the source to QA3
- Cleanup the DEV3 level (source and parts if applicable)
- Deploy to runtime environments (if required)
Please click refresh (repeatedly) to watch for the promotion of the modules.
At this time, a popup dialog appears at the bottom right indicating successful promotion.
Upon successful promotion, a pre-configured BMC AMI Common Enterprise Services (CES) webhook will trigger a Jenkins pipeline which will download the COBOL source from DevX Code Pipeline, retrieve the BMC AMI DevX Code Debug project from the Git repository, execute the unit tests with code coverage options, and send the source code, unit test results along with code coverage data to SonarQube for analysis. The SonarQube analysis will show where any errors may still exist in the code you are promoting as well as unit test and code coverage results.
As part of the DevX Code Pipeline download, a JSON file is created that contains the names of the programs that were downloaded.
{
"version": "1.0.0",
"programs": [
{
"version": "1.0.0",
"programName": "CWXTCOB",
"programLanguage": "COB",
"isImpact": false,
"application": "T080",
"stream": "CWEZ",
"level": "QA3"
},
{
"version": "1.0.0",
"programName": "CWXTDATE",
"programLanguage": "COB",
"isImpact": false,
"application": "T080",
"stream": "CWEZ",
"level": "QA3"
},
{
"version": "1.0.0",
"programName": "CWXTSUBC",
"programLanguage": "COB",
"isImpact": false,
"application": "T080",
"stream": "CWEZ",
"level": "QA3"
}
]
}
The BMC AMI DevX Code Debug plugin uses this list of program names in this JSON file to make sure that we ONLY execute test cases for these applications. As you can see in the screen shot below, our Git repository contains test cases for programs in addition to those listed in the JSON file:
The Jenkins pipeline, as a last step, also creates a Release pipeline in Jenkins for a Release Coordinator/Manager to orchestrate the release all the way to Production level in DevX Code Pipeline.
This is all performed as a background process. When you see the modules promoted to QA3, you may continue to the next step.
To watch the progression of your Jenkins pipeline build, do the following:
- In DevX Workbench, click on "Run → External Tools → Chrome" in the menu bar to launch Chrome browser externally. Chrome Opens with multiple tabs.
- Click on "Jenkins" browser tab.
The screenshot shows the Jenkins pipeline Stage View.
After the pipeline has completed, you can view the Console Output. A section of that output is shown below to highlight that BMC AMI DevX Code Debug used the JSON file to determine which test cases needed to be executed, and it also lists all of the test scenarios that it found from the repository that was retrieved from Git:
As the Task in the SET is being processed you can monitor the progress by Clicking the "refresh" button.
In the screen shot you can see the promotion has finished – the Tasks have been promoted to QA3 (see the Level value and the Operation in the Task List as well as the highlighted Level in the picture). Note the Operation of the programs reflects a Generate as the last operation. The SET processor performed a promotion and then a generation as DevX Code Pipeline recognized a generate was required for COB types at the QA3 level based on the configuration of this application.
At this point, you have successfully promoted your application to QA3!
Review DevOps Pipeline Results
Reviewing the results in SonarQube
In DevX Workbench, click on "Run → External Tools → Chrome" in the menu bar to launch Chrome browser externally (if not opened already). Chrome Opens with multiple tabs.
Click on "SonarQube" browser tab.
When the SonarQube home page is displayed, enter your Test Drive userid in the "Search for projects…" field on top-right corner, then press enter.
Click on the project name from the results.
This provides a simple and automated way to ensure that your code is being tested and analyzed for errors before it is promoted to production.
Orchestrating the release from Test to Prod (CD)
Let us assume all the testing at the QA3 level has been successfully completed and you are ready to promote to the STG level and, eventually, to the PRD level.
The DevOps Pipeline job has also created a new release (CD) pipeline in Jenkins to promote the code from QA3 level onto STG and, eventually, to PRD level in DevX Code Pipeline.
At this point, you can switch back to DevX Workbench to monitor the DevX Code Pipeline lifecycle view by clicking on "Refresh" button (circled). The Release pipeline already promoted the code to the pre-prod (STG) level using the REST API and also performed User Acceptance tests at pre-prod.
To switch back and forth between DevX Workbench and Chrome, use the "switch windows" button in the AppStream within your browser tab. This will show both applications side-by-side. You can click on them to bring them into focus.
Switch back to DevX Workbench to monitor the DevX Code Pipeline lifecycle view by clicking Refresh button (circled).
Notice that there is now a "check" 
icon in the STG level box. It indicates that user needs to take further action to promote code from this level.
Open the DevX Code Pipeline Life Cycle View.
Right-click on the icon and select "Approve" option to approve the request.
e
Once again, click refresh button – 
. Notice that the code is now promoted all the way up to PRD.
Notice that there is now a "check" icon in the PRD level box. It indicates that user needs to take further action to deploy code from this level.
Finally, to complete orchestration of Production Environment, Click "Approve" in the PRD level box which will Deploy the runtime binaries to the production runtime environment using the REST API
Click on "Refresh" 
on the DevX Code Pipeline Lifecycle view in DevX Workbench to see that the tasks have been promoted and deployed at PRD level
You can see in the updated screen that
- SET processing has begun
- The COBOL programs have been processed for a Promotion and are now at the PRD level
The Promotion and deployment to the PRD level has completed. The source and parts for the COBOL programs were moved to the PRD level Life-Cycle libraries and the STG level libraries were cleaned up as appropriate.
Cleanup
As a Developer you are now finished with your Assignment, so it can be closed. Assignments must be closed manually and are usually closed to declutter the DevX Code Pipeline Container List View.
To Close the Assignment first Close the "Tasks" View by clicking on the "Close Window" button.
To Close your Assignment, Right Click on your Assignment and select "Close".
A Close confirmation panel is presented.
Click OK to complete the Close.
Congratulations! This completes the BMC AMI DevX DevOps pipeline Test Drive script.
Your job was to change a COBOL program. As a part of this exercise, you have been able to use BMC AMI DevX Workbench for Eclipse and BMC AMI DevX Code Pipeline to:
- Create an Assignment and Release container
- Add and check out 3 COBOL programs
- Change one program using DevX Workbench Editor
- Promote the COBOL programs from DEV3 to QA3
- Trigger a DevOps Pipeline script in Jenkins (via CES Webhook) which does the following:
- Download and feed into SonarQube, COBOL Source, Unit tests and Code Coverage results from DevX Code Debug
- View and analyze SonarQube results
- Use a Jenkins Release Pipeline to Orchestrate a release in DevX Code Pipeline to Production
- Close your Assignment to complete the change cycle
Next Steps
Should you wish to restart this Test Drive script, follow the restart instructions at the beginning of this script.