Setting up BMC Deployment Engine

The steps to set up BMC Deployment Engine are shown in the following figure:

Before you begin

Ensure that your environment meets the following requirements:

• Virtual machine—A fresh and dedicated physical server or virtual machine with the following configuration for the Jenkins server:
  

  Component	Node	vCPU	Operating System	RAM (GB)	Disk (GB)
  Jenkins server	1	2	CentOS 7.x and CentOS Core 8.x are certified. RHEL 7.x and 8.x are certified. Equivalent releases of Fedora and Oracle Linux are supported.	Minimum 4 GB	40

• Kubernetes cluster—Kubernetes cluster is accessible from the Deployment Engine server.
  For Kubernetes cluster requirements, see System Requirements.
• Ansible 2.9—is configured, and working fine.
  You must install Ansible 2.9 for setting up BMC Deployment Engine. 

• Yum utility—is installed, configured, and working fine.
  Make sure that the yum client is installed and is accessible, and you can install certified repos stable version of applications. 
  Run the following commands to update your system libraries and packages to the latest available version.

  yum update -y
  yum upgrade -y
  yum clean all

• Unzip utility—is installed.
  Use the following command to install the unzip utility:

  sudo yum install unzip

• Perl—Perl (version 5.x +) is installed and is accessible from shell prompt.
  Use the following command to set up Perl:

  sudo yum install perl

• Perl-Data-Dumper package—Set up Perl-Data-Dumper package by using the following commands:
  

  sudo yum makecache
  sudo yum -y install perl-Data-Dumper

  • Make sure that /etc/resolv.conf has proper DNS entries.

  • From any remote machine run the following commands:
    

    nslookup <jenkins-server-name> 
    nslookup <jenkins-server-ip>

    both the commands should resolve via DNS from all computers where clusters are setup.

  • from /etc/hosts make sure to have only relevant entries in this file. If an entry is not correct, delete the entry.
  • Remove the ivp6 entry for the localhost and ensure that the hostname resolves to the correct DNS IP address.
  • Run the yum repolist command to view all accessible and OS matching repositories.  The yum commands should be able to install the required repository software. 

•  Download the BMC_Helix_Innovation_Suite_And_Service_Management_Apps_Version_21.30.06.zip file from EPD to the Deployment Engine server.
  This file contains the following files:
  • BMC_Remedy_Deployment_Manager_Configuration_Release_21306007.zip - This file contains the repositories that go in to git.
  • BMC_Remedy_Deployment_Engine_Setup_21.30.06.zip
• Download the BMC_Remedy_Deployment_Engine_Setup_21.30.06_Hotfix.zip file. This file contains the BMC Deployment Engine automation script.
• Download the BMC_Helix_Innovation_Suite_Performance_Hotfix_Version_21306.zip file from EPD.
  This file contains the ITSM_REPO.zip file that includes the latest templates for the Compact, Small, Medium, and Large deployment sizes.

For details, see Downloading-the-installation-files.

Back to top

Creating and configuring users

BMC Deployment Engine uses the following two users:

• git—Owner of git repositories where deployment code is present. This user requires sudo access.
• jenkins—Jenkins server runs as jenkins user. This is a non-interactive user.

You must create and configure these users.

To add users

Perform the following steps to add the users:

To add the git user to the jenkins group and the jenkins user to the git group

Perform the following steps to add the users:

To provide sudo access to the git user

Run the following command to provide sudo access to the git user:

To configure passwordless sudo access for the git user

Perform the following steps to configure passwordless sudo access to the git user:

1. Run the following command:

   sudo visudo

2. Uncomment the following line:

   %wheel ALL=(ALL) NOPASSWD: ALL

3. Comment the other lines that start with %wheel. 
   For example:

    %wheel ALL=(ALL) ALL

4. Save the changes.

   Back to top

Running the BMC Deployment Engine automation script

The BMC Deployment Engine automation script installs the following software:

• Python
• Git
• kubectl
• unzip
• Helm
• Java
• additional packages, such as MS SQL tools, unixODBC, xmlstarlet, dos2unix, jq

The automation script also installs Jenkins locally on a node (In earlier versions of BMC Helix IT Service Management Deployment, you had to manually set up Jenkins on a local node).

Perform the following steps to run the BMC Deployment Engine automation script:

1. Login as git user:

   ssh git@<hostname>

   or switch to the git user by using the following command:

   su - git
2. Copy the BMC_Helix_Innovation_Suite_And_Service_Management_Apps_Version_21.30.06.zip file downloaded from EPD and extract the files to the git user home directory or any other directory owned by the git user.
   • BMC_Remedy_Deployment_Manager_Configuration_Release_21306007.zip

   • BMC_Remedy_Deployment_Engine_Setup_21.30.06.zip

     Important

     Do not use the BMC_Remedy_Deployment_Engine_Setup_21.30.06.zip file.
     Use the BMC_Remedy_Deployment_Engine_Setup_21.30.06_Hotfix.zip file mentioned in Step 3 because this file contains updates to RHEL 8 and CentOS 8. 

3. Copy the BMC_Remedy_Deployment_Engine_Setup_21.30.06_Hotfix.zip file downloaded from EPD and extract the files to the git user home directory or any other directory owned by the git user.

4. Change the ownership of these files to git user by using the following command: 

   sudo chown -R git:git <filename>
5. Unzip BMC_Remedy_Deployment_Engine_Setup_21.30.06_Hotfix.zip file. 
6. Change the directory to DE1.0 to navigate to exact location.
7. Update build.properties and customize the following parameters according to your requirement:
   • ITSM_REPO_GIT_ZIP=</path/to/BMC_Remedy_Deployment_Manager_Configuration_Release_21.30.xx.zip>
     
   • GIT_REPO_USER=git
   • GIT_USER_HOME_DIR=~git
   • JENKINS_USER=jenkins
   • JENKINS_INSTALL_DIR=/var/lib/jenkins
   • JENKINS_HOSTNAME=<fully-qualified-hostname-or-ip-address-where-DE-setup-is-planned>
   • ANSIBLE_NODE_ROOT_DIR=~/ansible_node_root_dir
   • JENKINS_NODE_ROOT_DIR=~/jenkins_node_root_dir
     (This is the path that the Jenkins_node needs for various operations that are internal to Jenkins)
   • GIT_REPO_DIR=~/git_repo/
     ( This is the location where the entire Deployment Engine git code is copied by the installer for setting up the pipelines.)
   • JENKINS_CONFIG_FILES_DIR=~/Jenkins_Config_Files
     (The Jenkins job pipeline uses this location to store its pipeline jobs.)

8. After updating the build.properties file, run the BMC Deployment Engine automation script to setup the Jenkins job pipeline framework.
   The log file is generated in the Home directory and it can be used for debugging setup related issues.

   $ sudo perl setup-Helix-ITSM-onPrem.pl  2>&1 | tee ~/BMC-HELIX-DE-AUTO.log.$$

   Important

   The build.properties file of BMC Deployment Engine automation script has a KUBERNETES_VERSION parameter which only accepts Kubernetes versions 1.21 and earlier.

   We recommend you to set the value KUBERNETES_VERSION parameter as 1.21. This step will setup the kubectl version as 1.21.

   After the script completes running, update kubectl version to 1.23 or 1.24 as per the cluster version.

Back to top

Performing post-installation configurations

To install plugins

1. Login to Jenkins server by using the following URL:
   http://<Jenkins server host name>:<Jenkins port>

   The default password is available in $JENKINS_INSTALL_DIR/secrets/initialAdminPasswordFor example: /var/lib/jenkins/secrets/initialAdminPassword

2. On the Jenkins User Interface, select Install Suggested Plugins.
3. In the Admin User Creation wizard, provide a preferred username and password and run the wizard.

4. Verify that ssh works fine with passwordless login from root user to git@<jenkins_server_name> and git user to git@<jenkins_server_name> by using the following command:

   sudo ssh git@<jenkins_server_name>
   ssh git@<jenkins_server_name>
5.  Add the kubeconfig file on the Jenkins server.
   1. Download the kubeconfig.yaml file to the $HOME directory.

   2. Copy the kubeconfig.yaml file to $HOME/.kube/config.

      cp <kubeconfig complete file path> $HOME/.kube/config

To update the Jenkins credentials

1. Login to the Jenkins server by using the following URL:
   http://<Jenkins server host name>:<Jenkins port>/credentials
2. Add the credentials in the ID: kubeconfig file.
   1. Select kubeconfig and click the name of the kubeconfig credential.
      
      
   2. Click Update.
   3. Select the Replace checkbox.
   4. Click Choose File and select the kubeconfig file
   5. Click Save.
3. Add the credentials in the ID: github account.
   1. Select github and click the name of the github credential.
   2. Click Update.
   3. Click Change Password and enter the git user password.
   4. Click Save.
4. Add the credentials in the ID: ansible_host account.
   1. Select ansible_host and click the name of the ansible_host credential
   2. Click Update.
   3. Click Change Password and enter the git user password.
   4. Click Save.
5. Add the credentials in the ID: ansible file.
   1. Select ansible and click the name of the ansible credential.
   2. Click Update.
   3. Click Change Password and enter the git user password. 
   4. Click Save.
6. Add credentials in the ID: git account.
   1. Select git and click the name of the git credential.
   2. Click Update.
   3. Click Change Password and enter the git user password.
   4. Click Save.
7. (For cloud environments) Update git user private key credentials
   1. Navigate to http://<Jenkins server host name>:<Jenkins port>/credentials/store/system/domain/_/
   2. Click Add Credentials.  

   3. Enter the following details:

      Kind	SSH Username with private key
      ID	git_pk
      Username	git
      Private Key	Click Enter. Paste the contents of git user private key value (/home/git/.ssh/id_rsa)

   4. Save the details.

Depending on your environment, you can choose to update the node configuration in a cloud or non cloud environment.

To update the node configuration in a cloud environment

1. Navigate to http://<Jenkins server host name>:<Jenkins port>/computer.
2. Perform the following steps for both nodes, the one with the actual hostname and another with the git hostname.
   1. Click the node name. 
   2. Click Configure.
   3. Under Credentials, select the git user ssh key credential and click Save.
   4. Click Launch Agent.

To update the node configuration in a non cloud environment

1. Navigate to http://<Jenkins server host name>:<Jenkins port>/computer.
2. Perform the following steps for both nodes, the one with actual hostname and another with git-hostname.
   1. Click the node name.

   2. Click Launch Agent.

      If the status of the node is already 'online', then skip Step b.

To add the Jenkins libraries

Perform the following steps to add the pipeline-framework library:

1. On the Jenkins home page, click Manage Jenkins.
2. Select Configure System.
3. In Global Pipeline Libraries, add the pipeline-framework library as shown in the following figure:
4. In the Project Repository field, specify the complete path of pipeline-framework.git according to the environment.
   <GIT_REPO_DIR> value can be fetched from build.properties file.
   
5. Make sure you provide exact location of pipeline-framework.git according to the environment.
6. Add JENKINS-27413-workaround-library as shown in the following figure:
7. In the Project Repository field, specify the complete path of JENKINS-27413-workaround-library.git according to the environment.
8. Select Load implicitly.
   
9. Save the changes.

To add sizing templates to the ITSM_REPO Git repository

1. Log in to the Jenkins server with the Git repository owner user, such as git.
2. Navigate to the Git repositories directory, such as /home/git/Git_Repo
3. Copy the ITSM_REPO.zip file that you downloaded from EPD.
   The BMC_Helix_Innovation_Suite_Performance_Hotfix_Version_21306.zip file contains the ITSM_REPO.zip file.

1. Rename the existing ITSM_REPO by using the following command:

   mv ITSM_REPO ITSM_REPO_old

2. Extract the ITSM_REPO.zip file by using the following command:

   unzip ITSM_REPO.zip

   After you unzip the file, the Git repository is replaced with the new deployment size templates.
   

3. Apply owner user permissions to the ITSM_REPO repository by using the following command:

   chown -R git:git ITSM_REPO

Running the deployment pipelines in dry-run mode

Dry-run is a mandatory step to update the pipeline configuration for any changes to the BMC Helix Innovation Suite and Service Management Installer.

To run the deployment pipelines in dry-run mode

1. Navigate to Jenkins Dashboards to view all the pipelines required for deployment.
2. Select each pipeline and click Build with Parameters.

3. Click Build.
   The build job status fails, which is expected.

   • The agent-add-pipeline and HELIX_DR pipelines do not require a dry-run.
   • For HELIX_ONPREM_DEPLOYMENT pipeline, enter the value of the AGENT parameter.
     The AGENT parameter value is the Jenkins agent to run the pipeline.
     Do not select any pipelines in the PRODUCT_DEPLOY section.
   • For the other pipelines, run Build with Parameters with default values present in the pipeline.

Back to top

Troubleshooting

Refer to the troubleshooting information if you encounter any of the following issues while setting up BMC Deployment Engine.

Issue symptom

During .pl script execution, you might receive the following message:
This system is not registered with an entitlement server. You can use subscription-manager to register.
         Log: Query :  Please, suggest should we disable subscription manager and proceed with setting up prereq for ITSM onprem deployment,  Y(yes) or N(no) ? :

Resolution

• If the Jenkins server is configured with a local repository from where the required packages can be downloaded, enter ‘yes’ and proceed.
• If the Jenkins server is registered with any subscription manager, enter ‘no’ and proceed. In this case, the required packages are retrieved online through the subscription manager.

Issue symptom

The installation script fails or aborts for some reason.

Resolution

1. Review the logs to check for errors.
2. Stop Jenkins by using the systemctl command.
3. Check if port 8080 is free by using netstat -anp | grep 8080 command and then rerun automation script command.
4. Make sure to save log files.

Issue symptom

HELIX_ONPREM_DEPLOYMENT pipeline fails with the following scripts not permitted error.

Resolution

1. Right click the error message link and open it in a new tab.
2. Click Approve.
   
3. Click the latest failed job on the HELIX_ONPREM_DEPLOYMENT pipeline tab.
   

1. Click Rebuild.
   
2. Enter the required parameters and click Rebuild.
   
   
3. You might get other scripts not permitted error. Repeat the same steps to approve the script execution and rebuild HELIX_ONPREM_DEPLOYMENT jobs.

For other issues, see Troubleshooting-deployment-pipeline-failure-issues.

For additional questions about BMC Deployment Engine, see FAQ.

Where to go from here