This documentation supports an earlier version of BMC Helix IT Service Management on-premises deployment. To view the documentation for the latest version, select 23.3.04 from the Product version picker.


Setting up BMC Deployment Engine on a server with internet access

Set up BMC Deployment Engine to deploy BMC Helix Service Management applications.

BMC Deployment Engine is set up on a fresh single virtual machine.

BMC Deployment Engine consists of the following tools:

  • Jenkins—Jenkins is the primary deployment tool.
  • Ansible—The Jenkins pipeline automation is written by using Ansible and shell scripts.
  • Git—Git is used to store the deployment artifacts.
  • kubectl—is the Kubernetes client that is used to run the Kubernetes commands.
  • Helm—Helm is the package manager used to deploy applications in Kubernetes.

These tools are used by the Jenkins pipelines to deploy BMC Helix Service Management applications.

 

Related topics

System-requirements

BMC-Deployment-Engine-overview

FAQ

Skills required

  • Using Jenkins
  • Using Linux

See Recommended-skill-set-and-trainings

The steps to set up BMC Deployment Engine are shown in the following figure:

NewDESetup.png

Before you begin

Important

  • To login to the BMC Deployment Engine server, use SSH clients, such as PuTTY, MobaXterm, mRemoteNG, or equivalent tool supported by HOST/GUEST operating system. This server is referred to as Jenkins server in the rest of the document.
  • While configuring BMC Deployment Engine, use a user with sudo access. The sudo access permission is required to install application packages such as Jenkins.
  • Make sure that BMC Deployment Engine server is registered with the Red Hat subscription manager and a valid subscription pool ID.
  • Unless specified, the user refers to non root user for this setup.
  • BMC Deployment Engine uses the following two users:
    • git—Owner of git repositories where deployment code is present. This user requires sudo access.
    • jenkins—Jenkins server runs as a Jenkins user. This is a non-interactive user.
  • Make sure that the user has a default shell as sh or bash.
  • Make sure that the Jenkins server has access to the internet to access the online Jenkins repository for OS software update or upgrade as well as install tools required to set up Deployment Engine and supporting packages.
  • Make sure that the Jenkins server has direct internet access without a proxy.
  • The nomenclature used in pipeline development is as follows:
    • The Home directory of non root user is referred to as <HOME> in all subsequent steps. By default, it is /home/git.
    • <user> refers to the git repo owner. Jenkins user owns the Jenkins setup. The git user is a member of the git and Jenkins groups.
      The Jenkins user is member of the Jenkins and git groups.
    • <hostname> is the host name/IP address of the Jenkins server, which is resolvable by all participating entities, such as databases and clusters in this environment.
    • <GIT_REPO_DIR> is the location where the entire Deployment Engine git code is copied by the installer for setting up the pipelines. The recommended size of this directory must be 10GB or more.
    • <Library_REPO_DIR> is the location where the installer copies the required libraries that need to be configured manually after installation.

Ensure that your environment meets the following requirements:

  • Virtual machine—A fresh and dedicated physical server or virtual machine with the following configuration for the Jenkins server:

    Component

    Node

    vCPU

    Operating System

    RAM (GB)

    Disk (GB)

    Jenkins server

    1

    2

    • Cent OS 7.x and 8.x are not supported.
    • RHEL 8.x and 9.x, Rocky Linux 8.x and 9.x are certified.

    Minimum 8 GB

     

    100

    Make sure that the /home, /var, /usr mount points have a minimum of 15 GB free space and /tmp has a minimum of 5 GB free space.

  • Kubernetes or OpenShift cluster—Kubernetes cluster is accessible from the Deployment Engine server. 
    For Kubernetes cluster requirements, see System Requirements.

  • Yum or DNF utility—is installed, configured, and working fine. 
    Make sure that the yum client is installed and accessible, and you can install certified repos stable versions of applications. 
    Run the following commands by using a user with sudo access to update your system libraries and packages to the latest available version.

    sudo yum update -y
    sudo yum upgrade -y
    sudo yum clean all

  • Unzip utility—is installed.
    Use the following command to install the unzip utility:

    sudo yum install unzip

  • Perl—Perl (version 5.x +) is installed and is accessible from shell prompt.
    Use the following command to set up Perl:

    sudo yum install perl -y

  • Perl-Data-Dumper package—Set up Perl-Data-Dumper package by using the following commands:

    sudo yum makecache
    sudo yum -y install perl-Data-Dumper
    • Make sure that /etc/resolv.conf has proper DNS entries.

    • From any remote machine run the following commands:

      nslookup <jenkins-server-name> 
      nslookup <jenkins-server-ip>

      both the commands should resolve via DNS from all servers where clusters are setup.

    • from /etc/hosts make sure to have only relevant entries in this file. If an entry is not correct, delete the entry and make sure that you clean up all the stale entries in the file.
    • Remove the ivp6 entry for the localhost and ensure that the hostname resolves to the correct DNS IP address.
    • Run the yum repolist or dnf repolist command to view all accessible and OS matching repositories.  The yum or dnf commands should be able to install the required repository software. 
  • Download the following files from EPD to the Jenkins server.
    • BMC_Helix_Innovation_Suite_And_Service_Management_Apps_Version_23.3.01.002.zip
      Use the BMC Helix Innovation Suite & Service Management Apps Security Hotfix 2 Version 23.3.01 option to download the file.
    • BMC_Remedy_Deployment_Engine_Setup_23.3.01.001.zip
      Use the BMC Helix Deployment Engine 23301 Hotfix1 option to download the file.
      For details, see Downloading-the-installation-files.

Back to top

To create and configure users

BMC Deployment Engine uses the following two users:

  • git—Owner of git repositories where deployment code is present. This user requires sudo access.
  • jenkins—Jenkins server runs as jenkins user. This is a non-interactive user.

You must create and configure these users.

To add users

Perform the following steps to add the users:

Important: Make sure that you do not use special characters in the git user password.

sudo useradd git -m
sudo passwd git
sudo useradd jenkins -m
sudo passwd jenkins

To configure users and groups

Perform the following steps to configure the users and groups:

sudo usermod -a -G git jenkins
sudo usermod -a -G jenkins git

To configure sudo access to the git user

Run the following command to configure sudo access to the git user:

sudo usermod -aG wheel git

To configure passwordless sudo access for the git user

Perform the following steps to configure passwordless sudo access to the git user:

  1. Run the following command:

    sudo visudo

  2. Uncomment the following line:

    %wheel ALL=(ALL) NOPASSWD: ALL

  3. Comment the other lines that start with %wheel. 
    For example:

     %wheel ALL=(ALL) ALL

  4. Save the changes.

    Back to top

To copy the ssh keys

Important: If you are using BMC Helix Service Management 23.3.01 hotfix 1, you must manually create the ssh keys. Perform the following steps only if you are using the the BMC_Remedy_Deployment_Engine_Setup_23.3.01.001.zip file for BMC Deployment Engine setup.

  1. Log in to BMC Deployment Engine as a git user.
  1. Configure the SSH keys for the git user to enable pipeline access to Git repositories.

    1. Generate the SSH key by running the following command and accept all the defaults.

      ssh-keygen

    2. Copy the ID by running the following command:

      ssh-copy-id git@<jenkins_server_name>

    3. Verify that the passwordless ssh login works from git to git user by using the following command:

      ssh git@<jenkins_server_name>

  2. Log in to BMC Deployment Engine as a jenkins user.
  3. Configure jenkins ssh keys to git so that the pipeline dry-run step is successful.

    1. Generate the SSH key by running the following command and accept all the defaults.

      ssh-keygen

    2. Copy the ID by running the following command:

      ssh-copy-id git@<jenkins_server_name>

    3. Verify that you are logged in as git user and the passwordless ssh login works by using the following command:

      ssh git@<jenkins_server_name>

To run the BMC Deployment Engine automation script

The BMC Deployment Engine automation script installs the following software:

  • Ansible
    Ansible 2.15 is certified.
  • Python
  • Git
  • kubectl
  • Helm
  • Java
  • PostgreSQL client
  • Docker
  • additional packages, such as MS SQL tools, unixODBC, xmlstarlet, dos2unix, jq

The automation script also installs Jenkins locally on a server (In earlier versions of BMC Helix Service Management Deployment, you had to manually set up Jenkins on a local node).

Along with the software and Jenkins, the BMC Deployment Engine automation script installs the plug-ins listed in the plugins.txtfile.

Perform the following steps to run the BMC Deployment Engine automation script:

  1. Log in as a git user:

    ssh git@<hostname>

    or switch to the git user by using the following command:

    su - git
  2. Copy the BMC_Helix_Innovation_Suite_And_Service_Management_Apps_Version_23.3.01.002.zip file downloaded from EPD and extract the files to the git user home directory or any other directory owned by the git user.
    • BMC_Remedy_Deployment_Manager_Configuration_Release_23.3.01.002.zip

    • BMC_Remedy_Deployment_Engine_Setup_23.3.01.002.zip—Do not use this file.

      Important

      Do not use the BMC_Remedy_Deployment_Engine_Setup_23.3.01.002.zip file, instead use the BMC_Remedy_Deployment_Engine_Setup_23.3.01.001.zip file.

  3. Copy the BMC_Remedy_Deployment_Engine_Setup_23.3.01.001.zip file downloaded from EPD and extract the files to the git user home directory or any other directory owned by the git user.

  4. Change the ownership of these files to git user by using the following command: 

    sudo chown -R git:git <filename>
  5. Unzip BMC_Remedy_Deployment_Engine_Setup_23.3.01.001.zip file. 
  6. Change the directory to DE1.0.
  7. Update build.properties and customize the following parameters according to your requirements:
    • ITSM_REPO_GIT_ZIP=</path/to/BMC_Remedy_Deployment_Manager_Configuration_Release_23.3.01.002.zip>
    • JENKINS_CONFIG_FILES_ZIP_PATH=<path to Jenkins_Config_Files.zip file>
    • LIBRARY_REPO_ZIP_PATH=<path to LIBRARY_REPO.zip>
    • JENKINS_HOSTNAME=<fully-qualified-hostname-or-ip-address-where-DE-setup-is-planned>

    • KUBERNETES_VERSION=<Kubernetes cluster version>

      Important

      Specify the entire version such as KUBERNETES_VERSION=1.25.13

      For the supported Kubernetes version, see System-requirements. Make sure that you specify the correct Kubernetes version value. See Kubernetes version in Kubernetes documentation.

    • GIT_REPO_USER=git
    • GIT_USER_HOME_DIR=~git
    • JENKINS_USER=jenkins

    • HELM_VERSION=<Helm version>

      Important

      Specify the version HELM_VERSION value as 3.13.0

       

    • JENKINS_INSTALL_DIR=/var/lib/jenkins
    • HTTP_PROTOCOL=http
    • JENKINS_PORT=8080
    • ANSIBLE_NODE_ROOT_DIR=~/ansible_node_root_dir
    • JENKINS_NODE_ROOT_DIR=~/jenkins_node_root_dir
      (This is the path that the Jenkins_node needs for various operations that are internal to Jenkins)
    • GIT_REPO_DIR=~/git_repo/
      ( This is the location where the entire Deployment Engine git code is copied by the installer for setting up the pipelines.)

    • JENKINS_CONFIG_FILES_DIR=~/Jenkins_Config_Files
      (The Jenkins job pipeline uses this location to store its pipeline jobs.)

      Important

      Do not modify the values of the JENKINS_INSTALL_DIR, HTTP_PROTOCOL, and JENKINS_PORT parameters.

      For information about the supported Kubernetes, OpenShift, and Helm versions, see System-requirements.

  2. After updating the build.properties file, run the BMC Deployment Engine automation script to setup the Jenkins job pipeline framework.
    The log file is generated in the Git user home directory and it can be used for debugging setup related issues.

    $ sudo perl setup-Helix-ITSM-onPrem.pl  2>&1 | tee ~/BMC-HELIX-DE-AUTO.log.$$

Back to top

To perform post-installation configurations

Complete the following configurations:

Important

Due to recent changes in the Red Hat repositories, ansible-core 2.16 is installed by default on RHEL 8.x systems. Ansible 2.16 is not compatible with BMC Helix Service Management deployments.

If you are using RHEL 8.x, downgrade Ansible version to 2.15 by using the following commands:

sudo dnf downgrade ansible-core-2.15.3-1.el8.x86_64
sudo yum install python3.11-jmespath -y

No.

Action

Steps

1

Install plug-ins

  1. Log in to the Jenkins server by using the following URL:
    http://<Jenkins server host name>:<Jenkins port>

    The default password is available in $<JENKINS_INSTALL_DIR>/secrets/initialAdminPasswordFor example: /var/lib/jenkins/secrets/initialAdminPassword

  2. On the Jenkins User Interface, select Install Suggested Plugins.
  3. In the Admin User Creation wizard, provide a preferred username and password and run the wizard.
  4. Verify that ssh works fine with passwordless login for the git and jenkins users:

    1. Verify the git user login to git@<jenkins_server_name> and jenkins@<jenkins_server_name> by using the following commands:

      ssh git@<jenkins_server_name>
      ssh jenkins@<jenkins_server_name>

    2. Verify the jenkins user login to git@<jenkins_server_name> by using the following commands:

      ssh jenkins@<jenkins_server_name>
      ssh git@<jenkins_server_name>

       

  2.  Add the kubeconfig file on the Jenkins server.
    1. Download the kubeconfig.yaml file to the $<git user home> directory.

    2. Copy the kubeconfig.yaml file to $<git user home>/.kube/config directory.

      cp <kubeconfig complete file path> $<git user home>/.kube/config

2

Update the Jenkins credentials

  1. Login to the Jenkins server by using the following URL:
    http://<Jenkins server host name>:<Jenkins port>/credentials
  2. Add the credentials in the ID: kubeconfig file.
    1. Select kubeconfig and click the name of the kubeconfig credential.
      JenkinsCredentials.png
    2. Click Update.
    3. Select the Replace checkbox.
    4. Click Choose File and select the kubeconfig file
    5. Click Save.
  3. Add the credentials in the ID: github account.
    1. Select github and click the name of the github credential.
    2. Click Update.
    3. Click Change Password and enter the git user password.
    4. Click Save.
  4. Add the credentials in the ID: ansible_host account.
    1. Select ansible_host and click the name of the ansible_host credential
    2. Click Update.
    3. Click Change Password and enter the git user password.
    4. Click Save.
  5. Add the credentials in the ID: ansible file.
    1. Select ansible and click the name of the ansible credential.
    2. Click Update.
    3. Click Change Password and enter the git user password. 
    4. Click Save.
  6. Add credentials in the ID: git account.
    1. Select git and click the name of the git credential.
    2. Click Update.
    3. Click Change Password and enter the git user password.
    4. Click Save.
  7. (For cloud environments) Update git user private key credentials
    1. Navigate to http://<Jenkins server host name>:<Jenkins port>/credentials/store/system/domain/_/
    2. Click Add Credentials.  

    3. Enter the following details:

      Kind

      SSH Username with private key

      ID

      git_pk

      Username

      git

      Private Key

      1. Click Enter.
      2. Paste the contents of git user private key value (/home/git/.ssh/id_rsa)
    4. Save the details.

Depending on your environment, you can choose to update the node configuration in a cloud or non cloud environment.

3
 

Update the node configuration in a cloud environment

  1. Navigate to http://<Jenkins server host name>:<Jenkins port>/computer.
  2. Perform the following steps for both nodes, the one with the actual hostname and another with the git hostname.
    1. Click the node name. 
    2. Click Configure.
    3. Under Credentials, select the git user ssh key credential and click Save.
    4. Click Launch Agent.
    5. Verify that the launch agent is successful.

Update the node configuration in a non cloud environment

  1. Navigate to http://<Jenkins server host name>:<Jenkins port>/computer.
  2. Perform the following steps for both nodes, the one with actual hostname and another with git-hostname.
    1. Click the node name.
    2. Click Launch Agent.
      Important: If the status of the node is already 'online', then skip Step b.
    3. Verify that the launch agent is successful.

4

 Add the Jenkins libraries

Perform the following steps to add the pipeline-framework library:

  1. On the Jenkins home page, click Manage Jenkins.
  2. Select System.
  3. In Global Trusted Pipeline Libraries, add the pipeline-framework library as shown in the following figure:
  4. In the Project Repository field, specify the complete path of pipeline-framework.git according to the environment.
    <GIT_REPO_DIR> value can be fetched from build.properties file.
    pipelineframework_1.png
  5. Make sure you provide exact location of pipeline-framework.git according to the environment.
  6. Add JENKINS-27413-workaround-library as shown in the following figure:
  7. In the Project Repository field, specify the complete path of JENKINS-27413-workaround-library.git according to the environment.
  8. Select Load implicitly.
    JenkinsLibrary_1.png
  9. Save the changes.

Important

After you complete the BMC Deployment Engine configurations, remove the sudo access of the Git user by using the following command:

gpasswd --delete git wheel

To run the deployment pipelines in dry-run mode

Dry-run is a mandatory step to update the pipeline configuration for any changes to the BMC Helix Innovation Suite and Service Management Installer.

Important

Even though you see the Build with Parameters option for all the parameters, you have to still perform a dry-run.

  1. Navigate to Jenkins Dashboards to view all the pipelines required for deployment.
  2. Select each pipeline and click Build with Parameters.

  3. Click Build.
    The build job status fails, which is expected.

    • The agent-add-pipeline pipeline does not require a dry-run.
    • For HELIX_ONPREM_DEPLOYMENT pipeline, enter the value of the AGENT parameter.
    • The AGENT parameter value is the Jenkins agent to run the pipeline. Provide the value of the node that has the name git-<hostname>.
    • In the HELIX_ONPREM_DEPLOYMENT pipeline, do not select any pipelines in the PRODUCT-DEPLOY section.
    • For the other pipelines, run Build with Parameters with default values present in the pipeline.

After the dry run is successful, verify that correct version of BMC Helix Service Management is populated in the PLATFORM_HELM_VERSION and SMARTAPPS_HELM_VERSION parameters.

Back to top

Troubleshooting

Refer to the troubleshooting information if you encounter any of the following issues while setting up BMC Deployment Engine.

Important

When logging a case with BMC Support for any issues that occur while setting up BMC Deployment Engine, we recommend you to include the log file and build.properties file to facilitate debugging the issues.

Issue symptom

During deployment engine script execution, you might receive the following message:
This system is not registered with an entitlement server. You can use subscription-manager to register.
         Log: Query :  Please, suggest should we disable subscription manager and proceed with setting up prereq for ITSM onprem deployment,  Y(yes) or N(no) ? :

Resolution

  • If you are using Rocky Linux 8 or Rock Linux 9 operating system, enter yes and proceed.
  • If you are using RHEL 8 or RHEL 9 operating system, enter no and proceed. In this case, register your system with Red Hat for a valid subscription.

Issue symptom

The installation script fails or aborts for some reason.

Resolution

  1. Review the BMC-HELIX-DE-AUTO.log file to check for errors.
  2. Fix the errors.
  3. Stop Jenkins by using the systemctl command.
  4. Check if port 8080 is free by using netstat -anp | grep 8080 command and then rerun automation script command.
  5. Make sure to save log files.

Issue symptom

HELIX_ONPREM_DEPLOYMENT pipeline fails with the following scripts not permitted error.

method hudson.model.Run getLog
method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild

Resolution

  1. Right click the error message link and open it in a new tab.
  2. Click Approve.
    KnownIssues1.png
  3. Click the latest failed job on the HELIX_ONPREM_DEPLOYMENT pipeline tab.
    FailedJob.png
  1. Click Rebuild.
    Rebuild.png
  2. Enter the required parameters and click Rebuild.
    EnterParameters.png
  3. You might get other scripts not permitted error. Repeat the same steps to approve the script execution and rebuild HELIX_ONPREM_DEPLOYMENT jobs.

For other issues, see Troubleshooting-upgrade-and-pipeline-failure-issues.

For additional questions about BMC Deployment Engine, see FAQ.

Where to go from here

Next task

Back to process

If you are finished installing BMC Helix Platform services, return to the appropriate installation or upgrade process:

 

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*