Setting up a Harbor repository to synchronize container images

 The following video (2:54) provides a summary of how to set up a Harbor repository in BMC Helix Service Management.

https://youtu.be/2kPk7ui1IoA

Repository content

The images required for BMC Helix Service Management installation are located in the following repositories in BMC DTR:

Process to access container images from a local Harbor repository

Set up a Harbor repository and synchronize your Harbor repository with BMC DTR by using the access key. The following images shows the actions to synchronize your Harbor repository with BMC DTR.

Before you begin

• Download the following files:
  • 25401001_ITSM_Platform_Images.txt
  • 25401001_ITSM_SmartApps_Images.txt
  • 25401001_ITSM_Pipeline_Images.txt
  • 25401001_SupportAssistTool_Images.txt
  • 254_Helix_Platform_Images.txt
  • 253_Helix_Platform_Images.txt
  • 25401001_Chainguard_Images.txt
  • 210503HF12_SmartReporting_Images.txt

These files contain the list of images that are synchronized from BMC DTR. You can use these files to verify your Harbor repository after you synchronize it with BMC DTR.

• Download the 254_Helix_Platform_Images_Digest.txt and 25401_ITSM_Images_Digest.txt files.

This file contains the Docker image digests for verification.

• Make sure that you have downloaded the key to access the container images, image_pull_push.sh, image_sync_to_private_registry.sh, and image_sha_digest.sh files from the BMC Electronic Product Distribution (EPD) site.
  See Downloading the installation files.
• Make sure you have downloaded the delete_images.py utility and the ignore_images.txt file.
  The delete_images.py  utility deletes the obsolete container images from a Harbor repository. The ignore_images.txt file contains the images that must not be deleted from the Harbor repository.
• Make sure that you meet the Harbor repository requirements.
• Install the JQ command tool.

Task 1: To create a Harbor registry

1. In your local system, download the latest version of Harbor by using the following command:

   wget https://github.com/goharbor/harbor/releases/download/v2.1.4/harbor-offline-installer-v<version>.tgz

   Example:

   wget https://github.com/goharbor/harbor/releases/download/v2.1.4/harbor-offline-installer-v2.1.4.tgz

2. Unzip the TAR file by using the following command:

   tar xvzf harbor-offline-installer*.tgz

3. Navigate to the harbor directory by using the following command:

   cd harbor

4. Copy the configuration template by using the following command:

   cp harbor.yml.tmpl harbor.yml

5. Edit the harbor.yml file and add the values for the following parameters:

   Success

   Best practice
   We recommend that you use the HTTPS configuration and install Harbor by using self signed SSL certificates. See Configure HTTPS Access to Harbor .

   Parameter	Description
   hostname	Name of the host where you want to install Harbor.
   harbor_admin_password	Password for the Harbor application admin username.

   In the harbor.yml file, uncomment the https related config section, and add the values for parameters such as port, certificate, and  private_key.
   For example:

   # https related config
   https:
     # https port for harbor, default is 443
    port: 443
     # The path of cert and key files for nginx
    certificate: /data/cert/dsmlab.bmc.com.crt
    private_key: /data/cert/dsmlab.bmc.com.key

6. Run the following command:

    sudo ./install.sh
7. Verify that you can access the Harbor registry by using the admin username and password to log in.

Task 2: To create a project

1. In the Harbor admin UI, navigate to the Administration menu, and click Projects.

2. Create a new project and specify the project name as bmc.
   This project is used to synchronize the container images.

Task 3: To synchronize a Harbor repository with BMC DTR

You must synchronize your Harbor repository with BMC Helix Innovation Suite and BMC Helix Platform Common Services container images in BMC DTR.

1. Log in to a Linux system.
2. Copy the image_pull_push.sh, image_sync_to_private_registry.sh, 25401001_ITSM_Platform_Images.txt, 25401001_ITSM_SmartApps_Images.txt, 25401001_ITSM_Pipeline_Images.txt, 25401001_SupportAssistTool_Images.txt, and 25401001_Chainguard_Images.txt.
3. Based on the BMC Helix Platform Common Services version you are using, copy the 254_Helix_Platform_Images.txt or 253_Helix_Platform_Images.txt files to the system.
4. Copy the file to the system.
5. Create a new file named images.txt.
6. Synchronize the BMC Helix Service Management platform container images:
   1. Copy the 25401_ITSM_Platform_Images.txt to the images.txt file.

   2. Open the image_sync_to_private_registry.sh file and update the following parameter values:

      Parameter	Description
      SOURCE_REGISTRY_HOST	Specify the value as docker.io
      SOURCE_REGISTRY_USER	Specify the user ID generated with the Personal Access Token (PAT).
      SOURCE_REGISTRY_PASSWORD	Specify the Personal Access Token (PAT) that you generated from Docker.  For information on how to generate the token, see Downloading the installation files.
      IMAGE_REGISTRY_HOST	Host name of your local registry.
      IMAGE_REGISTRY_USERNAME	Specify the user name to log in to your local registry.
      IMAGE_REGISTRY_PASSWORD	Specify the password to log in to your local registry.
      IMAGE_REGISTRY_PROJECT	Specify the value as bmc

   3. Convert the images.txt file to UNIX format by using the following command:

      dos2unix images.txt

   4. Run the image_sync_to_private_registry.sh file by using the following command:

      Warning

      Important

      Before you run the image_sync_to_private_registry.sh file, make sure that you have installed the Docker Engine. For more information, see Harbor repository requirements.

      ./image_sync_to_private_registry.sh
7. Synchronize the smart application, pipeline, BMC Helix Platform services, and Support Assistant tool container images by performing the following steps:
   1. Copy the file to the images.txt file.

   2. Run the image_sync_to_private_registry.sh file by using the following command:

      ./image_sync_to_private_registry.sh

Task 4: To verify image digest in the Harbor registry

Verify the integrity of container images in your Harbor registry by validating the digests. This verification ensures that only trusted, untampered images are used for BMC Helix Service Management installation. Verifying image integrity helps prevent pod startup issues that may arise due to synchronization discrepancies between the BMC DTR and your local registry.

Perform the following steps:

1. In the image_sha_digest.sh file replace the following image parameter value:

   Replace image="${original_image/containers.bmc.com\/bmc\//${new_registry}/}" with

   image="${original_image/docker.io\/bmchelix\//${new_registry}/}"

2. To verify the image digest in your Harbor registry for the BMC Helix Service Management images, run the following command:
3. ./image_sha_digest.sh <Harbor_registry_location> 25401_ITSM_Images_Digest.txt 

   Example command:
   ./verify_image_digest.sh harbor.customer.org/repo 25401_ITSM_Images_Digest.txt

   This command automates the validation of container image digests by taking the following actions:

   1. Registry replacement—Replaces all instances of containers.bmc.com/bmc/ in image references with the specified Harbor registry location.
   2. Digest retrieval—Uses docker manifest inspect verbose to fetch the digest of each updated image.
   3. Digest comparison—Compares the actual digest with the expected digest for each image.
   4. Logging—Logs the result of each comparison as either match or mismatch.
   5. Exit status—Terminates with exit code 1 if any mismatches are detected; otherwise, exits with code 0.

    Example command output:
   Starting digest verification using registry: harbor.customer.org/repo
   
   Checking image:attach:xwiki:Service-Management.On-Premises-Deployment.BMC-Helix-Service-Management-Deployment.brid25401.Installing.Preparing-for-installation.Setting-up-a-Harbor-repository-to-synchronize-container-images.WebHome@filename harbor.customer.org/repo/adeops-util:567-25201-RTM
   Match: harbor.customer.org/repo/adeops-util:567-25201-RTM
   
   Checking image:attach:xwiki:Service-Management.On-Premises-Deployment.BMC-Helix-Service-Management-Deployment.brid25401.Installing.Preparing-for-installation.Setting-up-a-Harbor-repository-to-synchronize-container-images.WebHome@filename harbor.customer.org/repo/cmdb-core:25201-RTM
   Mismatch: harbor.customer.org/repo/cmdb-core:25201-RTM
      Expected: sha256:xxxx...
      Found   : sha256:yyyy...
   
   Failed: The following images failed digest verification:
    - harbor.customer.org/repo/cmdb-core:25301-RTM

    

4. To verify the image digest in your Harbor registry for the BMC Helix Platform Common Services images, run the following command:​./image_sha_digest.sh <Harbor_registry_location> 254_Helix_Platform_Images_Digest.txt 

5. If you encounter a mismatch for an image, resynchronize the image.

(Optional) To remove obsolete container images

Use the delete_images.py utility to remove obsolete container images from your Harbor repository to optimize storage and retain only approved images. The utility reads a list of images from a text file, skips any image listed in the  ignore_images.txt file, and performs authenticated deletions.

1. Make sure that you have downloaded the delete_images.py and ignore_images.txt files.
2. Install Python 3.7 or later on your system.
3. Create a file named all_images.txt and add the repository content that you want to remove.
4. To preview the images to be deleted, run the delete_images.py utility in dry run mode  by using the following command:
   python3 delete_images.py [--dry-run] [--insecure]
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   This command prompts for inputs.
5. Enter the value for the following fields:

   Field	Description
   Enter Registry URL	Specify the URL of your Harbor registry. Example: https://harbor.myorg.com
   Enter registry username	Specify the Harbor registry user name.
   Enter registry password	Specify the Harbor registry password.
   Enter path to image list file	Specify the all_images.txt file path.
   Enter path to ignore list file	Specify the ignore_images.txt file path.
   Enter image prefix	Specify the value as bmc/.

6. To delete the images, run the delete_images.py utility by using the following command:
   python3 delete_images.py --insecure
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   The command output example is as follows:
   📦 Found 25 images to process.
   🚫 3 ignore patterns loaded.
7. On the Do you really want to continue? (yes/no) prompt, type yes.

   This action permanently deletes the images from the Harbor registry.

8. To view the images that are deleted, see the deletion_log_YYYYMMDD_HHMMSS.csv file.

   A timestamped CSV log file is generated for each run of the utility.

   Information

   Important

   When images are deleted from the Harbor registry, the space they occupy is not automatically released. You must perform garbage collection to free up space by removing blobs that are no longer referenced by a manifest from the file system. For more information, see Garbage Collection  in Harbor documentation.

    

To access images from an air-gapped environment

The following image describes the process to access the BMC Helix Innovation Suite platform and application container images from an air-gapped environment:

Perform the following steps to access images from an air-gapped environment:

1. Create and configure a Harbor registry in a demilitarized zone (DMZ).
2. Create and configure a local Harbor registry in your network.
3. Synchronize your Harbor repository in a DMZ with BMC DTR.
4. Synchronize your Harbor repository in a DMZ with your local Harbor repository in your network.
   Perform the steps in To synchronize a Harbor repository with BMC DTR by modifying the following fields:
   • Source registry—Name of the DMZ Harbor registry that you configured to synchronize with your local Harbor repository.
   • Source resource filter: Name—Path of the image in your local Harbor repository that you want to synchronize to your DMZ Harbor repository.

Where to go from here