Configuring the on-premises gateway for high availability on Docker and Podman containers
Before you begin
- To use the BMC Helix object storage:
- Access the UI for the on-premises gateway instances that are part of the high-availability deployment and make sure that the same destination is added for all instances.
- Make sure that the access key and secret key used in the destination have all [*] permissions.
- To use a private object storage, make sure that a standalone S3-compatible storage instance is set up, and you have obtained the credentials that have permissions to create and read objects. Contact your system administrator to obtain the credentials.
To prepare the environment for data collection by using a webhook connector
If you are using a webhook connector to collect data from a third-party product in a high-availability deployment, you can use a load balancer (for example, F5) to direct the webhook data traffic to the active on-premises gateway instance.
Perform the following steps to prepare the environment for data collection by using a webhook connector:
- Configure the load balancer.
- Create a DNS record (for example, VIP_HII) for the virtual IP (VIP).
- Create a pool, and add the on-premises gateway instances as members, as shown in the following example:-ltm pool VIP_HII {
description "Gateway Pool"
members {
aus-pun-01.abc.com:https {
address 192.168.111.xx
session monitor-enabled
state down
}
aus-pun-02.abc.com:https {
address 192.168.112.xx
session monitor-enabled
state down
}
}
monitor VIP_HII
}The above example creates a pool for VIP_HII, with two members: aus.pun-01.abc.com (IP address: 192.168.111.xx) and aus.pun-01.abc.com (IP address: 192.168.112.xx)
Create the monitor to check the status of an on-premises gateway instance, as shown in the following example:
ltm monitor https VIP_HII {
adaptive disabled
defaults-from https
interval 5
ip-dscp 0
recv false
recv-disable none
send "GET /hii/api/mediator/v3/standbyStatus HTTP/1.1\r\nHost: VIP_HII\r\nConnection: Close\r\n\r\n""
time-until-up 0
timeout 16
}
- While including the webhook collector URL in the third-party product (for example, Entuity), replace the on-premises gateway host name with the DNS record in the webhook collector URL. For example, an updated webhook collector URL looks like:
https://VIP_HII/hii/api/mediator/v3/push/9mn-6c97-4c2e-8pc5-12c0asdfd?token=385261281::Y40OSC49QZA11Q8A1H9H6::MnVLk69TNyCEponsthHJ1Hj1uKcjTB
For more information about configuring the URL for Entuity, see Integrating with Entuity via webhook.
(For BMC Helix storage included with the BMC Helix SaaS deployment) To configure on-premises gateway instances for HA
- Log on to the on-premises gateway instance that you want to add as the primary instance.
- Navigate to the /IIGateway_INSTALL_DIR/hii directory.
- Open the docker-compose.yaml file or podman-compose.yaml file by using a text editor.
- Locate the mediator > environment section, and set the following properties:
- USE_ADE_STORAGE: Specify the object storage type to be used in the HA deployment. Set the property to true (default value is false).
USE_ADE_STORAGE: "true" HIIG_HA_CLUSTER_ID: Specify a unique cluster ID (an integer from 0 to 9) in case you have multiple clusters, each cluster containing multiple on-premises gateway instances. The default value is 0, which indicates that you have only one cluster.
DATA_PUSH_INTERVAL: Specify the interval (in milliseconds in epoch format) at which an on-premises gateway instance should push data to the BMC Helix object storage (default and minimum 300000 milliseconds).
DATA_PUSH_INTERVAL: "300000"SOURCE_HOSTNAME: Replace $HOSTNAME with the name of the server where the on-premises gateway is deployed.
- USE_ADE_STORAGE: Specify the object storage type to be used in the HA deployment. Set the property to true (default value is false).
- Restart the container services by using the following commands:
For Docker deployments:
docker-compose down
docker-compose up -dFor Podman deployments:
podman-compose down
podman-compose -f podman-compose.yaml up -dWhen you restart the services, the on-premises gateway instance is added as the primary instance.
- Log on to the subsequent on-premises gateway instances and repeat steps 2 to 5.
The subsequent instances are added as secondary instances in the storage.
(For private object storage) To configure on-premises gateway instances for HA
- Log on to an on-premises gateway instance that you want to add as the primary instance.
- Navigate to the /IIGateway_INSTALL_DIR/hii directory.
- Open the docker-compose.yaml file or podman-compose.yaml file by using a text editor.
- Locate the mediator > environment section, and depending on the storage type you are using, set the following properties:
- USE_ADE_STORAGE: The object storage type to be used in the HA deployment.
Make sure that it is set to false.
USE_ADE_STORAGE: "false" MINIO_ACCESS_KEY: Access key for the S3-compatible storage.
MINIO_SECRET_KEY: Secret key for the S3-compatible storage.
MINIO_SERVER_URL: API endpoint URL for the S3-compatible storage.
Example# MinIO
https://vx-push-dev26.abc.com:9000
# Amazon S3
https://s3.us-west-1.amazonaws.comMINIO_BUCKET_NAME: The name of the bucket you want to create on the S3-compatible storage instance.
Important: If you are using the same S3-compatible storage server for multiple HA environments (for example, Test and Production), make sure that each bucket has a unique name.DATA_PUSH_INTERVAL: The interval (in milliseconds) in epoch format at which an on-premises gateway instance should push data to the storage bucket (default and minimum 300000 milliseconds).
The following example shows sample values:MINIO_ACCESS_KEY: "P3pWEoNUEmZB8i0zJAnC"
MINIO_SECRET_KEY: "SgA3ntRrdM3nzUGpvKjRMQ2FJZNHujfngxJgTb"
MINIO_SERVER_URL: https://vx-push-dev26.abc.com:9000
MINIO_BUCKET_NAME: "helix-hii-backup"
DATA_PUSH_INTERVAL: "300000"
- USE_ADE_STORAGE: The object storage type to be used in the HA deployment.
- Save and close the file.
- If your object storage is using custom or CA-signed certificates, import them into the on-premises gateway instance.
- Restart the container services by using the following commands:
For Docker deployments:
docker-compose down
docker-compose up -dFor Podman deployments:
podman-compose down
podman-compose -f podman-compose.yaml up -dWhen you restart the services, the instance is added as a primary instance and storage bucket, helix-hii-backup is created. The bucket contains the configuration backup file, Leader_backup.json, which stores the instance configuration backup with timestamp. The backup file contains a tag for the on-premises instance.

- Log on to the subsequent on-premises gateway instances and repeat steps 2 to 7.
The subsequent instances are added as secondary instances in the bucket.
To import the custom or CA-signed certificates into the on-premises gateway
If you are using MinIO as the object storage, import the custom or CA-signed certificates into the on-premises gateway instances.
- Obtain the custom or CA-signed certificates from the MinIO server and save it in the /<IIGateway_INSTALL_DIR>hii/conf/certs directory.
- Navigate to the /<IIGateway_INSTALL_DIR>/hii/conf/certs directory.
- Copy the cacerts file present in the swp-mediator container at /opt/java/lib/security to the /<IIGateway_INSTALL_DIR>/hii/conf/certs directory by using the following command:
For Docker deployments:
docker cp swp-mediator:/opt/java/lib/security/cacerts /<IIGateway_INSTALL_DIR>hii/conf/certs/cacertsFor Podman deployments:
podman cp swp-mediator:/opt/java/lib/security/cacerts /<IIGateway_INSTALL_DIR>hii/conf/certs/cacerts
Import the certificates into the copied cacerts file by using the keytool utility:
keytool -importcert -file minio.crt -keystore cacerts -alias minio- Navigate to the /IIGateway_INSTALL_DIR/hii directory.
- Open the docker-compose.yaml file or podman-compose.yaml file by using a text editor.
- Search for the mediator section.
Add the following line to the volumes section:
- ./conf/certs/cacerts:/opt/java/lib/security/cacerts- Save and close the file.
- Go to Step 7 to continue configuring the on-premises gateway.
To troubleshoot HA issues
You might encounter the following issue if you are using the MinIO instance that is present in the BMC Helix IT Operations Management on-premises environment.
Issue
The on-premises gateway fails to upload the Leader_backup.json file to the MinIO instance, and the following error message appears in the swp-mediator container logs:
Resolution
- Open the ingress-nginx-controller ConfigMap for the NGINX Ingress controller.
- Add the following parameter and save the ConfigMap:
ignore-invalid-headers: "false" - Perform a rolling restart of NGINX Ingress controller pods.
- Restart the container services on the on-premises gateway host by using the following commands:
For Docker deployments:
docker-compose down
docker-compose up -dFor Podman deployments:
podman-compose down
podman-compose -f podman-compose.yaml up -d