Backing up critical infrastructure components

In a BMC Helix IT Operations Management (BMC Helix ITOM) environment, specific data lake components are crucial for maintaining service operations. These components, such as PostgreSQL and OpenSearch events, store critical configuration, event, and metric data. To minimize data loss, we recommend that you configure local backups of the PostgreSQL and OpenSearch events on the primary stack by using the following information.

Before you begin

Make sure you configure the MINIO_DR_STORAGE_SIZE parameter according to the deployment size category.

To configure the MINIO_DR_STORAGE_SIZE parameter, follow these steps:

1. Log in to the controller or bastion machine, where the deployment manager is located.
2. Navigate to the helix-on-prem-deployment-manager/configs/ directory.
3. Open the <DEPLOYMENT_SIZE>.config  file in the vi editor.
   The available deployment sizes are compact, small, xsmall, medium, large, itsmcompact, itsmsmall, and itsmxlarge.
   For example, for a small deployment category, open the respective file by using the following command:
   vi small.config
4. Change the value of the MINIO_DR_STORAGE_SIZE according to the critical infrastructure backup deployment sizes. For more information, see Sizing and scalability considerations.
5. Save and close the file.

To configure the data backup

1. Navigate to helix-on-prem-deployment-manager/utilities/infra-backups directory.

2. In the infra-backups.config file, specify the values of the following parameters: 

   Parameter	Description	Example
   BUCKET_NAME	Specify the name of the bucket used to back up data on MinIO.  Use the following conventions while assigning a bucket name: Allowed: Length must be between 3 and 63 characters. Use lowercase letters, numbers, and hyphens (-). The name must start and end with a letter or number. Not allowed: Do not use uppercase letters, periods (.), or underscores (_). Do not start the name with xn--, as it may be interpreted as Punycode; for example, xn–bucketname. Do not place hyphens next to periods; for example, my-.bucket.com or my.-bucket. Do not end the name with a hyphen (-) or with -s3alias, because -s3alias is reserved for MinIO bucket access point alias name.	BUCKET_NAME=helix-backup
   SITE_NAME	Specify a name to identify the site from which you want to take a backup of the data.  Use only lowercase letters and numbers. Do not add any blank spaces. Do not use special characters except hyphens.	SITE_NAME=India
   NAMESPACE	Specify the namespace where you have installed BMC Helix ITOM.	NAMESPACE=helix-cluster3
   DR_BACKUP_INTERVAL_IN_HOUR	Specify the backup interval in hours. You can set values between 1 to 24. We recommend that you set the value of this parameter according to the size of your data. If your data size is large, you must take a backup less frequently. The value that you specify defines the interval for backing up your data. The following is an example guideline to help you determine how frequently to back up according to the size of your data: Small-sized data: Backup every 1 hour Medium-sized data: Backup every 2 hours Important: The backup interval applies to all the components that you back up. Example 1: If you set the value of this parameter to 1 hour, data backup is performed at the start of every hour as per the cron schedule (0 */1 * * *).  If your current cluster time is 2:15 P.M. on November 2: The first backup will occur at 3:00 P.M. This will be a complete data backup. Subsequent backups will occur at 4:00 P.M., 5:00 P.M., 6:00 P.M., and so on. These will be incremental backups.  At 3:00 P.M. on November 3, a complete data backup will occur. Example 2: If you set the value of this parameter to 3 hours, data backup is performed at the start of every third hour as per the cron schedule (0 */3 * * *).  If your current cluster time is 2:15 P.M. on November 2: The first backup will occur at 3:00 P.M. This will be a complete data backup. Subsequent backups will occur at 6:00 P.M., 9:00 P.M., and so on. These will be incremental backups.  At 3:00 P.M. on November 3, a complete data backup will occur.	DR_BACKUP_INTERVAL_IN_HOUR=2
   DR_MAX_BACKUP_TO_RETAIN	Specify the number of days for which you want to retain the backed-up data.   Important: The retention period applies to all the components that you back up.	DR_MAX_BACKUP_TO_RETAIN=5

3. Use the following configuration to enable backup for the PostgreSQL and OpenSearch components: 

   DR_K8SOBJECTS=yes
   DR_PTPOSTGRES=yes
   DR_VICTORIAMETRICS=no
   DR_LOGS_ELASTICSEARCH=no
   DR_EVENTS_ELASTICSEARCH=yes
   DR_KAFKA=no
   DR_ZOOKEEPER=no
   DR_MINIO=no
   DR_DRS_REPO=no
   DR_OPTIMIZE=no
   DR_AIOPS_PVC=no

   Important
   To configure backup monitoring, you must set the DR_K8SOBJECTS parameter to yes.
4. Navigate to the helix-on-prem-deployment-manager/utilities/infra-backups directory and run the following script to enable the component backup:
   infra-backups.sh

Backup configuration logs are saved in the helix-on-prem-deployment-manager/logs directory.

To validate the data backup

1. Log in to the controller.

2. To confirm that the cronjobs were successfully configured, run the following command: 

   kubectl -n <itom namespace> get cronjob | grep dr-

   Sample output:
   
   
   Verify that all the backup cron jobs are running according to your configured schedule.

3. After the first backup job is complete, log in to the MinIO web console and go to the Object Browser.
4. Go to the bucket that is configured to back up data; for example, helix-backup.
5. Open the site folder (<SiteName>; for example, India) where you are backing up your data, and then open the backupStatus folder.
6. Open the backup.log file and verify that there are no errors.
7. Download and open the last_backup.json file and validate that there are no errors.

To disable the backup of the data