Upgrading BMC Helix Platform services version from 22.1.02 to 22.4.00

To use BMC Helix platform services 22.4.00, upgrade BMC Helix Platform services from version 22.1.02 to 22.4.00.

The following image shows the upgrade process:

Before you begin

• Create a new working directory.
  For example, 22.4.00. 
• Download the deployment manager BMC_Helix_Platform_Services_for_Service_Management_Version_22.4.00.zip and extract the files to the new working directory.
  To download the deployment manager from EPD, see Downloading-the-installation-files. 
  The ZIP file contains the following files:
  
  • helix-on-prem-deployment-manager-22.4.00.sh—This file contains the deployment manager.

• • hotfix-22.4.00.001-10.tar.gz—This file contains the 22.4.00 hotfix 1 artifacts.
• Download the Cleanup_Bmchelixlogging.shscript.

• Back up all the PersistentVolumeClaim (PVC) data. 

  Important

  BMC does not have a recommendation for this step. You can use your preferred method to back up the PVC data.

• Set up the infra.config file.
• Clean up BMC Helix Logging namespace.

To set up the infra.config file

1. In your working directory, navigate to the configs/infra.config file.

2. In the infra.config file, set the following parameter values:

   Important

   Do not copy the older version of the infra.config file. Use only the new infra.config file to set the parameters. 

   Parameter	Example value	Description
   IMAGE_REGISTRY_HOST	containers.bmc.com (or local repo if copied down)	Image registry from where the nodes on the cluster download the images. If you have synchronized the images to a local Harbor registry, make sure the Harbor registry is set up with HTTPS.
   IMAGE_REGISTRY_USERNAME	abc@bmc.com	User name to log in to BMC DTR. If you use a local Harbor registry to synchronize with BMC DTR, specify the user name to log in to your local registry.
   ENVIRONMENT	poc	Type of environment such as poc, dev, and qa. Do not use special characters for the environment value. You can use the same environment value while performing the BMC Helix Service Management installation.
   NAMESPACE	dark-helmet	Namespace in which to install the services. You must have separate namespaces to install BMC Helix Platform services and BMC Helix Innovation Suite  and applications.
   LB_HOST	host-india-app.mydomain.com	Host for load balancer for BMC Helix Innovation Suite. Specify the BMC Helix Innovation Suite URL.
   LB_PORT	443	Port for load balancer.
   TMS_LB_HOST	tms-private-poc.mydomain.com	Host for tenant management system. Specify the host of the load balancer that points to the tenant management system service.
   Domain	mydomain.com	Domain name of the Load Balancer
   MINIO_LB_HOST	minio-private-poc.mydomain.com	URL for Minio storage.
   MINIO_API_LB_HOST	minio-api-poc.mydomain.com	Use MinIO API ingress to create buckets by using the command line.
   CLUSTER_TYPE	""	Either values openshift or ocp for OpenShift. If CLUSTER_TYPE is not set to openshift or ocp, cluster type is treated as a Kubernetes cluster.
   COMPANY_NAME	photon2	Parameter in the tenant URL formation like $COMPANY_NAME-$TENANT_TYPE-$ENVIRONMENT.$DOMAIN Do not use special characters for the Company name. COMPANY_NAME value is used to generate the tenant URL.
   TENANT_EMAIL	pqr@mycompany.com	Email address of the admin user of initial tenant.
   TENANT_FIRST_NAME	TestName	First name of the admin user for initial tenant.
   TENANT_LAST_NAME	TestLastName	Last name of the admin user for initial tenant.
   TENANT_TYPE	private	The tenant type. Valid values: public or private This will be used in the tenant URL and BMC Discovery appliance URL formation. For BMC Helix Operations Management : $COMPANY_NAME-disc-$TENANT_TYPE-$ENVIRONMENT.$FQDN  $COMPANY_NAME-$TENANT_TYPE-$ENVIRONMENT.$FQDN Example: acme-private-poc.acme.com
   TENANT_DOMAIN_HOST	acme-private-poc.acme.com	The tenant domain. This URL is for BMC Helix Portal. You must enter the parameter value in the following format:$TENANT_NAME-$TENANT_TYPE-$ENVIRONMENT$.DOMAIN
   COUNTRY	"United States"	Matches the value in the OS locale. Important Add the country name within quotation marks. For example:"India" Do not use abbreviation in country names. Click  here to view a list of the supported country names.
   NFS_STORAGE_CLASS	""	Blank "" This parameter is not required for BMC Helix Service Management.
   SMTP_HOST	mailhost.mycompany.com	SMTP host name of IP address accessible from cluster SMTP parameters are required for the emails that are sent to the administrator for tenant activation after the BMC Helix Platform deployment is complete. All SMTP mail servers are supported. To use a temporary SMTP server to receive BMC Helix Platform services installation emails, see the knowledge article000396217.
   SMTP_PORT	25	An integer value for the port of the SMTP server.
   SMTP_USERNAME	abc@mycompany.com	User name to connect to the SMTP server. If SMTP_AUTH value is set to NONE, keep the SMTP_USERNAME and SMTP_PASSWORD values blank as shown below: SMTP_USERNAME="" SMTP_PASSWORD=""
   SMTP_FROM_EMAIL	helix-rd@mycompany.com	A valid email ID for the From address in all emails.
   SMTP_TLS	"false"	The SMTP server TLS. The value can be true or false. If not in use, specify the value as false.
   SMTP_AUTH_DASHBOARD	true	SMTP authentication. Valid values are True or false.
   SMTP_AUTH	PLAIN	SMTP authentication type. One of the following values: PLAIN This value is case sensitive. If you set the value as PLAIN, it is mandatory to set valid values for SMTP_USER and SMTP_PASSWORD. LOGIN This value is case sensitive. If you set the value as LOGIN, it is mandatory to set valid values for SMTP_USER and SMTP_PASSWORD. NONE This value is case sensitive. Use this value when you want to skip SMTP authentication. If you set the value as NONE, set the user name and password values as shown below: SMTP_USERNAME="" SMTP_PASSWORD=""
   OPS_GROUP_EMAIL	ops-grp@mycompany.com	ops email address. All emails related to tenant activities such as tenant creation, tenant registration, and tenant offboarding are sent to your organization's operations team.
   APPROVAL_GROUP_EMAIL	grp-rd@mycompany.com	Email address for approval. When a new tenant is created, an email is sent for tenant approval to this email group.
   PG_STORAGE_CLASS	ceph-block-storage	Storage class used. Usually there is one Storage class configured for all the infra services. Repeat the same value in that case.
   VMSTORAGE_STORAGE_CLASS	onprem-storage	Storage class for VictoriaMetrics.
   VMAGGSTORAGE_STORAGE_CLASS	onprem-storage	Storage class for VictoriaMetrics.
   ES_MASTER_STORAGE_CLASS	block-store-class	Storage class for Elasticsearch master nodes.
   ES_DATA_STORAGE_CLASS	block-store-class	Storage class for Elasticsearch data nodes.
   MINIO_STORAGE_CLASS	onprem-storage	Storage class for Minio.
   EFS_STORAGE_CLASS	""	Blank ""
   REDIS_HA_GLOBAL_STORAGECLASS	block-store-class	Storage class for REDIS.
   KAFKA_STORAGECLASS	block-store-class	Storage class for Kafka.
   ESLOG_MASTER_STORAGE_CLASS	block-store-class	Storage class for Elasticsearch log.
   ESLOG_DATA_STORAGE_CLASS	block-store-class	Storage class for Elasticsearch log.
   MINIO_STORAGE_CLASS	acme-block-storage	Storage class for MinIO. Usually, a single storage class by using block storage is configured for all the infra services. Repeat the same value if configured in this manner.
   AIOPS_STORAGE_CLASS	""	Blank ""
   CUSTOM_CA_SIGNED_CERT_IN_USE	false	Flag to use self-signed or custom CA certificate. Default value is false. If you are using a self-signed or custom CA certificate, set the value to true. Important: If you are using a self-signed or custom CA certificate, make sure that you use the same custom certificate during BMC Helix Platform and BMC Helix Service Management installation.
   OPT_STORAGE_CLASS	""	Blank ""
   REPOPV_MOUNT_PATH	""	Blank ""
   MIGRATORPV_MOUNT_PATH	""	Blank ""
   ETLPV_MOUNT_PATH	""	Blank ""
   CLIENT_ROOT_CERT	""	Blank ""
   SMART_SYSTEM_USERNAME	""	Blank ""
   INGRESS_CLASS	nginx	Ingress class used while deploying Ingress controller. Change if multiple ingress controllers are on the cluster. If you have more than one ingress controllers in your cluster, use INGRESS_CLASS to specify the ingress class name that you want to use.
   INGRESS_API_VERSION	true	Flag to indicate if the Ingress controller version is 1.2.0 or higher. True if your Ingress controller version is 1.2.0 or higher.
   HELM_BIN	/usr/local/bin/helm	Absolute path of the HELM binary.
   KUBECTL_BIN	/usr/bin/kubectl	Absolute path of the kubectl binary.
   OC_BIN	/usr/local/sbin/oc	Cluster type. Set if CLUSTER_TYPE is openshift or ocp.
   KIBANA_LB_HOST		The BMC Helix Logging ingress uses this value. This value depends on the self-signed, CA-signed certificate, or custom certificate. If the value of the CUSTOM_CA_SIGNED_CERT_IN_USE parameter is true, use the DNS configured for the self-signed certificate. If the value of the CUSTOM_CA_SIGNED_CERT_IN_USE parameter is false, use the DNS configured for the CA-signed certificate. If the value of the CUSTOM_CA_SIGNED_CERT_IN_USE parameter is true, use the DNS configured for the self-signed certificate.
   RSSO_CUSTOM_JAVA_KEYSTORE_IN_USE	true	Flag to enable custom JAVA keystore for RSSO SAML keystore. If you want to use custom JAVA keystore for RSSO SAML keystore configuration, set the RSSO_CUSTOM_JAVA_KEYSTORE_IN_USE variable to true. Perform the following steps: Set the RSSO_CUSTOM_JAVA_KEYSTORE_IN_USE variable to true. Rename the java keystore file to rsso_custom_java_keystore. Save this file in the commons/certs directory. The path of this file would be: commons/certs/rsso_custom_java_keystore The commons/certs/rsso_custom_java_keystore file will be mounted in the RSSO container at the following location: /etc/rsso_custom_java_keystore
   RUN_AS_USER	null	The security context that the components will use. This variable is considered only if the value of the CLUSTER_TYPE variable is openshift or ocp. For the Kubernetes cluster, use the following value: null Set the correct context for this variable according to the OpenShift namespace. For example, in OpenShift, run the following command to get the ID range: oc describe namespace <namespace-name>
   FS_GROUP	null	The security context that the components will use. This variable is considered only if the value of the CLUSTER_TYPE variable is openshift or ocp. For the Kubernetes cluster, use the following value: null Set the correct context for this variable according to the OpenShift namespace. For example, in OpenShift, run the following command to get the ID range: oc describe namespace <namespace-name>
   RUN_AS_GROUP	null	The security context that the components will use. This variable is considered only if the value of the CLUSTER_TYPE variable is openshift or ocp. For the Kubernetes cluster, use the following value: null Set the correct context for this variable according to the OpenShift namespace. For example, in OpenShift, run the following command to get the ID range: oc describe namespace <namespace-name> The command will change for each namespace.
   OPT_FSGROUP	""	Blank "" Leave blank. This parameter is not required for BMC Helix Service Management.
   ML_FSGROUP	""	Blank "" This parameter is not required for BMC Helix Service Management.
   CUSTOM_SERVICEACCOUNT_NAME	helix-onprem-sa	Custom service account name. Default value is helix-onprem-sa. If there are no permissions to create ServiceAccount, Role, RoleBinding, perform the following steps: Replace the default value of helix_onprem_sa with the required value. Create a role and rolebinding from the commons/yaml_files/role_rolebinding.yaml file. Create a ServiceAccount from the commons/yaml_files/serviceAccount.yaml file.  If there are permissions to create ServiceAccount, Role, RoleBinding, retain the CUSTOM_SERVICEACCOUNT_NAME value as helix-onprem-sa as shown below: CUSTOM_SERVICEACCOUNT_NAME=helix-onprem-sa

   Important

   Make sure that the ENVIRONMENT variable has the same value that was used in the 22.2.01 version of the product.
   Make sure that the INGRESS_CLASS variable has the same value that you configured while installing Kubernetes. 
   

3. Save the file.

To cleanup BMC Helix Logging namespace

1. Back up logs from Kibana into an excel file.
   
   1. Log in to Kibana dashboard.
   2. On the Kibana home page, click .
   3. In the Analytics area, click Discover.
   4. Use a filter to refine your search, click Refresh query, and save your search.
   5. Select Share > CSV Reports.
   6. Click Generate CSV.
      
   7. When prompted, click Download report and save the report.
2. Cleanup BMC Helix Logging namespace.
   
   1. Log in to the system from where you installed BMC Helix Platform services earlier.

   2. Export helm and Kubectl binary path.
      For example:

      export HELM_BIN=/usr/local/bin/helm
      export KUBECTL_BIN=/usr/bin/kubectl
   3. Copy the ./Cleanup_Bmchelixlogging.sh file to the system.

   4. Clean up the bmc-helix-logging namespace by using the following command: 

      ./Cleanup_Bmchelixlogging.sh <bmc-helix-logging namespace>

      <bmc-helix-logging namespace> is the name of the bmc-helix-logging namespace that you used in the 22.2.01 version.

      The Cleanup_Bmchelixlogging.sh script deletes the PVC, pods, and jobs from the BMC Helix Logging namespace.

Task 1: To migrate data from Bitnami PostgreSQL to Patroni PostgreSQL

1. Make sure that you have the exec permission on the existing Bitnami PostgreSQL pods.

1. In the configs/deployment.config file, update the following parameters:

   Parameter	Value
   INSTALL_MODE	upgrade
   _PTPOSTGRESS	yes
   INFRA	yes

   Important

   Retain all other values as no in this file.

2. In the commons/certs/secrets.txt file, update the default values of the parameters.
   These values can be the same as added in the previous deployment.

3. Deploy the Patroni PostgreSQL server by running the following command:

   ./deployment-manager.sh

4. On the system where you deployed the Patroni PostgreSQL server, navigate to the utilities/migration directory by using the following command:

   cd helix-on-prem-deployment-manager/utilities/migration
5. In the dbmigration.sh script, specify the Bitnami PostgreSQL password and Patroni PostgreSQL password.
   Bitnami PostgreSQL password is your base PostgreSQL database password and Patroni PostgreSQL password is your new PostgreSQL database password.
   The following image shows an example:
   

1. Run the following command to start the migration:

   ./dbmigration.sh

2. (Optional) View the data migration logs by running the following command:

   ls helix-on-prem-deployment-manager/utilities/migration/logs

   Example output:

   migration.log                 migration.log.20230105122017  migration.log.20230105122415  migration.log.20230105122440  migration.log.20230105123110
   migration.log.20230105121719  migration.log.20230105122308  migration.log.20230105122432  migration.log.20230105122949  migration.log.20230105123850

Task 2: To upgrade BMC Helix Platform services

1. Navigate to your working directory.

2. In the configs/deployment.config file, set the following parameter values:

   Parameter	Value
   INSTALL_MODE	upgrade
   _PTPOSTGRESS	no
   _KAFKA	yes
   _VICTORIAMETRICS	yes
   _RSSO	yes

3. For the product services to which you are licensed to, set the value to yes. For all other services, set the value to no.
   To find the product services to which you are licensed to, see the deployment.config file from the version 22.2.01.

From the working directory, run the deployment manager to upgrade BMC Helix Platform services by using the following command:

After the upgrade is complete, you will get the following message:
Completed Helix On-prem Installation.

(Optional) To view the logs during the upgrade, run the following command:

Step 4: To scale down the Bitnami PostgreSQL pods

After the upgrade is complete, scale down the Bitnami PostgreSQL pods by using following commands:

To apply the hotfix

1. Log in to the controller or bastion machine from where the Kubernetes cluster is accessible.
2. Create a new directory; for example, ITOM_HotFix_22.4.00.001. 
3. Copy the hotfix-22.4.00.001-10.tar.gz file that you downloaded from EPD to the new directory.

4. Extract the hotfix-22.4.00.001-10.tar.gz file by using the following command:

   tar xvf hotfix-22.4.00.001-10.tar.gz
5. Navigate to the hotfix directory.
6. If you are using a local repository for accessing container images, make sure that you synchronize the images listed in the hotfix/new-image-list.txt to the local repository.
7. If you have not installed BMC Helix ITSM Insights, in the hotfix/new-service-list.config file, modify the ITSMINSIGHT_SERVICES parameter value to blank, ITSMINSIGHT_SERVICES=
8. Run the hotfix script file hf_

9. Run the hotfix script file hf_script.sh by using the following command:

   bash hf_script.sh <full path of the helix-on-prem-deployment-manager directory> 

   Replace <full path of the helix-on-prem-deployment-manager directory> with the full path of the directory where you installed BMC Helix Platform services 22.4.
   Example:

   bash hf_script.sh /data/22.4.00/helix-on-prem-deployment-manager

   The hf_script.sh script creates a copy of helix-on-prem-deployment-manager in the path that you specified in the command and the directory is named helix-on-prem-deployment-manager_HF1.For example, a new directory /data/22.4.00/helix-on-prem-deployment-manager_HF1 is created. No changes are made to the original directory helix-on-prem-deployment-manager.
   The hf_script.sh script installs the following chart:

   • tas chart with version 542
   • ade-arservices-register chart with version 10
   • aif-core-service chart with version 22.4.00.001-8
10. Reactivate the BMC Helix ITSM Insights service.
    1. Log in to the tctl utility.
       1. From the folder where you downloaded the tctl utlity, edit the config file and add the details according to your environment.

       2. From the directory where the config file is located, run the following command:

          tctl login
       3. On theBMC Helix Single Sign-On authentication page, enter the user name and password and authenticate.
          The user name is the admin. This user is a local user in the tmsrealm realm, and is different from the BMC Helix Single Sign-On admin user. 
       4. Click Log in.

    2. Create the inactive.json file by using the following details:

       {
          "status": "INACTIVE"
       }

    3. Create the active.json file by using the following details:

       {
         "status": "ACTIVE",
         "service_url": "<BMC Helix Innovation Studio host>:<port>",
          "metadata": {
                           "itsm_rest_url": "<BMC Helix Innovation Studio host>:<port>",
                           "api_endpoint": "<BMC Helix Innovation Studio host>:<port>",
                           "webhook_activation_image": "<Webhook activation image path>"
                       }
       }

    4. Inactivate the BMC Helix ITSM Insights service by using the following command:

       tctl update tenant-service <tenant id> -i <ITSM_INSIGHTS service id> -f inactive.json

    5. Activate the BMC Helix ITSM Insights service by using the following command:

       tctl update tenant-service <tenant id> -i <ITSM_INSIGHTS service id> -f active.json

Where to go from here

Upgrading-BMC-Helix-IT-Service-Management-to-21-3-06