Upgrading BMC Helix IT Operations Management from version 23.1.02 to 23.4.00


Upgrade BMC Helix IT Operations Management ( BMC Helix ITOM ) from version 23.1.02 to 23.4.00.

You might experience downtime during the upgrade, which is to be expected but does not affect the upgrade.

image-2023-10-16_15-33-0-1.png

Before you begin

Make sure you have performed all the steps listed in the Preparing for upgrade topic.


To clean up EFK

Important

Perform this task if you had deployed BMC Helix Logging in the 23.1.02 version of BMC Helix ITOM.


BMC Helix Logging used Elasticsearch, Fluent Bit, and Kibana (EFK) to collect, store, and view log data:
 In 
BMC Helix ITOM version 23.1.02, FluentBit was a DaemonSet. Starting with BMC Helix ITOM version 23.4.00, FluentBit is a deployment. Due to this change, you must perform the following steps to clean up Elasticsearch, Fluent Bit, and Kibana (EFK).

  1. Back up logs from Kibana into a Microsoft Excel file.

    See, To export logs to a csv file.
    1. On the Kibana home page, click 23.1.02_FlentBit_Hamburger.png.
    2. In the Analytics area, click Discover .
    3. Use the filter to refine your search, and then click Refresh query .
      23.1.02_FlentBit_FiltersRefreshQuery.png

    4. Save your search.
    5. Select Share > CSV Reports.
      23.1.02_FlentBit_CSVReport.png
    6. Click Generate CSV.
      • The system displays a similar message:
        23.1.02_FlentBit_Message1.png
    1. When prompted, click Download report and save the report.
      23.1.02_FlentBit_Message2.png
  2. Run the following commands to export the kubectl and Helm binary:

    export KUBECTL_BIN=<path of KUBECTL_BIN>

    export HELM_BIN=<path of HELM_BIN>

    Copy the path of KUBECTL_BIN and HELM_BIN from the infra.config file. See, Configuration-file-settings.

  3. Download the efk-cleanup.sh script.
  4. Run the efk-cleanup.sh script to clean up the EFK pods:

    ./efk-cleanup.sh <namespace>

    <namespace> is the namespace where you deployed BMC Helix IT Operations Management.

    This script removes the Kubernetes objects such as EFK pods, PVCs, secrets and so on.


To delete the helm revision secrets of the Kubernetes CronJob

Important

Perform this task only if you upgraded to Kubernetes version from 1.24 to 1.25 or 1.26.

  1. Run the following commands to export the kubectl and Helm binary:

    export KUBECTL_BIN=<path of KUBECTL_BIN>

    export HELM_BIN=<path of HELM_BIN>

    Copy the path of KUBECTL_BIN and HELM_BIN from the infra.config file. See, Configuration-file-settings.

  2. Download the Cronjob_helm_revision_delete.shscript.
  3. Copy the Cronjob_helm_revision_delete.sh script to helix-on-prem-deployment-manager folder.
  4. Run the Cronjob_helm_revision_delete.sh script to delete the helm revision secrets of the CronJob:

    ./Cronjob_helm_revision_delete.sh <namespace>

    <namespace> is the namespace where you had deployed BMC Helix IT Operations Management .


To upgrade BMC Helix IT Operations Management


Important

Contact BMC support if you changed the default BMC Helix Single Sign-on (RSSO) password while installing BMC Helix ITOM and encounter the following error:
WARNING: RSSO Login NOT successful.

  1. In the new_working_directory/ configs/deployment.config file, make sure that your deployment size is the same as your previous deployment size; for example, small, compact, medium, or large.
    To confirm your deployment size, see the value set for the DEPLOYMENT_SIZE parameter in the configs/ deployment.config  file in BMC Helix ITOM version 23.4.00.
  2. In the new_working_directory/ configs/deployment.config file, update the following values:
    1. Set the value of INSTALL_MODE to upgrade.
    2. Set the values of the following services to yes.
      • _INFRA
      • _PTPOSTGRES
      • _KAFKA
      • _REDIS
      • _RSSO
      • _VICTORIAMETRICS
      • _ELASTICSEARCH
      • _MINIO
    3. For all the BMC Helix ITOM applications for which you are licensed, set the value to yes; for all other services, set the value to no.
      For example, if you are only licensed to use BMC Helix Operations Management and BMC Helix Continuous Optimization:
      • To use BMC Helix Operations Management, set value of MONITOR to yes.
        Optionally,
        • If you want to use AIOps, set AIOPS_SERVICES to yes.
        • If you want to use Log Analytics, set INTELLI_INT_SERVICES to yes.
      • To use BMC Helix Continuous Optimization, set the value of OPTIMIZE to yes

        Best practice
        To see the product services for which you are currently licensed, see the configs/deployment.config file in BMC Helix ITOM version 23.1.02.

  3. If you had configured data backup and restoration in the BMC Helix IT Operations Management version 23.1.02, the following message is displayed:
    Backup is enabled. Executing upgrade would disable the current backup configuration. 
    Post successful upgrade, You will need to manually enable the DR.
    Would you like to continue? [y|n]

    Enter y to proceed with upgrade.
    You can configure disaster recovery after the upgrade.  

  4. From the new working directory ( new_working_directory ) run the deployment manager to upgrade BMC Helix IT Operations Management:

    ./deployment-manager.sh

    After the upgrade is complete, you will get the following message:
    Completed Helix On-prem Installation.

  5. (Optional) To view the logs during the upgrade, run the following command:

    tail -f logs/deployment.log
  6. If you had deployed BMC Helix IT Operations Management in a multitenant environment, follow the steps to restore the smart-graph-tenant-config configmap and its content:
    1. Copy the contents from the smart-graph-tenant-config configmap file that you backed up before the upgrade.
    2. Run the command to open the smart-graph-tenant-config configmap:

      kubectl -n <namespace> edit cm smart-graph-tenant-config

    3. Paste the contents that you copied from the backed-up smart-graph-tenant-config configmap file in the new smart-graph-tenant-config configmap and save it.
    4. Run the following command to restart the smart graph controller pod: 

      kubectl rollout restart deploy/smart-graph-controller -n <namepsace>

Performing post upgrade task

Important

Perform this task only if you upgraded to Kubernetes version 1.25 or higher.

After the upgrade is complete, the namespace pod security admission must be changed to restricted.

Run the following command to change the cluster level pod security admission:

  kubectl label --overwrite ns <namespace> \
 pod-security.kubernetes.io/enforce=restricted \
  pod-security.kubernetes.io/enforce-version=latest \
  pod-security.kubernetes.io/warn=restricted \
  pod-security.kubernetes.io/warn-version=latest \
  pod-security.kubernetes.io/audit=restricted \
  pod-security.kubernetes.io/audit-version=latest

<namespace> is the namespace where you had deployed 

BMC Helix IT Operations Management

 .

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*