Default language.

Troubleshooting upgrade issues


The preferred method for upgrading a Patch, Service Pack, or version is from the appliance’s UI. For more information, see Performing-the-upgrade.

Upgrade stops with "Entering emergency mode" message

Situation

Very rarely the upgrade to an Oracle Linux 9 based appliance (BMC Discovery 23.3 and later) stops, and the console displays output that looks like the following screenshot:

screenshot-1.png

  In particular, in the screenshot, note:

  1. The error RPM: error: db5 error(-nnnnn) from dbenv->open at the top of the console.
  2. The "Entering emergency mode" error at the bottom.

Root Cause

The problem occurs when the RPM database's lock files become desynchronised during the installation of packages. The desynchronization causes this phase of the upgrade to abort.

The problem is not a cause for concern. The upgrade from a version of BMC Discovery earlier than version 23.3 to BMC Discovery 23.3 or later requires an operating system upgrade to Oracle Linux 9. This part of the upgrade is safe: the process creates a sandboxed copy of the operating system files that it upgrades in isolation, and it only copies those files into the actual operating system if it is successful. The failure means that the sandbox has become corrupted. Continuing the upgrade deletes the corrupted sandbox and creates a new one, resolving the problem.

Resolution

To enable the upgrade to retry the phase and continue:

  1. At the console, type exit to exit emergency mode.  The host reboots.
  2. Once rebooted, log in to the machine as the tideway  user.
  3. Run the following command:
    sudo rm /usr/tideway/var/upgrade/extract/ol9_upgrade/last_script
    Removing this file clears the error condition caused by the failure and permits the upgrade process to try again.
  4. Reboot the machine with the following command: sudo reboot now 
    When the host reboots, the upgrade process retries the failed phase.

Steps after upgrade

See post upgrade steps in Upgrading BMC Discovery.

As well as the notes on that page you should refer to the postupgrade_Vnn_TODO.log written out by the upgrade. This contains tailored advice of the tasks that must be completed on that particular appliance and these must be completed for correct future behavior.

Recovering from a lost connection using screen

If you lose the connection to the appliance during the upgrade and you have used screen, you can reconnect to the appliance and recover the virtual terminal running the upgrade. To do this:

  1. Reconnect to the appliance and login as the tideway user.
  2. List the current screen sessions. Enter:

    [tideway@appliance01 ~]$ screen -ls
    There is a screen on:
    23274.pts-0.appliance01 (Detached)
    1 Socket in /var/run/screen/S-tideway.

    You can re-attach to it with a simple command:

    [tideway@appliance01 ~]$ screen -r

    The virtual terminal is recovered and you can see how the upgrade is progressing.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*