How to apply rolling patch updates with no downtime

This topic walks you through the process of how to apply rolling patch updates to your infrastructure with no downtime using BMC Server Automation. This example uses a Patch Catalog smart group to patch the servers behind the load balancer so that there is no downtime to the business services that rely on the web servers.

It includes the following sections:

The video at right from BMC Communities (8:25) provides a quick demonstration of the process.

  https://www.youtube.com/watch?v=Fp-sBE-7PmA&feature=youtu.be

Overview

In this example, the environment has a simple structure: one load balancer (called HA Proxy in this example) and three web servers (running Red Hat Linux) behind the load balancer.

The load balancer is used to distribute requests to the three web servers. This example shows how you can patch the servers behind the load balancer so that there is no downtime to the business services that rely on the web servers.

What do I need to do before I begin?

Prior to performing the steps in this example, you must have:

To create and run the Patching Job

StepExample screen
  1. Expand the Depot folder and navigate to a subfolder where the previously created patch catalog is located.
  2. Right-click the patch catalog smart group and select Analyze patches.

Define the general settings on the New Red Hat Patching Job General panel.

    1. In the Name field, provide a name for this job.
    2. Verify that the value in the Save in field is where you want to store this job. You can browse to another location if necessary.
    3. The Specify a Catalog field is set to the catalog in which you created the patch catalog smart group.
    4. Click Next

The options on the Analysis Options panel are pre-selected for you.

  • Update mode, which checks for outdated RPMs based on what is in the catalog.
  • The include list, which contains the patch catalog smart group that includes a limited a specific Red Hat errata.

Click Next.

On the Remediation Options panel:

  1. Ensure that the Create remediation artifacts field is cleared. In this example, we are only analyzing patches. You can create remediation artifacts later.
  2. Click Next.

On the Targets panel, select the servers that are the targets of this Linux Patching Job.

  1. In the left panel, navigate to a server smart group or to an individual server. In the example, the smart group is WebServers.
  2. Click the > button to move the selection from the left panel to the right panel.
  3. Continue to select groups or servers until you have a complete list of servers for the analysis.
  4. Click Next
On the Default Notifications panel, click Next, as we are not configuring any default notification settings. The defaults are used for all runs of this job unless you override them with notification settings for a scheduled job.

On the Schedules panel, you can choose to execute it immediately or you can set up an execution schedule for the job.

For this example, we want to run the job immediately, but do not want to create a schedule.

  1. Select Execute job now to indicate the job should run as soon as you finish the wizard. 
  2. Click Finish to complete the patching wizard and create and execute your job.

Once the job complete, view the results of the patching job:

  1. Right-click the Patching Job in the folder under the Jobs folder.
  2. Select Show Results.
    The job results appear in the pane at right. 

As you can see, the job successfuly completed for all three of the web servers.

The next step is to see if the patch was missing, and if so to remediate the problem.

To apply the updates

StepExample screen

In the BSA console, under Jobs, navigate to the folder for your Windows Patching Job. The examples in this procedure use the folder structure Jobs > Patch Analysis Jobs > Windows Patch Analysis.

  1. Right-click the Patching Job in the folder under the Jobs folder.
  2. Select Show Results.
  1. In the Windows Patching Job results, click the Server View
    The right pane shows the server name, and the number of missing erratas. (You can also deploy patches and hotfixes from the object view. See Choosing remediation targets and patches.) The next step will be to deploy the patch to the servers.
  2. Right-click Successful Targets and select Remediate All Servers.

The New Patch Remediation Job wizard opens. The Remediation Job creates the following items:

  • A Deploy Job for each server specified in the Remediation Job. A Deploy Job updates a server with the patch software.
  • A Batch Job that you can use to run all of the Deploy Jobs.
  • BLPackages, which are software packages containing the patches needed for the Deploy Jobs.

On the General panel:

  1. In the Name field, enter a suitable job name. This example uses the hotfix number.
  2. In the Save in field, enter or browse to a location in the Jobs folder where you want to save this Remediation Job.
  3. Click Next

On the Remediation Options panel:

  1. In the Package name prefix field, type a suitable prefix for package names. The default is the Remediation Job name.
  2. In the Save package(s) in field, type or browse to the location in the Depot folder where you want to save the software packages (BLPackages) that this job creates.
  3. In the Save Batch/Deploy Job(s) in field, type or browse to the location where you want to save the Batch Job and the Deploy Jobs that this job creates.
  4. Usually, you can leave the ACL Policy for Package(s) Deploy Job(s) field blank.
  5. Click Deploy Job Options

On the Deploy Job Options panel, on the Job Options tab:

  1. In the Logging Level field, select an option. This examples uses the default of Errors and warnings.

  2. For Flow control, this example uses By server.
  3. From the Reboot Options drop-down list, select an option. (See Assigning default values for Deploy Jobs for explanations of these options.) This example uses the Ignore item defined reboot setting and reboot at the end of job reboot option. Typically, this option is appropriate.
  4. Accept the defaults for other fields. 

This example uses the default settings for the Deploy phases tab. For information about these options, see Deploy Job - Phase Options.

However, we do want to add pre-execution and post-execution commands. To do so, click Pre/Post Commands.

In this example, we add a pre-execution command to remove the HA proxy server prior to the patch being applied, and then add it back after job execution is complete.

After adding the commands, click OK to close the Pre/Post Commands dialog, and then click OK again to close the Deploy Job options dialog.

On the Phases and Schedules Tab:

  1. Select Do not execute.
  2. Click OK to return to the Remediation options panel.
  3. Click Deploy Job Properties.

On the Deploy Job Properties panel:

  1. Set the Commit Max Parallel Targets option to 1. This ensures that the patch will be applied to one server at a time.
  2. Accept the defaults for the other options.
  3. Click OK to return to the Remediation Options panel.
  4. Click Next.

On the Job Run Notifications, click Next.

Bypassing this panel will use the default notifications that were set up in the Patch Analysis Job in the previous walkthrough.

  1. On the Schedule panel, select Execute job now.
  2. Click Finish to accept the default settings for the remaining two wizard panels, Properties and Permissions.

The Remediation Job creates the Deploy Job and a Batch Job required to apply the patch.

The executing job appears in the Tasks in Progress view on the console. After the Remediation Job executes, you can view its results under the original Patching Job with which it is associated.

After the Remediation Job completes, right click the Remediation run and select Open Generated Batch Job.

On the Batch Job Options panel, select Execute Jobs Sequentially under Execution options. In case the Remediation Job creates multiple Deploy Jobs, this option ensures that the Deploy Jobs are executed sequentially by the Batch Job, so that one server at a time is patched.

Save and close the job.

  1. Locate the new Batch Job in the job folder.
  2. Right click the job, and select Execute.
  1. In the Object Explorer locate the folder in which the Remediation Job stores the Batch Job and the Deploy Jobs that it created, (This location was specified in step 4 above).
  2. Right-click the job and select View Results.
  3. Review the status of the patch deploy phases.

As you can see in the example, the first server has been patched and rebooted, and the patch is being applied to the second server.

The servers will be brought down for patching one at a time, and any incoming requests are routed to online the servers.

Once you see that all three servers are up and running, then the patching process is complete, without incurring any downtime.

Where to go from here

To review other specific patch management examples, see Example patching scenarios

Was this page helpful? Yes No Submitting... Thank you

Comments