Implementation process for patch management

Patch metadata

All patch vendors publish some kind of information that basically says that a given patch is needed based on operating system, application version, or existing patches. 

We call this information the patching metadata, because it tells us information about the patch: specifically where the patch applies. In BSA, this metadata is stored in the patch catalog. 

Patch catalog

A patch catalog provides a place to store metadata about patches. Patch catalogs store vendor patch metadata, and are the source of information used for analysis, downloading patch payloads, and deployment. 

Patch catalogs can be designed for specific needs. For example, a patch catalog can used for a particular operating system, such as Microsoft Windows 2008 or 2012. With well designed patch catalogs, it is easier to select the patches that should be used when evaluating the patch configuration of a particular server.

Patching Job

You run a Patching Job to identify missing patches on your servers. A Patching Job uses the information contained in the patch catalog to analyze one or more target servers for patch compliance. 

There are three parts to the patch process:

• Analysis — The Patch Analysis Job analyzes the configuration of target servers and determines the required patches.
• Remediation — The Patch Remediation Job downloads the payload from the vendor sites to the Patch Repository, packages the payload as a BLPackage, and creates a Deploy Job to apply the patches.
• Deploy — The Deploy Jobs created during remediation are then run to deliver the required patches to targets requiring remediation. 

You can choose to run only the analysis part of a Patching Job, and then run remediation later, or you can run remediation immediately after the analysis.  

Standard Patching

Standard patching refers to applying critical or important vendor patches. The standard patching process normally involves either reusable or purpose-built Patch Smart Groups. A typical patching process includes:

• Surveying large groups of servers, using a standard policy, for reporting and assessment purposes. This is done by setting up and executing a Patch Analysis Job. The results of the job should give you a good idea of the amount of work required to patch the environment (do many servers require more than 10-15 patches?), and the amount of time estimated. You don't typically generate the remediation packages with this job.
• Setting up smaller patching remediation jobs, usually aligned by specific criteria, such as maintenance window, business unit, change control/request or server owner groups, or some combination of those. These patching jobs will generate remediation packages, and those remediation jobs will need to execute all within a particular maintenance window, or when a particular change is approved to run, and so on, depending on the criteria.
• Once staged and executed, the patching job will usually be re-run, for one of the following reasons:
  • 
    
    • Reporting purposes
    • To either certify that the group of servers no longer need patches
    • To do a second-pass deployment for any “still missing” patches or patches that didn’t deploy the first time.

Some companies perform this cycle twice, with the third Patching Job performing the verification/reporting, as it is sometimes difficult to achieve 99% compliance with one remediation pass. A typical full patch management lifecycle is 15-30 days for the full production environment, with another 15 days up front for pre-production.

Emergency patching

Also known as zero day or critical patching, emergency patching refers to either one specific patch, or a list of similar patches. The process is actually a lot like standard patching, but the change controls tend to be broader, and more servers tend to get remediated within the same job (or by using fewer jobs than the standard patch process).

Since this patch list is highly focused, testing overhead tends to be smaller, and the windows into which the patches are deployed tend to be much narrower. A typical window for emergency patching could be 10 days for the entire environment.

Patch Compliance dashboards are critical for emergency patching, and since these usually still need to run in some kind of seven day maintenance windows, the capability to quickly understand deployment success rates and patch compliance across specific groups or populations is very important.

Deployment success rates and patch compliance for emergency patching are mostly measured using job results, and also with trailing reports in BMC Decision Support for Server Automation.

Windows

Any Windows or Linux server

Red Hat Enterprise Linux (RHEL)

Based on the version of RHEL patching you are performing, the supported patch repository platforms are as follows: 

For RHEL 6 or earlier: Any RPM-based Linux server

For RHEL 7: Red Hat Enterprise Linux 6 or Red Hat Enterprise Linux 7

Oracle Enterprise Linux

Any RPM-based Linux server

SUSE Linux

The repository server can be any Linux server. However some SUSE-specific patches need to be stored only on a SUSE repository.

BMC strongly recommends that you use a SUSE Linux server for storing the patch repository.

AIX

Any AIX server

Solaris

Any Windows or Linux server

Note: If you are using Solaris 11 patches, you can only use a Solaris 11 server for storing the patch repository.

Ubuntu

Any Windows or Linux server

Debian

Any Windows or Linux server

Cent OS

Any Linux server

Fujitsu

Any Windows or Linux server

HP-UX

If you are using the offline patch downloader you can use any Windows or Linux server to store the patch repository. However, if you are using the VPC method you must store the patch repository only on a HP-UX server.

Set up roles and permissions

To create or update a catalog, you must be assigned a role that includes the necessary permissions. You can assign permissions to one role or divide them between several roles.

Although this process is not essential for patch management, BMC always recommends that you grant users the minimum set of permissions needed to perform actions. If you do not set up a patching administrator with a limited set of permissions, a superuser such as the BLAdmins role must perform patch management. Grant the minimum set of permissions to the patching administrator, as well as grant the minimum level of access to any servers where you will be setting up patching infrastructure.

Walkthrough-Restricting-permissions-for-a-patching-administrator

Minimum-permissions-for-patching

Configure global patch configuration parameter

Global Configuration parameters provide basic information used during patch catalog creation and updating, as well as for Patch and Remediation Jobs. The following parameter groups are available:

• All Operating Systems — Configuration parameter options for a proxy server.
• Platform-specific groups for each platform (such as Windows and Solaris) — Parameters that apply only to that specific platform type
• Shavlik URL Configuration — Configuration for connecting to Shavlik Technologies to download patch-related metadata for patching Windows software.

Configuring-global-configuration-parameters

Global-Configuration-parameter-list

(Windows only) Define the location of installation media for Microsoft Office patch deployment

To deploy Microsoft Office patches, BMC Server Automation must have access to a network location containing installation media for Microsoft Office. Because target servers can run different versions of Microsoft Office, you might need to specify a different location for each target server or smart group.

Defining location of Microsoft Windows installation media for Microsoft Office patch deployment

Choose offline mode or online mode

There are two patch management modes:

• Online mode — Patches are downloaded directly from the appropriate product site. Online mode indicates the BSA application server has direct internet access and downloads the patch metadata and payloads from the Shavlik networks (for Microsoft systems) or the OS vendor web sites.  
• Offline mode — Patches are pre-downloaded to a local repository and patches are applied from the repository. Offline mode is useful when, for whatever reason, the BSA application server doesn't have internet access. Use Offline mode if you work in an air-gapped environment. In Offline mode, you use the BMC offline Patch Downloader utility to download metadata and payload information to a server with Internet access. After downloading, you can transfer the metadata and payload information (using removable storage) to the patch repository within the air-gapped environment.

The Patch Downloader utilities run scripts that use XML configuration files (samples are provided with the product) containing required information such as the repository location, as well as filters used during downloading from the vendor website.

N/A

Create a patch repository for offline mode

In an offline environment, Internet access is not available to any of the servers in the environment. In this case, you work in Offline Mode using the Patch Downloader utility to download metadata and payload information to a server with Internet access. After downloading, you transfer metadata and payload information, using removable storage, to the patch repository within the air-gapped environment.

From the BMC EPD site, download the appropriate utilities for building your offline repository. The utilities are platform-specific. You must know which platform you plan to use to download your patches.

Use the utilities that you downloaded from the BMC EPD site to prepare the XML configuration files for downloading the patch content. To download the patch content, use the utilities and the XML configuration files that you prepared.

Offline-Patch-Downloader-utility

Downloading-patch-downloader-utilities

Creating-a-patch-catalog-in-offline-mode

Create a patch catalog in online mode

For both types of repositories, online and offline, you create a patch catalog using the BMC Server Automation Console. Patches are added to the catalog as depot objects according to filters that you define for the catalog.

Creating-a-patch-catalog

Walkthrough-Setting-up-and-managing-an-online-patch-catalog-for-Windows

Walkthrough-Setting-up-and-managing-an-online-patch-catalog-for-Linux

Stock the catalog

To ensure that you are working with valid patch content, you must run a Catalog Update Job before you run a Patch Job.

Updating-an-existing-catalog

Creating and running a Patching and Remediation Job

A Patching Job has two parts:

• Analysis — Analyzes the configuration of target servers and determines the required patches.
• Remediation —
  1. Downloads the payload from the vendor sites to the Patch Repository
  2. Packages the payload as a BLPackage
  3. Creates a Deploy Job to apply the patches  

You can choose to run only the Analysis part of a Patching Job, and then run Remediation later, or you can run Remediation immediately after the Analysis. 

Creating-a-Patching-Job

Walkthrough-Basic-Microsoft-Windows-patch-analysis

Walkthrough-Basic-Red-Hat-Linux-patch-analysis

Walkthrough-Basic-patch-remediation

Manually-remediating-servers

ACLPolicy.*

Create ACL policies to grant permissions to other roles that download patch objects.

ACLTemplate.*

Create ACL templates to grant permissions to other roles that download patch objects.

AIXPatchSoftware.*

AIX only: Create and read patch software.

AIXSoftware.*

AIX only: Create and read software.

BatchJob.*

Create and execute Batch Jobs that run concatenated Deploy Jobs

BLPackage.*

Create remediation packages and read their contents.

CustomSoftware.*

Linux and Windows only: Create Linux remediation jobs and read their contents.

DeployJob.*

Read and execute jobs that deploy patch packages.

DepotFile.*

Optional: Manage offline patch catalog metadata content.

DepotFolder.Read
DepotFolder.Write

Create the patch catalog in a depot folder or create remediation objects in a depot folder.

DepotGroup.Read
DepotGroup.Write 

Navigate to the patch catalog or remediation objects in a depot group.

JobFolder.Read
JobFolder.Write

Create Patch Analysis jobs and remediation jobs in a folder or navigate to them in a group.

JobGroup.Read
JobGroup.Write 

Create Patch Analysis jobs and remediation jobs in a folder or navigate to them in a group.

LinuxSoftware.*

Linux only: Create and read software.

PatchCatalog.*

Create and manage a patch catalog

PatchDownloadJob.*

Manage patch downloads.

PatchingAnalysisConfig.Modify

Optional: Manage global patch settings.

PatchingJob.Read

Read the jobs used as the basis of remediation.

PatchRemedationJob.*

Manage patch remediation jobs.

PatchSmartGroup.*

Create smart groups in the patch catalog.

Server.Browse
Server.Deploy
Server.Read
Server.Write 

Create a patch repository on a helper server, read the contents of the repository, read contents of target servers, deploy patches to target servers.

ServerGroup.Read

Allow user to browse to the helper server when selecting it and to browse to target servers.

SolarisSoftware.*

Solaris only: Create and read software.

WindowsSoftware.*

Windows only: Create and read software.

Depot folders

DepotFolder.Read
DepotFolder.Write
DepotGroup.Read
DepotGroup.Write 

Grant these permissions on the depot folder where you create a patch catalog and to all depot folders that are parents of the patch catalog folder.

Also grant these permissions on the depot folder where you create any remediation packages and to all parent job folders or groups.

Server functioning as a patch repository

Server.Read
Server.Browse

Grant these permissions to the server used as a patch repository.

Target servers

Server.Deploy
Server.Read

Grant these permissions on target servers.

Target server groups

ServerGroup.Read

Grant these permissions on any target server groups that hold the target server.

Job folder containing Patching and remediation jobs

JobGroup.Read
JobGroup.Write
JobFolder.Read
JobFolder.Write

Grant these permissions on the job folder where you create a Patching Job and to all parent job folders or groups.

Also grant these permissions on the job folder where you create any remediation jobs and to all parent job folders or groups.

Patching jobs

PatchingJob.Read

Grant this permission on any Patching Jobs.