Important

   

Starting version 8.9.03, BMC Server Automation is renamed to TrueSight Server Automation. This space contains information about BMC Server Automation 8.9.02 and previous versions. For TrueSight Server Automation 8.9.03 and later releases, see TrueSight Server Automation 8.9.

Example cleanup plan for large operational environments

This topic walks you through an example end-to-end process of setting up a comprehensive plan for cleaning up your BMC Server Automation (BSA) database in a large operational environment, where the database fills up quickly, and the standard weekly cleanup plan (described in Example weekly database cleanup plan) is not sufficient. This cleanup plan can also be used in situations where you have accumulated large amounts of BladeLogic data in the database due to failures in the basic weekly cleanups, and you need to temporarily apply a more forceful cleanup plan to catch up on database maintenance.

The cleanup plan described here involves a combination of cleanup jobs organized within Batch Jobs and scheduled to run on a frequent basis (mostly daily, some weekly). Setting up the cleanup plan involves the following tasks:

  1. Define retention periods.
  2. Create multiple cleanup jobs based on the default cleanup job that is provided out-of-the-box. Each of these jobs is designed to perform an individual cleanup function. 
  3. Organize the various cleanup jobs that you created (in the previous task) as the building blocks in more complex Batch Jobs — one for the daily cleanup of historical data, and the other for weekly hard deletion of additional data that is not fully deleted by the daily job.

This topic includes the following sections:

How to set and enable retention times

Before running Database Cleanup Jobs, you must set retention periods for two types of BSA data. You must also enable a retention policy, so that data that is older than the defined retention periods is marked for deletion. For more information about tasks related to retention periods and the retention policy, see Marking data for deletion.

 StepExample screen
1

Set a retention period for job run data and job results through the Property Dictionary. In this example we define a retention period of 30 days (the length of time recommended by BMC) for all jobs of all types.

  1. In the BMC Server Automation Console, from the Configuration menu, select Property Dictionary View.

  2. In the Property Class Navigation pane, expand the Built-in Property Classes node, and select the Job property class.

  3. In the property list on the right, select the Properties tab.

  4. Select the RESULTS_RETENTION_TIME property, and then click Edit Property.

  5. Ensure that the Use this default value check box is selected, and then, in the Default value column, enter 30 as your retention period value (in days).

Note


 To see how job run data is displayed in the GUI, click here.

Job runs are visible in any job's Results panel; each job run is accompanied by a green (success) or red (failure) icon and a date stamp.


 For information about additional options for setting retention times, click here.

In this example we have set one retention period for all jobs of all types. You can, optionally, set different retention periods for different types of jobs or even for specific jobs. The more specific retention periods override the more general retention periods. For more information about these options, see To set the retention period for job runs.

For more information about setting properties through the Property Dictionary, see Adding or modifying properties.

This retention time is not applied to the audit trail, job run events (that is, job run log messages), and job schedules. These data types accumulate faster than the other types of job data. Therefore, they have a shorter default retention time of 14 days.

 

2

We will be using the BMC Server Automation Application Server console (the blasadmin utility) to set a retention period for auto-generated objects and to enable the retention policy.

To start the Application Server Administration console, perform the following step:

  • Windows: From the Start menu, select Programs > BMC Software > BladeLogic Server Automation Suite > Utilities > Application Server Administration.
  • UNIX: From the directory where BMC Server Automation is installed, enter ./br/blasadmin

For alternative methods, see Starting the Application Server Administration console.

 
3Auto-generated objects typically include BLpackages involved in compliance remediation or patch remediation, as well as Deploy Jobs and Batch Jobs from these remediation processes. To specify a retention period for auto-generated objects, enter the following command:
set Cleanup AutoGeneratedRetentionTime 30
In this example, we are setting the retention time to 30 days, as recommended by BMC.
4To enable the retention policy. Enter the following command:
set Cleanup EnableRetentionPolicy true
5

For your changes to take effect, restart the Application Server. Perform the following steps:

  1. In the BMC Server Automation Console, select Configuration > Infrastructure Management.
  2. Expand the Application Servers node.
  3. Right-click the Application Server and select Restart.

For alternative methods, see Restarting Application Servers.

 

How to create jobs for individual cleanup functions

For greater control over the various cleanup functions, we will create multiple copies of an out-of-the-box NSH Script Job named BSA Recommended Database Cleanup Job, and customize each new job to perform an individual cleanup function (that is, to execute an individual delete command). In each of these new jobs, we will customize the job parameters, which are inherited from the BSA Recommended Database Cleanup Script. All these jobs will serve as the building blocks in the next task, when we organize the individual cleanup jobs in Batch Jobs.  (For more about the out-of-the-box script and job, see Performing online database cleanup.)

 StepExample screen
1

To create your first copy of the default cleanup job, perform the following steps:

  1. Expand the Jobs folder and navigate to BMC Maintenance > BSA Recommended Database Cleanup Job.
  2. Right-click the job and select Copy.
  3. Right-click the BMC Maintenance folder and select Paste.
    A dialog box tells you that the copy operation is occurring in the background. Shortly afterwards, a new job named Copy of BSA Recommended Database Cleanup Job appears in the folder, alongside the original job that you copied.
2To open the new job, right-click Copy of BSA Recommended Database Cleanup Job, and select Open.
3 In the content editor on the right, on the General tab, change the name of this new job to Cleanup AuditTrail, and the description to Cleanup of the Audit Trail. Then click Save  in the toolbar.
4

To customize job parameters, click the Parameters tab, and then make the following changes to parameter settings:

  • Through the Value column, set new values for the following parameters:

    ParameterExample value
    ExecutionMode


    HISTORY_ORDR



    ObjectTypeAuditTrail


    RetentionTime


    30 (same value as set for RESULTS_RETENTION_TIME)
  • Through the Flag runtime usage column, set the following parameters to Ignore. This will force the NSH Script Job to use default values for these parameters without requiring input.

    • MaxDuration
    • DurationDistribution
    • RoleName
    • TargetSize

  • Leave the remaining parameters (ContinueOnError and TargetDevices) with their default settings.

For more information about the script parameters, see Script parameters.


5 Save the job again to apply all changes. 
6

Repeat steps 1 to 5 for another 12 cleanup functions. The following table lists all 13 cleanup jobs that you need to create, and includes the name for each job (to rename in step 3) and the parameter settings that you need to adjust (in step 4). You need to create Cleanup Retention job only once.

Tip: To speed up the process of creating these jobs, choose a source job (in step 1) with parameter settings that are similar to those of the target job. For example, to create the Cleanup JobRunEvent job, copy the Cleanup AuditTrail job and adjust just the job name and description and the ObjectType parameter. You can also paste multiple copies of the same source job.

 

Job names and parameters to adjust in the cleanup jobs for individual cleanup functions

Job nameExecution
Mode
Object
Type
Retention
Time
Other
parameters
Cleanup AuditTrail


HISTORY_ORDR
 AuditTrail


30


Set to ignore:

MaxDuration
DurationDistribution
RoleName
TargetSize

Do not change:

ContinueOnError
TargetDevices

Cleanup JobRunEvent


HISTORY_ORDR
JobRunEvent
Cleanup JobSchedule


HISTORY_ORDR
JobSchedule


Cleanup SharedData


HISTORY_ORDR


SharedData
Cleanup AuditResult


HISTORY_ORDR
AuditResultset to ignore

Set to ignore:

MaxDuration
DurationDistribution
RetentionTime
RoleName
TargetSize

Do not change:

ContinueOnError
TargetDevices

Cleanup ComplianceResult


HISTORY_ORDR
ComplianceResult


Cleanup Deploy


HISTORY_ORDR


Deploy
Cleanup PatchResult


HISTORY_ORDR
PatchResult
Cleanup SnapshotResult


HISTORY_ORDR
SnapshotResult
Cleanup AppServerCachesCLEAN_ALL_ASset to ignore30

Set to ignore:

MaxDuration
DurationDistribution
ObjectType
RoleName
TargetSize

Do not change:

ContinueOnError
TargetDevices

Cleanup DatabaseCLEAN_DBset to ignoreset to ignore

Set to ignore:

MaxDuration
DurationDistribution
RetentionTime
ObjectType
RoleName
TargetSize

Do not change:

ContinueOnError
TargetDevices

Cleanup Shared ObjectsCLEAN_SHARED_OBJECTS
Cleanup FileServerCLEAN_FS
Cleanup RetentionRETENTION



How to organize the individual cleanup jobs into Batch Jobs for a comprehensive cleanup plan

We will now organize the various cleanup jobs that we prepared into two Batch Jobs — one for the daily cleanup of historical data, and the other for weekly hard deletion of additional data that does not get deleted by the daily job. The daily cleanup job is a relatively complex Batch Job, with another two Batch Jobs nested within it (one within the other). The hierarchical organization of cleanup jobs within these Batch Jobs is as follows:

  • Daily cleanup of historical data:
    Batch Job named Daily Cleanup. This job contains the following jobs, set to run sequentially:
    1. Cleanup Retention
    2. Batch Job named Cleanup Historical, with the following jobs set to run in parallel or sequentially:
        • Cleanup JobRunEvent
        • Cleanup AuditTrail
        • Cleanup JobSchedule
        • Cleanup SnapshotResult
          However, you cannot execute two or more Cleanup Historical batch jobs in parallel.
      • Batch Job named Cleanup Results, with the following cleanups set to run sequentially:
        1. Cleanup AuditResult
        2. Cleanup ComplianceResult 
        3. Cleanup PatchResult
        4. Cleanup Deploy
        5. Cleanup SharedData
    3. Cleanup AppServerCaches
  • Weekly hard delete cleanup:
    Batch Job named Weekly Cleanup. This job contains the following jobs, set to run sequentially:
    1. Cleanup Retention
    2. Cleanup Database
    3. Cleanup Shared Objects
    4. Cleanup FileServer

The following instructions guide you through the creation of these Batch Jobs. For the daily cleanup job, we will start with the creation of the most nested, lowest-level Batch Job, and work our way upward and outward until we have created the final daily Batch Job.

 StepExample screen
1

To create the most nested Batch Job, which groups together the cleanup of job results, perform the following steps:

  1. Under the Jobs folder, right-click the BMC Maintenance folder and select New > Batch Job.
    The New Batch Job window and the Select Jobs to Add to Batch Job dialog box open.

  2. From the displayed tree, select the jobs for cleanup of job results and add them to the Batch Job. Use the Ctrl key to select multiple jobs. Then click OK.

     Click for the list of jobs:
    1. Cleanup AuditResult
    2. Cleanup SnapshotResult
    3. Cleanup ComplianceResult 
    4. Cleanup PatchResult

    5. Cleanup Deploy

    The jobs that you added to the Batch Job appear in a hierarchical list on the left.

  3. If necessary, use the Move Up  and Move Down  arrows to organize the jobs in the correct order.
  4. Select the New Batch Job node in the hierarchical list on the left.
    Several tabs of Batch Job settings appear on the right.
  5. On the Batch Job Options tab, enter Cleanup Results as the name of the job, and (optionally) enter a description for the job.
  6. Under Execution Options, ensure that Execute jobs sequentially is the selected option. Then click OK to save the job.


2

To create the next Batch Job, which groups together the cleanup of historical data, perform the following steps:

  1. Right-click the BMC Maintenance folder and select New > Batch Job.

  2. Select the jobs for cleanup of historical data and add them to the Batch Job. Use the Ctrl key to select multiple jobs. Then click OK.

     Click for the list of jobs:
    • Cleanup Results (the Batch Job that you created in step 1)
    • Cleanup JobRunEvent
    • Cleanup AuditTrail
    • Cleanup JobSchedule

    • Cleanup SharedData
  3. Select the New Batch Job node in the hierarchical list on the left to display the tabs of Batch Job settings.
  4. On the Batch Job Options tab, enter Cleanup Historical as the name of the job, and (optionally) enter a description for the job.
  5. Under Execution Options, ensure that Execute jobs in parallel is the selected option. Then click OK to save the job.

3

To create the final Batch Job for daily cleanup of historical data, perform the following steps:

  1. Right-click the BMC Maintenance folder and select New > Batch Job.

  2. Select the jobs included in the daily cleanup Batch Job and add them to the Batch Job. Use the Ctrl key to select multiple jobs. Then click OK.

     Click for the list of jobs:
    1. Cleanup Retention
    2. Cleanup Historical (the Batch Job that you created in step 2)
    3. Cleanup AppServerCaches
  3. If necessary, use the Move Up and Move Down arrows to organize the jobs in the correct order in the hierarchical list on the left.
  4. Select the New Batch Job node in the hierarchical list on the left to display the tabs of Batch Job settings.
  5. On the Batch Job Options tab, enter Cleanup Historical as the name of the job, and (optionally) enter a description for the job.
  6. Under Execution Options, ensure that Execute jobs sequentially is the selected option.
  7. To schedule the job, click the Schedules tab, and then click New Schedule. Afterwards, in the Scheduling box, schedule a daily recurrence of the job, and click OK.
    In this example, we want to run the job 6 days a week (rather than all 7 days). Therefore, we select the Weekly option, and then select all days except for Sunday. We set a time late at night when the workload is normally low.
  8. Click OK to save the job.


Note

Cleanup of shared data might involve very large tables with large amounts of data about configuration files. If the Cleanup SharedData job takes takes a very long time (several hours) to execute, terminate the long-running cleanup job, and then execute the following SQL command to increase batch size from 10000 table rows to a much larger value (1 million in this example):
update dbm_module set batch_size=1000000 where module_id=18;

After running this SQL command, run the cleanup job again.

4

To create the weekly Batch Job for hard deletion of other types of data, perform the following steps:

  1. Right-click the BMC Maintenance folder and select New > Batch Job.

  2. Select the jobs included in the weekly cleanup Batch Job and add them to the Batch Job. Use the Ctrl key to select multiple jobs. Then click OK.

     Click for the list of jobs:
    1. Cleanup Retention
    2. Cleanup Database
    3. Cleanup Shared Objects
    4. Cleanup FileServer
  3. If necessary, use the Move Up  and Move Down  arrows to organize the jobs in the correct order in the hierarchical list on the left.
  4. Select the New Batch Job node in the hierarchical list on the left to display the tabs of Batch Job settings.
  5. On the Batch Job Options tab, enter Cleanup Historical as the name of the job, and (optionally) enter a description for the job.
  6. Under Execution Options, ensure that Execute jobs sequentially is the selected option.
  7. To schedule the job, click the Schedules tab, and then click New Schedule . Afterwards, in the Scheduling box, schedule a weekly recurrence of the job (in this example, every Sunday evening), and click OK.

    Note

    If you use BMC Decision Support for Server Automation to create reports, be careful that the weekly cleanup job does not overlap the weekly ETL job for loading data to the reporting data warehouse. Typically, you first run the ETL job, and only after the ETL job has finished you run the database cleanup job.

    It is not recommended to run cleanup while running updates of database statistics (Oracle or SQL Server), so as not to impact the performance of the environment.

  8. Click OK to save the job.

Where to go from here

If you set up this cleanup plan as a temporary means for catching up on the deletion of old data that accumulated in your database, you can follow the current plan for several weeks and then revert back to the basic plan described in Example weekly database cleanup plan.

For more complicated situations, where you have accumulated large amounts of BladeLogic data in the database, you might need to plan a period of downtime for your BMC Server Automation Application Server so that you can perform a one-time offline database cleanup, a more forceful cleanup mechanism.

In addition to the jobs described here, schedule a monthly cleanup of the agents and repeater server. Create cleanup jobs set to individual modes CLEAN_AGENT and CLEAN_REPEATER, and optionally include the two jobs within a batch job. For best results, keep these cleanup jobs separate from database cleanup (for example, do not include database cleanup in the same batch job).

Was this page helpful? Yes No Submitting... Thank you

Comments