Marking data for deletion

Before running your cleanup jobs, you must mark the data that you want to delete. This involves the following actions:

  1. Enable a retention policy, so that data that is older than a certain amount of time (that you define) can be deleted the next time that you run a database cleanup or file server cleanup.
    Note that by default the retention policy is not enabled, to avoid the possibility of deleting data unknowingly.
  2. Define retention periods, that is, the maximum amount of time to save data according to the retention policy at your organization. You define retention periods for three separate types of accumulated data:
    • Historical data from the results of job runs
    • Other types of historical data (audit trail, job run events, job schedules, and shared generic assets)
    • Depot objects automatically generated during compliance remediation or patch remediation, as well as Deploy Jobs and Batch Jobs from these remediation processes
  3. Execute the retention policy utility, the BLCLI command Delete executeRetentionPolicy. 
    Execution of the retention policy utility is included in the out-of-the-box database cleanup script and job as the RETENTION mode. The RETENTION mode is included in TYPICAL mode.

Note

When you execute the retention policy, objects in the database that are older than the retention policy are marked for deletion, or soft deleted. Soft deletion occurs also when you delete objects through the TrueSight Server Automation GUI.

More information about soft deletion...

When an object is soft deleted, an is_deleted column is set to a value of 1 in the relevant table row in the database. Sometimes additional rows of dependencies are also marked for deletion. Undoing a soft deletion directly in the database is not supported, due to the complexity of identifying all dependencies.

Permanent deletion objects, or hard deletion, occurs when you run a cleanup job or BLCLI command — such as cleanupHistoricalData (for the various types of historical data), cleanupDatabase (for workspace objects), or cleanupFileServer (for file payloads on the file server and corresponding database entries). In addition, for objects with data that is shared by other objects, such as a snapshot that serves as a baseline, the hardDeleteAllSharedObjects command plays a role in ensuring that all relevant objects are hard deleted. 

Tip

BMC typically recommends a retention period of 30 days for most types of data. To help you decide on a retention period that suits your needs and prevents old data from accumulating in the database, or to assess the effectiveness of an existing retention period in keeping your database clean, you can periodically run database queries to determine how much TrueSight Server Automation data is accumulated in database tables every month.

For several sample queries, click to expand...

The following sample queries return the number of database entries for several types of data that typically accumulate rapidly:

select trunc(event_date,'mon'),count\(*) from job_run_event group by trunc(event_date,'mon') order by trunc(event_date,'mon');
select trunc(log_date,'mon'),count\(*) from audit_trail group by trunc(log_date,'mon') order by trunc(log_date,'mon');
select trunc(start_time,'mon'),count\(*) from job_result_device group by trunc(start_time,'mon') order by trunc(start_time,'mon');
select trunc(start_time,'mon'),count\(*) from job_run group by trunc(start_time,'mon') order by trunc(start_time,'mon');

The following sections summarize the steps for marking data for deletion:

To enable or disable the retention policy utility

  1. Start the Application Server Administration console, as described in Starting-the-Application-Server-Administration-console.
  2. To enable or disable the retention policy utility, enter the following:
    set Cleanup EnableRetentionPolicy true|false
    where:
    true — Enables the retention policy utility, which lets you mark objects for deletion from the TrueSight Server Automation database. (Objects are deleted from the database the next time the database cleanup utility is run. In addition, files associated with the objects are deleted from the file server the next time the file server cleanup utility is run.)
    false — Disables the retention policy utility. You cannot mark objects for deletion during database cleanup.
  3. Restart the Application Server to have this change take effect.

To set the retention period for the results of job runs

You can set the number of days to retain the results of job runs before they are marked for deletion. BMC typically recommends a value of 30 days for this retention time. If the number of days is not set, the default value is 0, and job runs are not deleted.
To set the number of days to retain job results before marking them for deletion, do any of the following:

  • To set the default retention period for all jobs, set the RESULTS_RETENTION_TIME property for the Job property class in the Property Dictionary. For details on setting a property using the Property Dictionary, see Adding-or-modifying-properties.
  • To set the default retention period for all jobs of a specific type, set the RESULTS_RETENTION_TIME property for the specific job property class in the Property Dictionary (for example, the SnapshotJob property class). For details on setting a property using the Property Dictionary, see Adding-or-modifying-properties.

  • To set the default retention period for job runs of a specific job, set the RESULTS_RETENTION_TIME property using the Properties tab for a specific job. For information about setting property values using the Properties tab for a system object (such as a job), see the Setting-values-for-system-object-properties.

    Note

    When you set a retention period for all jobs or for all jobs of a specific type, the new default setting is applied to future jobs, but not to existing jobs.

    The most specific retention value is used when executing a retention policy. For example, a specific job value overrides any value defined for the specific job type or for all job types.

To set the retention period for other types of historical data

To define a retention period for other types of historical data  — audit trail, job run events, job schedules, and shared generic assets  — set a value for the RetentionTime parameter in the script-based cleanup job, and then schedule job execution. The value that you set for the retention period is passed on to the relevant underlying BLCLI command.

The default value is 14 days, and the recommended value is usually 30 days. The minimum allowed value is 3 days.

To set the retention period for automatically generated objects

Automatically generated objects typically include BLpackages involved in compliance remediation or patch remediation, as well as Deploy Jobs and Batch Jobs from these remediation processes. Such objects have an AUTO_GENERATED property set to true. To set the number of days to retain automatically generated objects before marking them for deletion, do the following:

  1. Start the Application Server Administration console, as described in Starting-the-Application-Server-Administration-console.
  2. Specify the retention period (in days) with the set command:
    set Cleanup AutoGeneratedRetentionTime <number of days>
    where:
    <number of days> is the number of days that job and depot objects are retained before being marked for deletion (when you execute the retention policy utility). BMC typically recommends a value of 30 days for this retention time. If the number of days is not set, the default value is 0, and objects are not deleted.
  3. Restart the Application Server to have this change take effect.

To execute the retention policy utility

To execute the retention policy utility, schedule the script-based cleanup job to run in TYPICAL mode or RETENTION mode (which runs the BLCLI command Delete executeRetentionPolicy). For more information about these modes, see Configuring-parameters-in-the-script-based-cleanup-job. Execution of the retention policy marks all the relevant objects in the database that are due to be deleted during the upcoming database cleanup.

To set the retention period for deleted data

To define number of days for which the deleted data (deleted through TrueSight Server Automation Console or BLCLI) should be retained for ETL before being permanently deleted:

  1. Start the Application Server Administration console, as described in Starting-the-Application-Server-Administration-console.
  2. Specify the retention period (in days) with the set command:
    set Cleanup RetainDeletedDataForETLInDays <number of days>
    where:
    <number of days> is the number of days for which the deleted data is retained for ETL before being permanently deleted (when you execute the cleanup utility). The default value is 15.

If you are not using TrueSight Server Automation - Data Warehouse, you can set this value to 0.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*