Cleanup considerations for snapshot


Database cleanup is performed when specific data is not required and the specified data retention time is exceeded. For snapshot job runs, some additional requirements are considered for cleanup. These requirements are based on baselining, change tracking, and rebaselining implementations. Rebaselining is one of the contributing factors for snapshot data growth because it collects all the information related to configuration items and server objects. The amount of data growth is directly proportional to the number of configuration items and server objects.

To understand the concepts of baselining, change tracking, and rebaselining, see Snapshot-baselining-change-tracking-and-rebaselining.

Conditions for cleaning up snapshot job runs

For cleaning up Snapshot Job runs, do note the following conditions:

Condition 1

For a given Snapshot Job, all the snapshot job runs from the initial baseline till the next immediate rebase line are dependent on each other and are considered a part of one group. Therefore, the snapshot cleanup task does not remove any job runs unless the retention period is not exceeded for all of them. (The entire group of job runs is deleted or nothing is deleted.)

Condition 2

The snapshot cleanup considers rebaselining that occurs only due to job versioning. If the rebaselining occurs due to other reasons, such as a template or component versioning, the cleanup task does not delete the old snapshot job runs even though they are eligible for deletion as per the retention period because the job runs from the baseline up to the latest job run are required for the proper functioning of the Snapshot Job.

Condition 3

If snapshot job runs are used in the Audit domain, these runs and their baselines cannot be deleted.

Forceful removal of snapshot job runs

You can forcefully remove snapshot job runs in the following scenarios:

  • If snapshot job runs are not deleted for a long time or the cleanup is not able to remove any runs due to rebaselining of jobs, you can do the following:
    1. Forcefully reversion the snapshot job by changing an attribute that does not alter the job configuration. For example, you can change the job description. 
    2. Run the snapshot job to generate a rebaseline for the job.
    3. Select a data retention period in such a way that all the previous job runs till the latest job version date are removed.
  • If a snapshot job run is referenced in an Audit Job, remove the audit job run or use the audit domain cleanup to delete the audit job run.

Example scenario

Alan is an administrator. He wants to capture the configuration data of servers for performing audits. He creates a Snapshot Job and schedules its run. He makes some modifications to the job and component template. After running the job for a few days, he runs the cleanup by selecting different retention periods. 

The following table describes the various states of Snapshot Job runs based on the actions that Alan performs, such as job and component template updates.

Snapshot Job runs

Day

Action summary

Versioning

Job runs

Snapshot Job behavior type

Conceptual group1

Component template (CT)

Job (J)

Day 0

Alan creates a Snapshot Job by using the component template. He schedules the job to be run every day on all targets.

1

1

JobRun1

Baseline

Group 1

Day 1

The job runs as scheduled after some changes in assets for some targets.

1

1

JobRun2

Change Tracking

Day 2

The job runs as per the schedule (No change).

1

1

JobRun3

Change Tracking

Day 3

Alan modifies the job by adding more targets and components. The job runs as per the schedule.

1

2

JobRun4

Rebaseline - (Job Version)

Group 2

(The rebase line that occurred on Day 6 is not considered as it is due to template versioning.)

Day 4

The job runs as per the schedule (No change).

1

2

JobRun5

Change Tracking

Day 5

The job runs as scheduled after some changes to assets for some targets.

1

2

JobRun6

Change Tracking

Day 6

Alan modifies the component template and runs the job.

2

2

JobRun7

Rebaseline - (Template Version)

Day 7

The job runs as scheduled after some changes to assets for some targets.

2

2

JobRun8

Change Tracking

Day 8

The job runs as per the schedule (No change).

2

2

JobRun9

Change Tracking

Day 9

The job runs as scheduled after some changes to assets for some targets.

2

2

JobRun10

Change Tracking

Day 10

Alan updates the job description and the job runs as scheduled.

2

3

JobRun11

Rebaseline - (Job Version)

Group 3

(The job version is not affected by job schedule changes. Therefore, the rebase line does not occur on Day 14.)

Day 11

The job runs as scheduled after some changes to assets for some targets.

2

3

JobRun12

Change Tracking

Day 12

The job runs as scheduled after some changes to assets for some targets. Alan referred the job run as a part of an Audit Job.

2

3

JobRun13

Change Tracking - (Audit Job Referred)

Day 13

The job runs as scheduled after some changes to assets for some targets.

2

3

JobRun14

Change Tracking

Day 14

Alan updates the job schedule and runs the job. 

2

3

JobRun15

Change Tracking

Day 15

The job runs as scheduled after some changes to assets for some targets.

2

3

JobRun16

Change Tracking

Day 16

Alan updates the DEBUG_MODE_ENABLED job property and runs the job.

2

4

JobRun17

Rebaseline - (Job Version)

Group 4

Day 17

The job runs as scheduled (No change).

2

4

JobRun18

Change Tracking

Day 18

The job runs as scheduled after some changes to assets for some targets.

2

4

JobRun19

Change Tracking

Day 19

Alan updates the property about the number of jobs to process in parallel and runs the job.

2

5

JobRun20

Rebaseline - (Job Version)

Group 5

Day 20

The job runs as scheduled (No change).

2

5

JobRun21

Change Tracking

Day 21

The job runs as scheduled after some changes to assets for some targets.

2

5

JobRun22

Change Tracking

Day 22 to 24

The job runs as scheduled (No change).

2

5

JobRun23-25

Change Tracking

1 The groups are based on the baseline and dependent job runs.

Alan runs the cleanup job on day 25. The following table describes the job runs that are considered for deletion for different retention periods.

Cleanup of Snapshot Job runs

Retention period

Conceptual groups considered for deletion

Job runs considered for deletion

Reason

20 days

Group 1 (Day 0 to 2)

JobRun1 to 3

The cleanup should delete all the job runs before Day 5 (Cleanup execution day - Retention period). However, job runs only from Group 1 are considered for deletion as condition 1 is satisfied.

  • All the job runs in Group 1 qualify for deletion as they exceed the retention period. Hence, they are deleted.
  • All the job runs in Group 2 do not exceed the retention period. Hence, Group 2 is excluded from deletion. 

17 days

Group 1 (Day 0 to 2)

JobRun1 to 3

The cleanup should delete all the job runs before Day 8 (Cleanup execution day - Retention period). However, job runs only from Group 1 are considered for deletion based on the condition 1.

  • All the job runs in Group 1 qualify for deletion as they exceed the retention period. Hence, they are deleted.
  • Group 2 is not selected for deletion due to condition 2.

10 days

Group 1 (Day 0 to 2) and Group 2 (Day 3 to 9)

JobRun1 to 10

The cleanup should delete all the job runs before Day 15 (Cleanup execution day - Retention period). However, job runs only from Groups 1 and 2 are considered for deletion due to conditions 1, 2, and 3.

5 days

Group 1 (Day 0 to 2), Group 2 (Day 3 to 9), and Group 4 (Day 16 to 18)

JobRun1 to 10 and 17 to 19

The cleanup should delete all the job runs before Day 20 (Cleanup execution day - Retention period). However, job runs only from Group 1, 2, and 4 are considered for deletion based on conditions 1, 2, and 3.

  • All the job runs in Group 1, Group 2, and Group 4 qualify for deletion as they exceed the retention period. Hence, they are deleted.
  • For Group 3 to be qualified for deletion, Alan must delete the Audit job runs or remove the Audit domain as a part of cleanup.
  • The job runs in Group 5 are not deleted because only some of them exceed the retention period.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*