Purging soft-deleted data by using the Reconciliation Engine
You can use a Purge activity to physically delete only instances that have been marked as deleted, or soft deleted. You can optionally further restrict the instances to be purged by using a Qualification Set.
The Purge activity is similar to the Delete activity, which removes instances regardless of whether they are soft deleted. For information about the Delete activity, see Deleting data by using the Reconciliation Engine.
- If you are purging data from datasets that also serve as source datasets in a Merge activity, you can choose to only delete instances that have also been soft deleted in the target dataset. This helps prevent "orphan" instances in the target dataset that would never get marked as deleted because no source ever merges with them.
- To ensure that users have a chance to review soft-deleted records, make Purge activities part of a stand-alone job that is executed manually or by workflow events.
To create a Purge reconciliation activity
- In the Reconciliation console, click Create Job or open an exist reconciliation job.
See Creating and editing a customized reconciliation job.
- In the Activities area, click New to define a new activity.
- For Type, select Purge.
Complete the following fields:
Type a unique name. The name cannot contain any characters that are not allowed in file names for the operating system of your server. For example, on a Windows server, your job name cannot contain the following characters: \ / : * ? " < > |.
Select a namespace using the controls with the Namespace field. Only classes in the selected namespaces can participate in the activity. For more information, see Dynamic dataset and qualification values in event-driven reconciliation jobs.
Select Active. You can later disable the activity by changing this to Inactive.
Continue on Error
Define whether the job continues if the activity has an error.
Checked — A job containing this activity continues to run if an error occurs in this activity.
Unchecked — A job containing this activity terminates if an error occurs in this activity.
Specify in what order you want this activity to run relative to other activities in a job. For example, if this activity has a value of 2 it runs before an activity with a value of 3. The sequence can be 0 to 1000, inclusive.
- From the Datasets list, select a dataset from which you want to purge data.
- From the Purge Instanceslist, select one of the following options.
- Identified & Unidentified — Both identified and unidentified instances are purged.
- Identified — Only identified instances are purged.
- Unidentified — Only unidentified instances are purged.
- For Verify in Target Dataset, select one of the following values:
- Checked — An identified instance is only purged from one of the datasets in the Datasets table if an instance with the same reconciliation identity in the Target Dataset is also marked as deleted. Unidentified instances are deleted.
- Unchecked — The target dataset is ignored, and both identified and unidentified instances are deleted.
This list is disabled if you selected Unidentified in the previous step because there is no way to verify an unidentified instance across datasets.
- If Verify in Target Dataset is enabled, select a dataset against which to validate the instances to be purged.
- To save the activity, click Done.
In the Job Editor, click Save.
When you have added activities to a job, you can execute the job manually or with a schedule. See Starting and stopping a reconciliation job.
Service Pack 1 and later
Prior to 8.1 Service Pack 1, the reconciliation engine used to exclude all the relationships during purge activity. As a result, if the relationship between the strong and weak CI got deleted then the weak CI would become orphan. To avoid such data integrity issues, relationships were excluded during the purge activity. However, from Service Pack 1 onwards the strong relationship (as defined in Common Data Model) can be purged safely, without causing any data integrity issues. For details about Relationships refer the data model, Relationships in the data model.