Deleting data

With BMC Helix Data Manager, you can delete data from a source or target system. You typically delete data when offboarding a company from a multi-tenant system. You can eﬀectively remove only the selected company without aﬀecting any other companies. You can also remove a subset of data from a system, for example, when creating a test system, you may want to remove older incidents from the data set.

BMC Helix Data Manager takes a safe approach to data deletion—it deletes only data that it is first exported to a data package. Then, in the event that data is erroneously deleted, you can restore it from the data package. You should perform delete operations against static data sets only. If the data set changes between the export and the delete operation, then the delete may leave data that should have been removed.

To use BMC Helix Data Manager to export data, you must first create a migration pack. For an export deletion, the migration pack must map to and from the same system. For more information, see Setting-up-migration-packs-mappings-and-templates and Running-an-export-migration-job.

The process of exporting data and then deleting it is managed in BMC Helix Data Manager as a deletion job.

Starting and monitoring deletion jobs

BMC Helix Data Manager can delete data from a source or target system. BMC Helix Data Manager takes a safe approach to data deletion—it deletes only data that it has first exported to a data package. So, if data was erroneously deleted, it can be restored from the data package. This feature is typically used in the following scenarios:

Clearing any sample data from a fresh installation
If you are using the Staged Upgrade methodology, you will migrate all data from your source system to a new system that was created by a fresh installation. Typically, an installation creates sample data that may clash with the data migrated from your source system. To ensure a high-quality outcome, we recommend running a Deletion job using the Clear Down template to remove sample data from the target system.

Oﬀboarding a Company from a multi-tenant system
BMC Helix Data Manager is able to eﬀectively remove only the selected Company without aﬀecting any other Companies. This feature can also be used to remove a sub-set of data from a system, for example, when creating a test system, you may want to remove older Incidents from the data set.

Prior to running a deletion job, you should copy the Clear Down Migration Pack template for your target Data dictionary and set the available to Published.

You can start the deletion process after a successful export.

At the top of the page a graphical illustration of the progress of the deletion job is displayed. The phases are:

Backup Data—Export the data to the file system.
Package Backup—Prepare the data package so that it can be reimported if data is erroneously deleted.
Delete Data—Delete the data that was backed up to the data package.
Complete—The deletion job is complete.

After each phase is complete, you are prompted to review the results and progress to the next phase. Use the Action menu to perform context-sensitive actions on the deletion job.

To run a deletion job

Select Data Migration > Delete Data from the navigation bar to display the Create Deletion Job dialog box.
Select the target system from which data will be deleted and confirm the migration pack, export template, and file systems that will be used for the deletion job.
Click Next to proceed.
Typically, no parameters need to be populated for the Clear Down Template; however, if you have customized this template, you may be prompted to enter parameters.
If required, provide the parameters, and click Next.
Review and confirm the details for the deletion job as illustrated in the example below.
(Optional) Set the number of threads for the Export process.
We recommend overwriting the default name for the deletion job.
Do not set flat export mode for deletion because it might remove data that you should preserve in a typical upgrade scenario.
Click Confirm Delete to create and display the deletion job.
The first step will export the data to a data package from which the data can be restored if it is deleted in error.
To commence the export, select Action > Start Export.
After the export is complete, if you are not using a shared file system, move the data package to the target file system so that the target database can read it.
Select Action > Scan File System.
Review the data that was exported to ensure that the data set is as you expect.
Deletion is permanent, but you can re-import the data from the data package after the deletion. If you have any doubts, create a database backup in addition to the BMC Helix Data Manager data package.
To start the deletion process, select Action > Delete Data.

After the process begins, the Summary tab is divided into four sections:

Job summary (top left)
Checklist (top right)
History (bottom left)
Progress tracker (bottom right)

Job summary

The initial view of the deletion job page shows the summary configuration. The following details appear on the Summary tab:

Field	Description
Environment and System	The name of the environment and system from which the data will be exported and then deleted.
Job Name	The name of the deletion job.
Status	The status of the deletion job. Possible values are: Draft—The job has been created. Submitted—The job has been submitted for the BMC Helix Data Manager runtime to start it. In Progress—The job is currently running. Complete—The job is complete. Cancelled—The job has been cancelled. Paused—The job has been paused.
Result	The result of completing the job. Possible values are: None—The job is not yet complete. Success—The overall result was successful; however, there may be some errors that occurred during the processing of the job. Failure—The job encountered a critical failure and was unable to complete successfully.
Updated	The date and time when the current view was last updated. This timestamp is automatically updated when the view is refreshed by the Refresh button or by the Auto-refresh option.

The Action menu allows you to trigger process-sensitive actions for the export job or to display the migration job.

History

The history section shows a list of all actions that were performed for the export job. The list shows a summary description and the timestamp the action occurred.

Checklist

The checklist shows the activities for a deletion job. The items are as follows:

Checklist item	Action
Validate Export Request	Validation that all required information has been submitted and that the export can be processed successfully.
Remedy System Connectivity	(Remedy AR System only) Checks that the BMC Helix Data Manager Runtime has connectivity to the BMC Helix ITSM system database.
Prepare Database for Export	Prepares the database for the export and deletion of data. This process might diﬀer depending on the underlying system's database type.
Export Data	Export of the data to file.
Package Data	After the data is exported, an xml file is created that acts as a checksum to ensure the package is valid.
Delete Data	Deletes the data that was backed up to the data package.
Clean up temporary data	Clean up of any temporary database tables or files that were created during the export process.

Progress tracker

The progress tracker graphically illustrates the progress of the export and packaging phases. The Export tab shows the number of processed forms or tables. The Package tab shows the number of packaged files. The Delete tab shows the number of processed forms or tables.

To view an existing deletion job

Open the Home page from the navigation bar, and click the Deletion Jobs tab, which lists all deletion jobs that have been created.
From this tab, perform any the following actions:
- Drill down to view the details of the deletion job—Select a job from the list, and click View.
- Refresh the list to obtain the latest information—Click Refresh.
- Filter the list of deletion jobs displayed—At the top of the column, enter text or select from the drop-down list you want to filter by. To sort the list, click a column heading.
When you drill down on a deletion job from the Home page, perform any the following functions on the View Deletion Job page that appears:
- View the details and progress of the deletion job
- Start the export process
- View the data package contents

To export and delete trees

The Export tab and the Delete sub-tab (on the Summary tab) show the forms or tables in the scope of the export and the scope of deletion as migration trees. These trees are updated during the processing of the export to display the latest progress.

The Export and Delete tabs use these icons:

	Queued for processing
	In progress
	Successful Delete
	Delete Failed
	The form or table has no data to delete

You can select any of the forms or tables in the tree to see the status, number of exported records, and the qualification to the parent. If a failure occurred, an error message is displayed.

The date and time the data for the migration tree was last updated is shown at the top right of the table. You can click Refresh to update the tree immediately, or click Auto-refresh to update the tree in real time. Do not enable the Auto-refresh on the Summary tab and the migration tree because the large amount of data can cause issues for the browser connection.