Running a delta data (incremental) migration job

An incremental (or delta) data migration is a special kind of data migration that is intended only to migrate data that has been changed in the source system since a selected date and time. Typically, the target system will already have been populated with the data which was changed prior to the selected date and time. The aim of a delta data migration is to fully synchronize two BMC Helix ITSM systems.

Filtering the data in this way is intended to reduce the overall migration time and is useful as part of an upgrade project where the final cutover data migration is time critical.

For an incremental data migration to be successful, it must replicate record creations and modifications but also data deletions. BMC Helix Data Manager fully supports all aspects of incremental data migration.

To perform an incremental data migration

1. Log in to BMC Helix Data Manager workbench.
2. Select Data Migration > Migrate Data.
3. Create an export migration job by selecting source and target systems, migration pack, and migration set.
4. In the Export Template list, select an export template with the incremental export setting.

5. (Optional) Set the Delete Tracking Enabled option to Yes.

   Important

   You can use this option if you are using the latest hotfix version of AR System server version 20.08.

   If you set this option to No, BMC Helix Data Manager uses the Form Deletion Template option to manage deleted records.
   See To set the deletion option.
   

6. (Optional) If you want to use the Form Deletion Template option to delete records, add the specific forms to the Form Deletion Set With UDM_Forms deletion list.
   Perform the following steps before you run the export job:
   1. Log in to HDM and select Configuration > Import HDM Meta Data.
   2. Select Scan.
   3. Select Form Deletion Set with UDM_Forms.cmt from the list and click Import Meta Data.
      After the import is successful, you can see the deletion list in the Configuration > Form Deletion list.
   4. Add the forms that you want to delete to the Form Deletion list.
   5. Click Save.
7. Run the export job.
8. Create an import job and select an import template with the incremental import setting.
9. If you have set the Delete Tracking Enabled option to No in the export job, you must select the Form Deletion template appropriate for your migration.
   See To set the deletion option.
10. Run the import job.

Incremental data migration will be successful only if you have migrated the baseline data previously and have followed the best practices for data management.  Contact BMC Support for details on how to effectively manage your data migration scenarios.

To set the deletion option

By using BMC Helix Data Manager, you can manage the data deletions during migration by using any one of the following methods:

• Delete Tracking Enabled option:
  • You can use this option if you are using the latest hotfix version of AR System server version 20.08.
    If you use the Delete Tracking Enabled option with an older version of AR System, the export will fail.
  • By using this option, BMC Helix Data Manager exports a deletion list file containing the form name, entry id, entry create date, deleted by and delete date data from the Entry Delete Audit form.
  • The Delete Tracking Enabled option performs the deletions faster than the Form Deletion Template option.
  • Set this option when you are creating the export incremental migration job.

  • Enable the Enable-Global-Entry-Delete-Audit CCS parameter to use this option.
    See Updating configuration settings by using the AR System Configuration Generic UI form.

  • In AR System, the Entry Delete Audit form stores AR forms and the data deleted in the AR system only. If you delete any record from the database, the data will not be deleted from your target system.
    
    
• Form Deletion Template option:
  • By using this option, BMC Helix Data Manager performs deletion checking by exporting an entry list for each form in the migration pack. The form to be deleted or modified must be present in the form deletion template. If not, the data will not be deleted.
  • When you set the Delete Tracking Enabled option to "Yes", the export job will not create an entry list. When you set the Delete Tracking Enabled option to "No", the export job will use the form deletion template functionality and will create an entry list.
  • During import, BMC Helix Data Manager deletes all the entry IDs that are not present in the entry list.
  • Form deletion checking is an expensive operation, and the Form Deletion template allows you to restrict the deletion checking appropriate for your change freeze and application usage. 
  • Set this option when you are creating the import incremental migration job.

To check for deleted forms

When performing an incremental data migration, you can configure BMC Helix Data Manager to check for record deletions across a range of selected forms. This list of forms is typically configured to include those in which record deletions occur as part of everyday usage of the BMC Helix ITSM application. You can extend this list to include custom forms or other forms where records are deleted manually or through custom workflow.

If you are using the Form Deletion Template option to delete records from specific forms, you must add the forms to the Form Deletion Set With UDM_Forms deletion list before you run the export.

Changes to the list of forms checked for deletions take place immediately and will be picked up the next time an import job is run. This allows changes to be made to suit your business objectives.

To view the Form Deletion page, select Configuration > Form Deletion from the navigation bar.