Estimating your data size for migration with the AREntryCounter utility

Determining data size

Before your start your data migration, perform the following steps on the source server to determine the size of your data:

1. On the source server, run the AREntryCounter utility on all the packages with only the primary DDM start date (-d) and no end date.
    For more information about working with the utility, see Running the AREntryCounter utility.
2. Review all the output count text files to check the volume of data.

3. Based on the volume of data, plan your DDM runs in batches with the start date and end date for each runs.

   Tracking DDM run details

   Use a spreadsheet to record the batch run details to track each run, re-run, and count validation results.

4. Run the utility with a shorter start date and end date intervals to determine the number of DDM runs that you need to perform.
   For example, you find that based on the count files, certain ITSM Foundation and BMC Atrium CMDB forms have a large volume of delta data. You must split and run multiple DDM batches with shorter date intervals for faster data migration.
5. Based on your data volume and your schedule, you might run DDM for all products in parallel for two or more date ranges in multiple Microsoft Windows machines. These machines must have BMC Remedy Migrator installed.
6. Optionally, if the custom and backend forms have a large amount of data, you can use the SQL method to migrate a large portion of that data.
   You can use DDM for the subsequent runs. For more information about migrating custom and backend forms, see Performing data migration of backend and custom forms.

Searching for additional data

After you determine your data size, you must check if the data created by the upgrade process conflicts with the entry IDs that are created or updated on the source server. In such a scenario, you must first backup the data that was created or updated by the installer and then delete that data on the destination server. After the final DDM run, import that data back. Perform the following steps on the destination server to search for additional data, before you start your data migration:

1. Check for post-upgrade additional data on the destination server.
   1. Record the timestamp when the last upgrade was successfully completed.
      You can determine this information from the installation log files.
   2. Run the AREntrycounter utility with the start date parameter for all the packages.
   3. Review the count.txt files generated by the utility and search for counts greater than "0".
      You must delete any post-upgrade data that is not created by the installer.
   4. Perform the following steps to delete any post-upgrade data:
      1. Disable any delete filters on the form that are in development mode, by using the Developer studio.
2. On the destination server, perform the following steps to check for the data created by the upgrade installer.
   1. Capture the DB backup time value from the Source (Current Production) server.
      The backup time must be the date and time when the DB backup was restored to the staging server to perform the upgrade.
   2. Run the AREntrycounter utility with the start date parameter for all the packages.
   3. Review the count.txt files generated by the utility and search for counts greater than "0.
   4. Export, save, and then delete any data that was created by the upgrade installer.
      This is because such records will generate ID conflicts when you migrate the delta data.

   5. Using the Developer studio, disable any delete filters on each of those forms in the base development mode, and then delete the data that was created by the upgrade installer on those forms.

       

      DDM generates errors for these issues if you were unable to find and delete them before any DDM runs. A Mismatch EntryID error is generated. To migrate the data without any errors, perform steps 2a through 2e in this procedure.

3. Ensure that you follow these guidelines when using the AR Import tool options:
   1. Use the same unique index value as the one in the DDM xml files for importing.
   2. Do not use request ID.
   3. Auto-map all fields and then remove request ID (field ID 1), if it is mapped.

   4. Select suppress filters.

       

Validating counts after every DDM run

Perform the following steps, after every DDM run:

1. On the source server, obtain the counts for the specific date range and product selection.
2. After you completed the DDM run for the specific date range and product selection, obtain the counts on the destination server for the same date and product selection as on the source server.
   Perform this step to ensure that the upgrade process created no additional records on the destination server, except for the configuration and other forms (Audit/AuditLogSystem). In case of Audit forms, additional data might have been created if the parent forms had Audit turned ON. If you found additional records or data during the comparison, you can safely ignored.

Validating final counts after completing migration

Perform the following steps, after you migrate your data:

1. On the source server and destination servers, run the AREntryCounter utility with the primary DDM date on all the packages with the start date and no end date parameter.
   For more information about working with the utility, see  Running the AREntryCounter utility.

2. On the source and destination servers, generate and review all the output count text files.

   Perform this step to ensure that the upgrade process created no additional records on the destination server, except for the configuration and other forms (Audit/AuditLogSystem). In case of Audit forms, additional data might have been created if the parent forms had Audit turned ON. If you found additional records or data during the comparison, you can safely ignore.

Running the AREntryCounter utility

The AREntryCounter utility is available in the \DeltaDataMigration\Utilities\migratorutilities directory. You must run the counts for each product. The output file is available in the same folder from which you run the Delta Data Migration Tool. You run the utility commands at the command prompt.

• AREntryCounter command options
• AREntryCounter code example

AREntryCounter command options

The following table describes the available options and syntax for the utility:

AREntryCounter code example

Use the following syntax to run the utility for your package files (forms). Ensure that you replace the keywords with your BMC Remedy Migrator installation path and the server and version information. If you have additional products that are installed on your server, for example, BMC Remedy Knowledge Management and BMC Process Designer, ensure that you verify and add additional scripts as required. Note that the following example does not utilize the DDM End date parameter.

Product forms with additional data

The BMC Remedy ITSM version 9.1.02 SP2 installer creates additional data in the following forms, as explained in the Search For Additional Data section:

Where to go from here