Estimating your data size for migration with the AREntryCounter utility
In a staged upgrade, after you replicate the production database and perform the upgrade steps, you begin the delta data period. This means that you are still actively adding data in various BMC Remedy ITSM forms on your current production system.
Before you can migrate this delta data, you must first estimate the volume of data with the AREntryCounter utlity, which will determine the exact count of the records in your forms. With the data estimates, you can then plan your delta data migration (DDM) runs for specific date ranges.
For products with a large volume of data, you might have to run DDM a few times with a smaller date range.
You run the AREntryCounter utility multiple times during the DDM process, as follows:
- Before any DDM run, check for additional data by running the utility on the destination server for all the products. Use the DB backup date and time for the run.
- Before your first DDM run on the source server, to determine the data counts and estimate the volume to plan the number of DDM runs that will be required.
- After your first DDM run, on the destination server, to search for additional data.
- After your first and all subsequent runs on the destination server, to validate counts.
- After the final DDM run for the full delta period on both the source and destination servers, to validate counts.
Determining data size
Before your start your data migration, perform the following steps on the source server to determine the size of your data:
- On the source server, run the AREntryCounter utility on all the packages with only the primary DDM start date (-d) and no end date.
For more information about working with the utility, see Running the AREntryCounter utility. - Review all the output count text files to check the volume of data.
Based on the volume of data, plan your DDM runs in batches with the start date and end date for each runs.
Tracking DDM run details
Use a spreadsheet to record the batch run details to track each run, re-run, and count validation results.
- Run the utility with a shorter start date and end date intervals to determine the number of DDM runs that you need to perform.
For example, you find that based on the count files, certain ITSM Foundation and BMC Atrium CMDB forms have a large volume of delta data. You must split and run multiple DDM batches with shorter date intervals for faster data migration. - Based on your data volume and your schedule, you might run DDM for all products in parallel for two or more date ranges in multiple Microsoft Windows machines. These machines must have BMC Remedy Migrator installed.
- Optionally, if the custom and backend forms have a large amount of data, you can use the SQL method to migrate a large portion of that data.
You can use DDM for the subsequent runs. For more information about migrating custom and backend forms, see Performing data migration of backend and custom forms.
Searching for additional data
After you determine your data size, you must check if the data created by the upgrade process conflicts with the entry IDs that are created or updated on the source server. In such a scenario, you must first backup the data that was created or updated by the installer and then delete that data on the destination server. After the final DDM run, import that data back. Perform the following steps on the destination server to search for additional data, before you start your data migration:
- Check for post-upgrade additional data on the destination server.
- Record the timestamp when the last upgrade was successfully completed.
You can determine this information from the installation log files. - Run the
AREntrycounter
utility with the start date parameter for all the packages. - Review the count.txt files generated by the utility and search for counts greater than "0".
You must delete any post-upgrade data that is not created by the installer. Perform the following steps to delete any post-upgrade data:
Disable any delete filters on the form that are in development mode, by using the Developer studio.
- Record the timestamp when the last upgrade was successfully completed.
On the destination server, perform the following steps to check for the data created by the upgrade installer.
Capture the DB backup time value from the Source (Current Production) server.
The backup time must be the date and time when the DB backup was restored to the staging server to perform the upgrade.Run the
AREntrycounter
utility with the start date parameter for all the packages.Review the count.txt files generated by the utility and search for counts greater than "0.
Export, save, and then delete any data that was created by the upgrade installer.
This is because such records will generate ID conflicts when you migrate the delta data.Using the Developer studio, disable any delete filters on each of those forms in the base development mode, and then delete the data that was created by the upgrade installer on those forms.
DDM generates errors for these issues if you were unable to find and delete them before any DDM runs. A Mismatch EntryID error is generated. To migrate the data without any errors, perform steps 2a through 2e in this procedure.
Ensure that you follow these guidelines when using the AR Import tool options:
Use the same unique index value as the one in the DDM xml files for importing.
Do not use request ID.
Auto-map all fields and then remove request ID (field ID 1), if it is mapped.
Select suppress filters.
Validating counts after every DDM run
Perform the following steps, after every DDM run:
- On the source server, obtain the counts for the specific date range and product selection.
- After you completed the DDM run for the specific date range and product selection, obtain the counts on the destination server for the same date and product selection as on the source server.
Perform this step to ensure that the upgrade process created no additional records on the destination server, except for the configuration and other forms (Audit/AuditLogSystem). In case of Audit forms, additional data might have been created if the parent forms had Audit turned ON. If you found additional records or data during the comparison, you can safely ignored.
Validating final counts after completing migration
Perform the following steps, after you migrate your data:
- On the source server and destination servers, run the AREntryCounter utility with the primary DDM date on all the packages with the start date and no end date parameter.
For more information about working with the utility, see Running the AREntryCounter utility. On the source and destination servers, generate and review all the output count text files.
Perform this step to ensure that the upgrade process created no additional records on the destination server, except for the configuration and other forms (Audit/AuditLogSystem). In case of Audit forms, additional data might have been created if the parent forms had Audit turned ON. If you found additional records or data during the comparison, you can safely ignore.
Running the AREntryCounter utility
The AREntryCounter utility is available in the \DeltaDataMigration\Utilities\migratorutilities directory. You must run the counts for each product. The output file is available in the same folder from which you run the Delta Data Migration Tool. You run the utility commands at the command prompt.
AREntryCounter command options
The following table describes the available options and syntax for the utility:
Option and value | Definition |
---|---|
| Count entries that were created or modified after a specific date. To estimate the volume of data, use the date and time of the database backup when you set up your staging system.
Where TZ is the timezone, for example, EST, PST, CST, IST, and so on.
Example: |
-e <DDM End date> | Count entries that were created or modified before a specific date. To estimate the volume of data, use the date and time of the database backup when you set up the system for a staged upgrade. Where TZ is the timezone, for example, EST, PST, CST, IST, and so on.
|
| Construct the form list from the given BMC Remedy Migrator package file |
| The output file for the result report |
| The password for authenticating with the AR System server |
| The AR System server name |
| The user name for authenticating with the AR System server |
| The AR System server port |
AREntryCounter code example
Use the following syntax to run the utility for your package files (forms). Ensure that you replace the keywords with your BMC Remedy Migrator installation path and the server and version information. If you have additional products that are installed on your server, for example, BMC Remedy Knowledge Management and BMC Process Designer, ensure that you verify and add additional scripts as required. Note that the following example does not utilize the DDM End date
parameter.
Package file version
On the destination server, use the same version of the AR_Core_Package.xml files as that of the source server. The version of the package file must be the same on both servers.
AREntryCounter.bat -k "<MigratorInstallationPath>\DeltaDataMigration\Packages\ARCore\<YOUR VERSION>\AR_Core_Package.xml" -d <DDM DATE> -s <SERVERNAME> -u <USERNAME> -p <PASSWORD> -x PORT -o AR_counts.txt
AREntryCounter.bat -k "<MigratorInstallationPath>\DeltaDataMigration\Packages\Atrium\CMDB\<YOUR VERSION>\AtriumPackage.xml" -d <DDM DATE> -s <SERVERNAME> -u "Demo" -p <PASSWORD> -x <PORT> -o AtriumCMDB_counts.txt
AREntryCounter.bat -k "<MigratorInstallationPath>\DeltaDataMigration\Packages\Atrium\AIE\<YOUR VERSION>\AIE_Package.xml" -d <DDM DATE> -s <SERVERNAME> -u "Demo" -p <PASSWORD> -x <PORT> -o AtriumAIE_counts.txt
AREntryCounter.bat -k "<MigratorInstallationPath>\DeltaDataMigration\Packages\Atrium\ProductCatalog\<YOUR VERSION>\ProductCatalogpackage.xml" -d <DDM DATE> -s <SERVERNAME> -u "Demo" -p "PASSWORD" -x PORT -o AtriumPCT_counts.txt
AREntryCounter.bat -k "<MigratorInstallationPath>\DeltaDataMigration\Packages\ITSM\<YOUR VERSION>\ITSM_AST_Package.xml" -d <DDM DATE> -s <SERVERNAME> -u "Demo" -p <PASSWORD> -x PORT -o Asset_counts.txt
AREntryCounter.bat -k "<MigratorInstallationPath>\DeltaDataMigration\Packages\ITSM\<YOUR VERSION>\ITSM_Change_Package.xml" -d <DDM DATE> -s <SERVERNAME> -u "Demo" -p <PASSWORD> -x PORT -o Change_counts.txt
AREntryCounter.bat -k "<MigratorInstallationPath>\DeltaDataMigration\Packages\ITSM\<YOUR VERSION>\ITSM_SD_Package.xml" -d <DDM DATE> -s <SERVERNAME> -u "Demo" -p <PASSWORD> -x PORT -o ServicDesk_counts.txt
AREntryCounter.bat -k "<MigratorInstallationPath>\DeltaDataMigration\Packages\ITSM\<YOUR VERSION>\ITSM_Fnd_Package.xml" -d <DDM DATE> -s <SERVERNAME> -u <Demo> -p <PASSWORD> -x PORT -o ITSMFoundation_counts.txt
AREntryCounter.bat -k "<MigratorInstallationPath>\DeltaDataMigration\Packages\SRM\<YOUR VERSION>\SRM_Package.xml" -d <DDM DATE> -s <SERVERNAME> -u "Demo" -p <PASSWORD> -x PORT -o SRM_counts.txt
AREntryCounter.bat -k "<MigratorInstallationPath>\DeltaDataMigration\Packages\SLM\<YOUR VERSION>\SLM_Package.xml" -d <DDM DATE> -s <SERVERNAME> -u "Demo" -p <PASSWORD> -x PORT -o SLM_counts.txt
Product forms with additional data
The BMC Remedy ITSM version 9.1.02 SP2 installer creates additional data in the following forms, as explained in the Search For Additional Data section:
Product name | Form name |
---|---|
BMC Remedy AR System | Roles |
Report Definition | |
Group | |
User | |
AP:Detail | |
AP:Signature | |
AR System Tags | |
FB:History | |
Report | |
FB:History Summary | |
BMC Atrium CMDB | BMC.CORE.CONFIG:BMC_CIToConfigBaseRelationship |
BMC.CORE.CONFIG:BMC_FederatedInterfaceLink_ | |
BMC.CORE.CONFIG:BMC_UIComponent_ | |
RE:Activity | |
BMC.CORE.CONFIG:BMC_ConfigBaseRelationship | |
BMC.CORE.CONFIG:BMC_FederatedProductLink_ | |
CMDB.SC:ServiceContextAuthenticationInfo | |
CMDB.SC:ServiceContextProperties | |
BMC.CORE.CONFIG:BMC_FederatedInterface_ | |
BMC.CORE:BMC_Component_ | |
BMC.CORE:BMC_Dependency_ | |
BMC.CORE:BMC_RequestableOffering_ | |
RE:Job | |
BMC.CORE:BMC_Price_ | |
BMC.CORE:BMC_BaseRelationship | |
BMC.CORE:BMC_BaseElement | |
BMC.CORE.CONFIG:BMC_CMDBComponent_ | |
RE:Activity_Group_Assoc | |
RE:Group | |
RE:Identify_Rules | |
BMC.CORE.CONFIG:BMC_NormalizationRule_ | |
BMC.CORE.CONFIG:BMC_ConfigBaseElement | |
BMC Remedy Change Management | CHG:ChangeRiskDerivedFactorsTemplate |
CHG:ChangeRiskFactorsTemplate | |
CHG:Infrastructure Change | |
ITSM Foundation | CTM:AuditLogSystem |
AST:AssetAdvancedSearchCriteria | |
SHR:SchemaNames | |
APR:SYS-Approval Definition | |
KPI:DataCollection | |
SHR:ATR:HelpSystemTopicLookup | |
CAI:CommandParamsMapping | |
SYS:Notification Messages | |
TMS:AuditLogSystem | |
NTE:CFG-Notification Events | |
BMC Service Request Management | SRD:CFG DefinitionNumGenerator |
SRD:SrvReqDefFrequency | |
SRD:AuditLog | |
BMC Service Level Management | SLM:Measurement |
SLM:EventSchedule | |
SLM:GoalSchedule | |
SLM:RuleAction | |
SLM:RuleActionSetValue_base | |
SLM:RuleDefinition | |
SLM:RuleEventData | |
SLM:RuleActionSequence | |
SLM:RuleCondition | |
SLM:RuleActionSetValueItem | |
SLM:Association |
Where to go from here
Next task | Go to Performing prechecks before migrating delta data. |
---|---|
Up to process | To return to the delta data migration process, go to Migrating delta data. |
Comments
You're missing the Incident application from the documentation.