Migrating data by using the Export-Import utility

Use the data Export-Import utility to perform data migration from BMC Helix Digital Workplace 20.02.x to 23.3.04 version.

The utility migrates BMC Helix Digital Workplace data from a noncontainerized environment to a containerized environment. It migrates all the existing database tables along with the system and business schema. However, the utility does not migrate the Liquibase tables, such as DATABASECHANGELOG and DWP_BUS_DATABASECHANGELOG.

Perform the following actions to migrate data:

The export utility creates a ZIP file such as export.zip that includes the following files:

• XML file—contains the BMC Helix Digital Workplace business and system schema data.
• JSON file—contains theBMC Helix Digital Workplace social data.

The import utility reads the XML and JSON files and makes API calls to write relevant BMC Helix Digital Workplace record definitions in BMC Helix Innovation Studio.

Before you begin

• Download the export zip utility and import zip utility files from BMC Electronic Product Distribution (EPD).

• 1. Log in to EPD.
     You must log in with your BMC Support ID.

  2. Select Product Downloads > Product Download Tool.
  3. On the All Products tab, in the search box, search for the BMC Helix Digital Workplace Advanced OnPrem product.
  4. Click the 23.3.04 product version.
  5. In the BMC Helix Digital Workplace Advanced OnPrem page, on the Product tab, select BMC Helix Digital Workplace OnPrem Data Migration Tool Version 23.3.03, and click Download.
     The DWP_23.3.03_Export_Import_Utility file is downloaded.

• Make sure that your system meets the following requirements:

  Requirements	Description
  Hardware	Minimum 8 CPU with 32 GB memory
  Software	CentOS 7.6, Windows 10, or Windows 11 Java 17

  If you do not have Java installed on your system, to install Java 17, by using the following command:

  yum install java-17-openjdk-devel

  Set the JAVA_HOME variable in your environment.

• Confirm that you have the following BMC Helix Digital Workplace database parameter details:

  Database	Parameters
  Microsoft SQL Server	Database host name Database service port JDBC URL Use the following URL format: jdbc:sqlserver://<dbhost>:<dbport>;DatabaseName=<database name> Database name Business schema user ID and password System schema user ID and password
  Oracle	Database host name Database service port JDBC URL Use the following URL format: jdbc:oracle:thin:@<dbhost>:<dbport>:<sid>jdbc:oracle:thin:@<dbhost>:<dbport>/<service name> Service name or service ID Business schema user ID and password System schema user ID and password

• Make sure that you meet the following requirements:
  
  • You have permissions to access the databases in the BMC Helix Digital Workplace source environment.
  • Do not use the social and BMC Helix Digital Workplace application services during the data migration.
  • BMC Helix Innovation Studio is up and running on the target environment.

• 
  
  • BMC Helix Digital Workplace 23.3.03 smart bundle is deployed on BMC Helix Innovation Studio.

To set up the export utility

1. Log in to the virtual machine where you want to run the export utility.

2. In your virtual machine, create a directory to unzip the export utility file by using the following command:

   mkdir <directory name>

   For example:

   mkdir export200203

3. Copy the export utility.zip file from the Export Utility_20.02.03 or Export Utility_20.02.02 directory based your source version to the directory that you create.

1. Unzip the ZIP file by using the following command:

   unzip <file name>

   For example:

   unzip unzip myit-data-transfer-20.02.01.000-BUILD-SNAPSHOT-zip-with-dependencies.zip

   The following file structure is displayed:
   
   

2. Set up the database connections.

1. 
   
   1. Navigate to the connection directory.

   2. Depending on the database that you want to use, update the mssql.connection.properties or oracle.connection.properties file with the database details.
      To update mssql.connection.properties file, refer the following example:

      #jdbc_url=
      host=XXX-CVV-DDF22334.<domain>.com
      port=1433
      business_db_name=DWP200203
      business_user=DWP200203_Business
      business_password= password
      system_db_name=DWP200203
      system_user=DWP200203_System
      system_password= password

      To update the oracle.connection.properties file, refer the following example:

      #jdbc_url=
      host=XXX-CVV-DDF22334.<domain>.com
      port=1521
      #service if preferable to use instead of sid
      business_service_name=ORA12C.<domain>.com
      #sid
      business_user=DWP
      business_password=password
      system_service_name=ORA12C.<domain>.com
      system_user=DWP_SYS
      system_password=password

      Important

      If you set the jdbc_url parameter, do not use the host, port, and service_name parameters.

      Make sure that you comment the parameters that you do not want to use in the mssql.connection.properties or oracle.connection.properties file.

To run the export utility

1. Modify the set_env.sh file.
   1. Navigate to the scripts directory.

   2. Edit the set_env.sh file, specify the parameter values based on the setup, and save the file.

      Setup	Description
      Non managed service provider (MSP)	Specify the parameter values to export data for the default tenant. The default tenant ID is 000000000000001. Example: java_home="/urs/local/jdk-17.0.2" readonly tenant_id=000000000000001 readonly db_type=MSSQL (Specify the value as ORACLE for an Oracle database)  readonly db_connection_properties_file="../connection/mssql.connection.properties"     (Specify the value as oracle.connection.properties for an Oracle database) readonly output_file_path="export.zip"
      MSP	Specify the parameter values to export data for the default tenant and all its subtenants. Make sure that you specify the default tenant ID with percent sign % in tenant_id field such as 000000000000001% Example: java_home="/urs/local/jdk-17.0.2" readonly tenant_id=000000000000001% readonly db_type=MSSQL   (Specify the value as ORACLE for an Oracle database) readonly db_connection_properties_file="../connection/mssql.connection.properties"    (Specify the value as oracle.connection.properties for an Oracle database) readonly output_file_path="export.zip" Important: If you want to export data for all the MSP tenants, use the non MSP setup method. Export data for the default tenant first and then its subtenants. Make sure that you specify the tenant_id parameter value as allTenants. Make sure that you update the output_file_path value for each tenant, such as for tenant 000000000000001, specify the output_file_path as 000000000000001_export.zip and for tenant 000000000000001-1, specify the value as 000000000000001-1_export.zip
      External user	An external setup has a default tenant and external user tenants. You must export data for the default tenant first and then the external tenants. For each export, make sure that you update the output_file_path name by using a naming convention so that the file is not overwritten. For example, for tenant 000000000000001, specify the name as 000000000000001_export.zip. For tenant 000000000000001-1, specify the name as 000000000000001-1_export.zip. Default tenant example: java_home="/urs/local/jdk-17.0.2" readonly tenant_id=000000000000001 readonly db_type=MSSQL    (Specify the value as ORACLE for an Oracle database) readonly db_connection_properties_file="../connection/mssql.connection.properties"     (Specify the value as oracle.connection.properties for an Oracle database) readonly output_file_path="000000000000001_export.zip" External tenant example: java_home="/urs/local/jdk-17.0.2" readonly tenant_id=000000000000001-1 readonly db_type=MSSQL   (Specify the value as ORACLE for an Oracle database) readonly db_connection_properties_file="../connection/mssql.connection.properties"      (Specify the value as oracle.connection.properties for an Oracle database) readonly output_file_path="000000000000001-1_export.zip"

2. From the command prompt, run the following command:

   ./run_pre_export.sh

   This command helps you estimate the data export time and provides the following information:

   • Number of records and total size of attachments in the ATTACHMENT and SOCIAL_ATTACHMENT tables

   • List of first 20 attachments greater than 5 MB from each table
   • Option to skip exporting the attachments greater than 5 MB.
   • Calculation of heap size required for data transfer.
     You need the heap size to set the java_opts parameter in the set_env.sh file.

3. Start the data export by using the following command:

   ./run_export.sh

   You can observe the logs in the console. After completion of data export, the log directory and new export.zip file are available in the scripts directory.

4. Perform delta data export.

   Important

   While performing the delta data export, make sure that you follow the same data export approach that you followed during the full data export.
   

   For example, if you performed the MSP data export for all the tenants at a time during full data export, perform an MSP data export for all the tenants at a time during delta data export.

   1. Navigate to the scripts directory.

   2. Edit the set_env.sh file, specify the parameter values, and save the file.

      Important

      Specify the since_date parameter value in the yyyy-MM-dd'T'HH:mm:ssZ format.

      The since_date parameter specifies the date and time from which you want to start data export.

      Example:

      java_home="/urs/local/jdk-17.0.2"
      readonly tenant_id=000000000000001
      readonly db_type=MSSQL    (Specify the value as ORACLE for an Oracle database)  
      readonly db_connection_properties_file="../connection/mssql.connection.properties"      
      (Specify the value as oracle.connection.properties for an Oracle database)   
      readonly output_file_path="export_delta.zip"   
      (You can provide new name to export file to avoid confusion with main/previously exported file)
      
      #Date in the ISO 8601 format yyyy-MM-ddTHH:mm:ssZ since which entities will be exported or imported.
      If not specified all entities will be loaded
      readonly since_date=2021-08-19T06:00:04Z
      
      

   3. Start the data export by using the following command:

      ./run_export.sh

To set up the import utility

1. Log in to the virtual machine where you want to run the import utility; use the same virtual machine where you run the export utility.

1. Create a directory to unzip import utility file by using the following command:

   mkdir <directory name>

   For example:

   mkdir import221006
2. Copy the import utility.zip from the Import Utility_23.3.03 directory to the directory that you create.

3. Unzip the ZIP file by using the following command:

   unzip <file name>

   unzip myit-data-transfer-23.3.03.000-BUILD-SNAPSHOT-zip-with-dependencies.zip

To run the import utility

1. Navigate to the scripts directory.
2. Copy the export.zip file to the scripts directory.
3. Update the environment variables by using any of the following ways:

1. • Run the following command each time before you start the data import.

     export INNOVATION_SUITE_ROOT_URL=<Innovation Studio URL> && export IS_USER=<Innovation Studio Administrator Username> && export IS_PASSWORD=<Innovation Studio Administrator User Password> && export SERVER_RPC_PORT=46262 && export SERVER_HOST=<External IP of platform-user-0 pod> && export IMPORT_FILE_PATH=export.zip
   • Set environment variables in bash profile file.

     1. Edit the bash_profile file by using the following command:

        vim .bash_profile

     2. Update the following environment variables in the bash_profile file:

        export INNOVATION_SUITE_ROOT_URL=<Innovation Studio URL>
        export IS_USER=<Innovation Studio Administrator Username>
        export IS_PASSWORD=<Innovation Studio Administrator User Password >
        export SERVER_RPC_PORT=46262
        export SERVER_HOST=<External IP of platform-user-0 pod>
        export IMPORT_FILE_PATH=<Export file name path along with zip file name>

        INNOVATION_SUITE_ROOT_URL example value is as follows:
        https://xxxx.onbmc.com/

     3. Save the file by using the following command:

        source ~/.bash_profile

2. Start the data import by using the following command:

   ./ run_import.sh

   You can observe the logs in the console. After completion of data import, you might need to restart the BMC Helix Digital Workplace Tomcat pod to view that the imported data.

3. (Optional) Import data for a subtenant or external user tenant .

   1. Update the IMPORT_FILE_PATH variable with the file name that contains the tenant exported data by using the following command:

      export IMPORT_FILE_PATH=<file name>

      For example, for an external user tenant with file name 000000000000001-1_export.zip, run the following command:

      export IMPORT_FILE_PATH=000000000000001-1_export.zip

   2. Start the data import by using the following command:

      ./ run_import.sh

      You can observe the logs in the console. After completion of data import, you might need to restart the BMC Helix Digital Workplace Tomcat pod to view that the imported data.
      
      

Troubleshooting

Refer to the troubleshooting information if you encounter issues during BMC Helix Digital Workplace data migration:

Where to go from here

Migrating-Remedy-on-premises-to-BMC-Helix-Service-Management-on-premises