Migrating data by using the Export-Import utility

Use the data Export-Import utility to perform data migration from BMC Helix Digital Workplace 20.02.x to 21.3.06 version.

The utility migrates BMC Helix Digital Workplace data from a noncontainerized environment to a containerized environment. It migrates all the existing database tables along with the system and business schema. However, the utility does not migrate the Liquibase tables, such as DATABASECHANGELOG and DWP_BUS_DATABASECHANGELOG.

Perform the following actions to migrate data:

The export utility creates a ZIP file such as export.zip that includes the following files:

XML file—contains the BMC Helix Digital Workplace business and system schema data.
JSON file—contains theBMC Helix Digital Workplace social data.

The import utility reads the XML and JSON files and makes API calls to write relevant BMC Helix Digital Workplace record definitions in BMC Helix Innovation Studio.

Before you begin

Download the export zip utility and import zip utility files from BMC Electronic Product Distribution (EPD).
1. Log in to EPD .
  You must log in with your BMC Support ID.
2. Select Product Downloads > Product Download Tool.
3. On the All Products tab, in the search box, search for the BMC Helix Digital Workplace Advanced OnPrem product.
4. Click the 21.3.06 product version.
5. In the BMC Helix Digital Workplace Advanced OnPrem page, on the Product tab, select BMC Helix Digital Workplace OnPrem Data Migration Tool Version 21.3.06, and click Download.
  The DWP_21.3.06_Export_Import_Utility file is downloaded.
Make sure that your system meets the following requirements:
Requirements Description
Hardware Minimum 8 CPU with 32 GB memory
Software
CentOS 7.6
Java 1.8 or 11
If you do not have Java installed on your system, to install Java 11, by using the following command:
```
yum install java-11-openjdk-devel
```
To install Java 1.8, use the following command:
```
yum install java-1.8.0-openjdk-devel
```
Set the JAVA_HOME variable in your environment.

Confirm that you have the following BMC Helix Digital Workplace database parameter details:

Database	Parameters
Microsoft SQL Server	Database host name Database service port JDBC URL Database name Business schema user ID and password System schema user ID and password
Oracle	Database host name Database service port JDBC URL Service name or service ID Business schema user ID and password System schema user ID and password

Make sure that you meet the following requirements:
- You have permissions to access the databases in the BMC Helix Digital Workplace source environment.
- Do not use the social and BMC Helix Digital Workplace application services during the data migration.
- BMC Helix Innovation Studio is up and running on the target environment.

- BMC Helix Digital Workplace 21.3.06 smart bundle is deployed on BMC Helix Innovation Studio.

To run the export utility

Important

You must export data from a single master tenant and its subtenants. BMC Helix Innovation Suite is a single tenant system and you cannot migrate data from multiple master BMC Helix Digital Workplace tenants to BMC Helix Innovation Suite.

Log in to the virtual machine where you want to run the export utility.
In your virtual machine, create a directory to unzip the export utility file by using the following command:
```
mkdir <directory name>
```
For example:
```
mkdir export200203
```
Copy the export utility.zip file to the directory that you create.
Unzip the myit-data-transfer-20.02.03.000-BUILD-SNAPSHOT-zip-with-dependencies.zip file by using the following command:
```
unzip myit-data-transfer-20.02.03.000-BUILD-SNAPSHOT-zip-with-dependencies.zip
```
The following file structure is displayed:

To set up database connections

Navigate to the connection directory.

Depending on the database that you want to use, update the mssql.connection.properties or oracle.connection.properties file with the database details.
To update mssql.connection.properties file, refer the following example:

#jdbc_url=
host=XXX-CVV-DDF22334.<domain>.com
port=1433
business_db_name=DWP200203
business_user=DWP200203_Business
business_password= password 
system_db_name=DWP200203
system_user=DWP200203_System
system_password= password

To update the oracle.connection.properties file, refer the following example:

#jdbc_url=
host=XXX-CVV-DDF22334.<domain>.com
port=1521
#service if preferable to use instead of sid
business_service_name=ORA12C.<domain>.com
#sid
business_user=DWP
business_password=password
system_service_name=ORA12C.<domain>.com
system_user=DWP_SYS
system_password=password

Important

If you set the jdbc_url parameter, do not use the host, port, and service_name parameters.

Make sure that you comment the parameters that you do not want to use in the mssql.connection.properties or oracle.connection.properties file.

To export data

Modify the set_env.sh file.

Navigate to the scripts directory.

Edit the set_env.sh file, specify the parameter values based on the setup, and save the file.

Setup	Description
Non managed service provider (MSP)	Specify the parameter values to export data for the default tenant. The default tenant ID is 000000000000001. Example: `java_home="/urs/local/jdk1.8.0_91"` `readonly tenant_id=000000000000001` `readonly db_type=MSSQL (Specify the value as ORACLE for an Oracle database)` `readonly db_connection_properties_file="../connection/mssql.connection.properties"` `(Specify the value as oracle.connection.properties for an Oracle database)` `readonly output_file_path="export.zip"`
MSP	Specify the parameter values to export data for the default tenant and all its subtenants. Make sure that you specify the default tenant ID with percent sign % in tenant_id field such as 000000000000001% Example: `java_home="/urs/local/jdk1.8.0_91"` `readonly tenant_id=000000000000001%` `readonly db_type=MSSQL (Specify the value as ORACLE for an Oracle database)` `readonly db_connection_properties_file="../connection/mssql.connection.properties"` `(Specify the value as oracle.connection.properties for an Oracle database)` `readonly output_file_path="export.zip"` Important: If you want to export data for all the MSP tenants, use the non MSP setup method. Export data for the default tenant first and then its subtenants. Make sure that you update the output_file_path value for each tenant, such as for tenant 000000000000001, specify the output_file_path as 000000000000001_export.zip and for tenant 000000000000001-1, specify the value as 000000000000001-1_export.zip
External user	An external setup has a default tenant and external user tenants. You must export data for the default tenant first and then the external tenants. Default tenant example: `java_home="/urs/local/jdk1.8.0_91"` `readonly tenant_id=000000000000001` `readonly db_type=MSSQL (Specify the value as ORACLE for an Oracle database) readonly db_connection_properties_file="../connection/mssql.connection.properties" (Specify the value as oracle.connection.properties for an Oracle database) readonly output_file_path="000000000000001_export.zip "` External tenant example: `java_home="/urs/local/jdk1.8.0_91" readonly tenant_id=000000000000001-1 readonly db_type=MSSQL (Specify the value as ORACLE for an Oracle database) readonly db_connection_properties_file="../connection/mssql.connection.properties" (Specify the value as oracle.connection.properties for an Oracle database) readonly output_file_path="000000000000001-1_export.zip "`

From the command prompt, run the following command:
```
./run_pre_export.sh
```
This command helps you estimate the data export time and provides the following information:
- Number of records and total size of attachments in the ATTACHMENT and SOCIAL_ATTACHMENT tables
- List of first 20 attachments greater than 5 MB from each table
- Option to skip exporting the attachments greater than 5 MB.
- Calculation of heap size required for data transfer.
  You need the heap size to set the java_opts parameter in the set_env.sh file.
Start the data export by using the following command:
```
./run_export.sh
```
You can observe the logs in the console. After completion of data export, the log directory and new export.zip file are available in the scripts directory.
Perform delta data export.
Important

While performing the delta data export, make sure that you follow the same data export approach that you followed during the full data export.
For example, if you performed the MSP data export for all the tenants at a time during full data export, perform an MSP data export for all the tenants at a time during delta data export.
1. Navigate to the scripts directory.
2. Edit the set_env.sh file, specify the parameter values, and save the file.
  Important
  
  Specify the since_date parameter value in the yyyy-MM-dd'T'HH:mm:ssZ format.
  The since_date parameter specifies the date and time from which you want to start data export.
  Example:
```
java_home="/urs/local/jdk1.8.0_91"
readonly tenant_id=000000000000001
readonly db_type=MSSQL   (Specify the value as ORACLE for an Oracle database) 
readonly db_connection_properties_file="../connection/mssql.connection.properties"      
(Specify the value as oracle.connection.properties for an Oracle database)  
readonly output_file_path="export_delta.zip"   
(You can provide new name to export file to avoid confusion with main/previously exported file)

#Date in the ISO 8601 format yyyy-MM-ddTHH:mm:ssZ since which entities will be exported or imported. 
If not specified all entities will be loaded
readonly since_date=2021-08-19T06:00:04Z
```
3. Start the data export by using the following command:
```
./run_export.sh
```

To run the import utility

Log in to the virtual machine where you want to run the import utility; use the same virtual machine where you run the export utility.
Create a directory to unzip import utility file by using the following command:
```
mkdir <directory name>
```
For example:
```
mkdir import210502
```
Copy the import utility.zip to the directory that you create.
Unzip the myit-data-transfer-21.05.01.000-BUILD-SNAPSHOT-zip-with-dependencies.zip file by using the following command:
```
unzip myit-data-transfer-21.05.01.000-BUILD-SNAPSHOT-zip-with-dependencies.zip
```

To import data

Navigate to the scripts directory.
Copy the export.zip file to the scripts directory.
Edit the bash_profile file by using the following command:
```
vim .bash_profile
```

Update the following environment variables in the bash_profile file:

export INNOVATION_SUITE_ROOT_URL=<Innovation Studio URL>
export IS_USER=<Innovation Studio Administrator Username>
export IS_PASSWORD=<Innovation Studio Administrator User Password >
export SERVER_RPC_PORT=46262
export SERVER_HOST=<External IP of platform-user-0 pod>
export IMPORT_FILE_PATH=<Export file name path along with zip file name>

INNOVATION_SUITE_ROOT_URL example value is as follows:
https://xxxx.onbmc.com/

Save the file by using the following command:
```
source ~/.bash_profile
```
Start the data import by using the following command:
```
./ run_import.sh
```
You can observe the logs in the console. After completion of data import, you might need to restart the BMC Helix Digital Workplace Tomcat pod to view that the imported data.
(Optional) Import a subtenant data or external user tenant data.
1. Update the IMPORT_FILE_PATH variable with the file name that contains the tenant exported data by using the following command:
```
export IMPORT_FILE_PATH=<file name>
```
  For example, for an external user tenant with file name 000000000000001-1_export.zip, run the following command:
```
export IMPORT_FILE_PATH=000000000000001-1_export.zip
```
2. Start the data import by using the following command:
```
./ run_import.sh
```

Important

After you perform a delta data migration, make sure that you restart the BMC Helix Digital Workplace Tomcat pod.

Troubleshooting

Refer to the troubleshooting information if you encounter issues during BMC Helix Digital Workplace data migration:

Issue	Resolution
Data export fails	Check the log files generated by the utility to find the errors. Check the connectivity between the virtual machine where the utility is running and BMC Helix Innovation Studio. Verify that you configured the following variable values correctly as per the target environment: INNOVATION_SUITE_ROOT_URL IS_USER IS_PASSWORD SERVER_RPC_PORT IMPORT_FILE_PATH
Data import fails	Start the data import again by using the failed_records.zip as an input to the import utility. Records that failed to import data are stored in failed_records.zip file.
Data import of a subtenant fails	Verify that the subtenant company is available in BMC Helix ITSM.

Where to go from here

Staged upgrade process

Requirements	Description
Hardware	Minimum 8 CPU with 32 GB memory
Software	CentOS 7.6 Java 1.8 or 11

Migrating data by using the Export-Import utility

Before you begin

To run the export utility

To set up database connections

To export data

To run the import utility

To import data

Troubleshooting

Where to go from here

Comments