Recovering data

The VIS File ETL that runs in your environment might fail to collect data due to physical damage of the hard disk or any other reason. In such cases, you can recover this data loss by configuring and running the existing or new ETL. Depending on the size of your environment and recovery period, data recovery methods change as described in the following table.

Recovering data in a small environment

If you want to recover only the recent data (for the last one or two days), you can configure the last counter value for the existing ETLs in your environment. 

To update the last counter value, do the following:

1. Log in to BMC Helix Continuous Optimization.
2. Navigate to Administration > ETL & SYSTEM TASKS > ETL tasks.
3. Click the existing VIS File Parser ETL name.
4. Click Lastcounter > Edit lastcounter.
5. In the Lastcounter field, specify the start date from when you want to recover data. For example, if the existing timestamp is “2019-07-29 23:59:00”, change it to “2019-07-27 23:59:00” to recover data from 7/28 up to the current date. Ensure that the timing of the recovery run does not impact the already scheduled daily ETL runs.
6. Save the changes.
7. If you have the ETL task chain, repeat these steps for the other ETLs in the task chain.

When the ETL runs as scheduled, it recovers data for the specified period.

To recover the past data (three days or more), configure the new VIS File Parser ETL with data recovery options. For more information, see Recovering data in a large environment.

Recovering data in a large environment

To recover data in a large environment, BMC recommends configuring a new VIS File Parser ETL. Depending on how dynamic your environment is, you can recover data with or without importing the object relationships. BMC recommends recovering data without object relationships for faster ETL execution. If you want to import the object relationships, see Recovering data with object relationships.

You can use this method to recover data for any historical date range. You can configure multiple VIS File Parser ETLs across multiple ETL Engine Servers to recover data for more days in a single run. Otherwise, it is recommended to limit the recovery period to one or two days.

Prerequisites

Copy the VIS files to be recovered from the Gateway Server to a directory on the ETL Engine Servers where you plan to run the recovery ETLs.

To recover data, do the following:

1. Log in to BMC Helix Continuous Optimization.
2. Navigate to Administration > ETL & SYSTEM TASKS > ETL tasks.
3. Click Add > Add ETL.
4. From the ETL module list, select BMC - TrueSight Capacity Optimization Gateway VIS file parser.
5. Select the required platforms.
6. Select the required data types. The data types must match with the data types configured for the regular daily running ETL.
7. Share the entity catalog with the regular daily running ETL.
8. On the Vis file parser configuration tab, select via file as extractor mode.
9. On the File location tab, configure these properties:
   
   1. Specify the directory on the local ETL Engine Server where the VIS files to be recovered are available.
   2. Select no for recursing into subdirectories.
   3. Select Archive parsed file in directory for after parse operation.
   4. Retain the default directory location for the local archive directory or specify a directory that exists on the local ETL Engine Server.
   5. Retain the default settings for the other properties.
10. Click Advanced and configure the datasets property as follows:
    From the Selected datasets list, remove [59] OBJREL - Object relationships and apply the changes.
11. Save the changes and schedule the ETL run.
    When the ETL runs as scheduled, it recovers data

Similarly, you can configure recovery ETLs on other ETL Engine Servers.

Recovering data with object relationships

If you are using the VIS File Parser ETL to import data from highly dynamic environments, you can also import the object relationships while recovering data to capture the parent and child relationship changes during the recovery period.

Prerequisites

Copy the VIS files to be recovered from the Gateway Server to a directory on the ETL Engine Servers where you plan to run the recovery ETLs.

To recover data, do the following:

1. Log in to BMC Helix Continuous Optimization.
2. Navigate to Administration > ETL & SYSTEM TASKS > ETL tasks.
3. Click Add > Add ETL.
4. From the ETL module list, select BMC - TrueSight Capacity Optimization Gateway VIS file parser.
5. Select the required platforms.
6. Select the required data types. 
   The data types must match with the data types configured for the regular daily running ETL.
7. Share the entity catalog with the regular daily running ETL.
8. On the Vis file parser configuration tab, select via file as extractor mode.
9. On the File location tab, configure these properties:
   
   1. Specify the directory on the local ETL Engine Server where the VIS files to be recovered are available.
   2. Select no for recursing into subdirectories.
   3. Select Archive parsed file in directory for after parse operation.
   4. Retain the default directory location for the local archive directory or specify a directory that exists on the local ETL Engine Server.
   5. Retain the default settings for the other properties.
10. Click Advanced and set the data recovery mode to true. This setting ensures that the existing entities are not removed from the Workspace domain tree, and the relationships are built for the newly imported entities.
11. Save the changes and schedule the ETL run.

Similarly, you can configure recovery ETLs on other ETL Engine Servers.