Recovering data

Some times, the VIS File ETL that runs in your environment might fail to collect data due to physical damage of the hard disk or any other reason. In such cases, you can recover this data loss by configuring the existing or new ETL with data recovery options. Depending on the size of your environment and recovery period, data recovery methods change as described in the following table.

Environment size	Recovery period	Data recovery method
Small	Recent data (one or two days)	Configure the existing VIS File Parser ETL.
Small	Past data (a week or more)	Configure a new VIS File Parser ETL.
Large	Recent data (one or two days)	Configure the existing VIS File Parser ETL.
Large	Past data (a week or more)	Configure a new VIS File Parser ETL.

Recovering the recent data

To recover data for the last one or two days, you can configure the existing VIS File ETL.

To recover data, do the following:

1. Log in to the TrueSight Capacity Optimization console.
2. Navigate to Administration > ETL & SYSTEM TASKS > ETL tasks.
3. Click the existing VIS File Parser ETL name.
4. Click Lastcounter > Edit lastcounter.
5. In the Lastcounter field, specify the start date from when you want to recover data. For example, if the existing timestamp is “2019-07-29 23:59:00”, change it to “2019-07-27 23:59:00” to recover data from 7/28 up to the current date. Ensure that the timing of the recovery run does not impact the already scheduled daily ETL runs.
6. Save the changes.
7. If you have the ETL task chain, repeat these steps for the other ETLs in the task chain.

When the ETL runs as scheduled, it recovers data for the specified period.

Recovering the past data without object relationships

To recover the data for a specified period (a week or more), you can configure a new VIS File Parser ETL with the data recovery options. You can configure the ETL to skip recovering the object relationships to improve the ETL performance. In such a case, the object relationships defined during the previous ETL run are used. For a large environment, you can configure multiple recovery ETLs to recover more data in a single run to save time.

Prerequisites

Copy the VIS files to be recovered from the Gateway Server to a directory on the ETL Engine Servers where you plan to run the recovery ETLs.

To recover data, do the following:

1. Log in to the TrueSight Capacity Optimization console.
2. Navigate to Administration > ETL & SYSTEM TASKS > ETL tasks.
3. Click Add > Add ETL.
4. From the ETL module list, select BMC - TrueSight Capacity Optimization Gateway VIS file parser.
5. Select the required platforms.
6. Select the required data types. The data types must match with the data types configured for the regular daily running ETL.
7. Share the entity catalog with the regular daily running ETL.
8. On the Vis file parser configuration tab, select via file as extractor mode.
9. On the File location tab, configure these properties:
   
   1. Specify the directory on the local ETL Engine Server where the VIS files to be recovered are available.
   2. Select no for recursing into subdirectories.
   3. Select Archive parsed file in directory for after parse operation.
   4. Retain the default directory location for the local archive directory or specify a directory that exists on the local ETL Engine Server.
   5. Retain the default settings for the other properties.
10. Click Advanced and configure the datasets property as follows:
    From the Selected datasets list, remove [59] OBJREL - Object relationships and apply the changes.
11. Save the changes and schedule the ETL run.
    When the ETL runs as scheduled, it recovers data

Similarly, you can configure recovery ETLs on other ETL Engine Servers.

Recovering the past data with object relationships

You can recover the past data with the object relationships for any historical period. The import of object relationships ensures that the parent and child relationship changes during the recovery period are visible in the TrueSight Capacity Optimization console. However, BMC recommends using this method only when your environment is highly dynamic and requires the object relationship data to be imported. You can configure multiple recovery ETLs to recover more data in a single run to save time.

Prerequisites

Copy the VIS files to be recovered from the Gateway Server to a directory on the ETL Engine Servers where you plan to run the recovery ETLs.

To recover data, do the following:

1. Log in to the TrueSight Capacity Optimization console.
2. Navigate to Administration > ETL & SYSTEM TASKS > ETL tasks.
3. Click Add > Add ETL.
4. From the ETL module list, select BMC - TrueSight Capacity Optimization Gateway VIS file parser.
5. Select the required platforms.
6. Select the required data types. 
   The data types must match with the data types configured for the regular daily running ETL.
7. Share the entity catalog with the regular daily running ETL.
8. On the Vis file parser configuration tab, select via file as extractor mode.
9. On the File location tab, configure these properties:
   
   1. Specify the directory on the local ETL Engine Server where the VIS files to be recovered are available.
   2. Select no for recursing into subdirectories.
   3. Select Archive parsed file in directory for after parse operation.
   4. Retain the default directory location for the local archive directory or specify a directory that exists on the local ETL Engine Server.
   5. Retain the default settings for the other properties.
10. Click Advanced and set the data recovery mode to true. This setting ensures that the existing entities are not removed from the Workspace domain tree, and the relationships are built for the newly imported entities.
11. Save the changes and schedule the ETL run.

Similarly, you can configure recovery ETLs on other ETL Engine Servers.