Unsupported content This version of the product has reached end of support. The documentation is available for your convenience. However, you must be logged in to access it. You will not be able to leave comments.

Backing up and restoring Elasticsearch server data

Elasticsearch servers store all data used in TrueSight Vulnerability Management. Backing up data in Elasticsearch servers should be part of the regular maintenance of a TrueSight Vulnerability Management deployment.

An automatic backup procedure is described below. In the procedure, you must configure some values that specify where data is stored and how often a backup should occur. 

In rare situations, you might need to restore Elasticsearch server data that was previously archived. A restore procedure is described below.

Important

Frequency for backups of Elasticsearch server data depends on how often you import scan files and map their contents to the contents of your TrueSight system. If you are using vulnerability data daily, then backups should occur daily.

This topic contains the following sections:

Before you begin

Ensure that you have downloaded the Stack Manager tool (TSVM<versionNo>-SM-LIN64.zip) from EPD.

To back up Elasticsearch data

This backup capability can store a snapshot of the data in the Elasticsearch server to a local or remote repository.

This procedure requires you to modify two configuration files.

If you have installed multiple instances of the Vulnerability Management server, you must ensure that each instance has an identical backup configuration.

  1. On the application server, configure the bmc-config.json file, as follows:
    1. Open bmc-config.json for editing.
      Typically, this file resides at <TSVM_INSTALL_DIR>/application/app/vulnerability-management-portal/data/configuration.

    2. In the file, search for the term ElasticSearchConfigManager. The section that includes that term appears as follows

      "com.bmc.dcaportal.index.service.core.ElasticSearchConfigManager" : {
           "indexsearch.enabled" : "true",
           "cluster.name" : "dca-index",
           "loader.maxBSAPageSize" : 1000,
           "aggregation.precision.count" : 500,
       "vulnerability.management.circuit.breaker" : 1500000,
       "circuit.breaker.threshold" : 95,
           "loader.threadPoolSize" : 5,
           "purge.interval" : 10,
             "index.backup.path" : "",
             "index.backup.timeinterval" : 1440
         },

    3. Provide the following values, as necessary:

      Option

      Explanation

      index.backup.path

      Provides the path to local or remote location where data should be stored. Be sure you have correct permissions for the location you specify.
      For example, you might enter:

      "index.backup.path" : "opt/bmc/truesight/INDEX_BACKUP", 

      index.backup.timeinterval

      Specifies how often a backup of the Elasticsearch server data should occur. The value you enter is in minutes. By default, a backup occurs every 24 hours.

    4. Save bmc-config.json.
  2. On the Elasticsearch server, configure the elasticsearch.yml file.
    1. Open elasticsearch.yml for editing.
      Typically, this file resides at <ES_INSTALL_DIR>/elasticsearch/infra-ext/es/DCAIndexService/config. 

    2. Insert the following entry anywhere in the file:

      path.repo: <Path_to_backup_directory> 

      For example, you might enter:
      path.repo: opt/bmc/truesight/INDEX_BACKUP
      The value you enter must match the value of index.backup.path that you specified in the bmc-config.json file. 

    3. Save elasticsearch.yml.

  3. On the application server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and restart the TrueSight Vulnerability Management services:

    python truesight-sm.py stop --deployment application
    python truesight-sm.py start --deployment application

  4. On the Elasticsearch server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory, and restart the Elasticsearch (DCA Index) service:

    python truesight-sm.py stop --deployment elasticsearch
    python truesight-sm.py start --deployment elasticsearch
  5. Repeat this procedure for each instance of the TrueSight Vulnerability Management server.

To restore Elasticsearch server data

Restoring Elasticsearch server data requires you to install a plugin that provides a web-based front end to the Elasticsearch server. After the plugin is installed, there are two possible restore procedures:

Installing the plugin for the web-based front end

  1. If your installation of TrueSight Vulnerability Management does not have Internet access, perform these preliminary steps. If your site does have Internet access, skip to step 2.
    1. Using a host that does have Internet access, visit https://github.com/mobz and download elasticsearch-head.zip.
    2. Move the downloaded file to a directory within your installation.

  2. On the application server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and stop the TrueSight Vulnerability Management services:

    python truesight-sm.py stop --deployment application

  3. On the Elasticsearch server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and make sure that the Elasticsearch (DCA Index) service  is running by checking its status:

    python truesight-sm.py status --deployment elasticsearch
  4. Navigate to the following directory: <ES_INSTALL_DIR>/elasticsearch/infra-ext/es/DCAIndexService/bin.
  5. Enter one of the following commands:
    • If you have Internet access:
      plugin -install mobz/elasticsearch-head
    • If you do not have Internet access:
      plugin --url file:<local_directory>/elasticsearch-head.zip --install mobz/elasticsearch-head 
      where <local_directory> is the location where you placed the downloaded ZIP file in step 1.

  6. On the application server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and restart the TrueSight Vulnerability Management services:

    python truesight-sm.py stop --deployment application
    python truesight-sm.py start --deployment application

Restoring the most recent snapshot

  1. On the application server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and stop the TrueSight Vulnerability Management application services.

    python truesight-sm.py stop --deployment application

  2. On the Elasticearch server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm and make sure that the Elasticsearch (DCA Index) service is running by checking its status:

    python truesight-sm.py status --deployment elasticsearch
  3. Start the plugin for the Elasticsearch server:
    • If you have Internet access, use the following URL: <hostNameElasticsearchServer>:<httpPortNo>/_plugin/head
      For example, enter tsvmserver.mycompany.com:9200/_plugin/head

    • If you do not have Internet access, use the following URL: http://localhost:9200/_plugin/head/

      Note

      To manage Elasticsearch servers, you must use the underlying Elasticsearch search engine. Elasticsearch requires you to open the http port (default 9200) on the node where management is being performed.

  4. If it is not already selected, click the Any Request tab.
  5. Stop all indices in the Elasticsearch server, as follows:
    1. In the Query section, enter _all/_close.
    2. From the drop-down menu at right, select POST.
    3. Click Request.
      SearchServerFrontEnd.png 
  6. Restore the latest snapshot.
    1. In the Query section, enter _snapshot/dca-index-repository/dca-index-snapshot/_restore.
    2. From the drop-down menu at right, select POST.
    3. Click Request.

  7. On the application server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and restart the TrueSight Vulnerability Management services:

    python truesight-sm.py stop --deployment application
    python truesight-sm.py start --deployment application

  8. On the Elasticsearch server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and restart the Elasticsearch (DCA Index) service:

    python truesight-sm.py stop --deployment elasticsearch
    python truesight-sm.py start --deployment elasticsearch
  9. Confirm that all vulnerability management asset and vulnerability data is available.

Restoring the most recent snapshot to a corrupted data folder

In situations where the Elasticsearch's data folder (<TSVM_INSTALL_DIR>/elasticsearch/infra-ext/es/DCAIndexService) has become corrupted, you must perform a restore procedure like the one described above but you must also run a command to manually create a repository for metadata.

  1. On the application server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and stop the TrueSight Vulnerability Management services:

    python truesight-sm.py stop --deployment application

  2. On the Elasticsearch server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and make sure that the Elasticsearch (DCA Index) service is running by checking its status:

    python truesight-sm.py status --deployment elasticsearch
  3. Start the plugin for the Elasticsearch server:
    • If you have Internet access, use the following URL: <hostNameElasticsearchServer>:<httpPort>/_plugin/head
      For example, enter tsvmserver.mycompany.com:9200/_plugin/head

    • If you do not have Internet access, use the following URL: http://localhost:9200/_plugin/head/

      Note

      To manage Elasticsearch servers, you must use the underlying Elasticsearch search engine. Elasticsearch requires you to open the http port (default 9200) on the node where management is being performed.

  4. If it is not already selected, click the Any Request tab.
  5. Stop all indices in the Elasticsearch server, as follows:
    1. In the Query section, enter _all/_close.
    2. From the drop-down menu at right, select POST.
    3. Click Request.
  6. Create a repository for metadata to replace the corrupted repository.

    1. In the Query section, enter the following:

      _snapshot/dca-index-repository
      {
      "type": "fs",
      "settings": {
      "location": "<Snapshot location>"
      }
      }
    2. From the drop-down menu at right, select PUT.
      For example, the web-based front end would look something like this:
      SearchServer2.png 
    3. Click Request.
  7. Restore the latest snapshot.
    1. In the Query section, enter _snapshot/dca-index-repository/dca-index-snapshot/_restore.
    2. From the drop-down menu at right, select POST.
    3. Click Request.

  8. On the application server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and restart the TrueSight Vulnerability Management services:

    python truesight-sm.py stop --deployment application
    python truesight-sm.py start --deployment application

  9. On the Elasticsearch server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and restart the Elasticsearch (DCA Index) service:

    python truesight-sm.py stop --deployment elasticsearch
    python truesight-sm.py start --deployment elasticsearch
  10. Confirm that all vulnerability management asset and vulnerability data is available.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*