Configuring clusters of Elasticsearch servers

After installing multiple Elasticsearch servers, you can configure them so they share data. Typically, you configure a cluster of Elasticsearch servers to replicate information so if one fails, other members of the cluster have access to the same information. You can install Elasticsearch on multiple servers before or after installing the application.

This topic includes the following sections:

Before you begin

Ensure that you have downloaded the Stack Manager tool (TSVM<versionNo>-SM-LIN64.zip) from the BMC Electronic Product Distribution (EPD) on the application server and Elasticsearch computers.

To configure multiple Elasticsearch servers

1. Install multiple Elasticsearch servers if you have not installed before installing the application.
   If you want to install multiple Elasticsearch servers, you must repeat the process for each instance.
2. Determine which instance of the Elasticsearch server functions as the master. Note the host name and IP address of the master server.
   This information is needed for the Elasticsearch servers to replicate their information between themselves. Any Elasticsearch server can function as the master.
3. On each Elasticsearch server, repeat the following steps:
   1. Open elasticsearch.yml for editing.
      Typically, this file resides at <ES_INSTALL_DIR>/elasticsearch/infra-ext/es/DCAIndexService/config.

   2. Insert the following entries anywhere in the file:
      

      network.bind_host: <IP address of the local host> 
      discovery.zen.ping.unicast.hosts: ["<IP address of master instance>"]  

      Note that for the master Elasticsearch server, these two entries would specify the same IP address.

   3. Save elasticsearch.yml.

   4. From the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and restart the Elasticsearch service (DCA Index Service) on each Elasticsearch server:

      python truesight-sm.py start --deployment elasticsearch

4. If you have installed Elasticsearch on multiple servers after installing the application, perform the following steps. Otherwise, skip to step 5.
   1. On the application server, modify the bmc-config.json file to specify the hosts that function as a cluster of Elasticsearch servers.
   2. Open the bmc-config.json file for editing.
      Typically, this file resides at <TSVM_INSTALL_DIR>/application/app/vulnerability-management-portal/data/configuration.
   3. In the file, find the following tag:
      "elasticssearch.hosts": 
   4. Modify the line so it now reads:
      "elasticssearch.hosts":"<Comma-separated list of hosts with HTTP port numbers>"
      where the comma-separated list identifies the hosts that function as Elasticsearch servers. The list does not have to include every host in the cluster. Instead, it can list only the Elasticsearch server hosts that route requests to other Elasticsearch servers. For example, 127.0.0.1:9300, 127.0.0.2:9301.

   5. Save bmc-config.json.

   6. On the application server machine, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and stop the application server by running the following command:

      python truesight-sm.py stop --deployment application

   7. Start the application server by running the following command:

      python truesight-sm.py start --deployment application

5. Create a PUT request:

   • Target this location: <hostName_any_ElasticsearchServer>:<httpPort>/settings
     For example, enter:
     indexserver1.mycompany.com:9200/settings


     Note

     To manage a cluster of Elasticsearch servers, you must use the underlying Elasticsearch search engine. Elasticsearch requires you to open the http port (default 9200) on the node where management is being performed.

   • In the body of the PUT request, enter the following:

     {
        "index":{
           "number_of_replicas":1
        }
     } 

     The number of replicas should equal the total number of Elasticsearch servers minus one. For example, if there are two Elasticsearch servers, enter a value of 1. If there are six Elasticsearch servers, enter a value of 5. Entering a value like this for number of replicas ensures that information is replicated between all Elasticsearch servers. 

      Click here for an example of a PUT request.

     There are many tools you can use to create a PUT request. The following example uses the cURL command line utility.

     curl -XPUT 'localhost:9200/my_index/_settings' -d '

     {

         "index" : {

             "number_of_replicas" : 1

         }

     }’ 

6. Send the PUT request.
   After sending this request to any node used as an Elasticsearch server, the cluster of Elasticsearch servers begins to replicate all data between themselves.
7. Confirm that replication was successful using the web-based front end for the Elasticsearch server.
   1.  Install the plug-in for the web-based front end.

      1. If your installation of TrueSight Vulnerability Management does not have Internet access, perform these preliminary steps. If your site does have Internet access, skip to step 2.
         1. Using a host that does have Internet access, visit https://github.com/mobz and download elasticsearch-head.zip.
         2. Move the downloaded file to a directory within your installation.

      2. On the application server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and stop the TrueSight Vulnerability Management services:

         python truesight-sm.py stop --deployment application

      3. On the Elasticsearch server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and make sure that the Elasticsearch (DCA Index) service  is running by checking its status:

         python truesight-sm.py status --deployment elasticsearch

      4. Navigate to the following directory: <ES_INSTALL_DIR>/elasticsearch/infra-ext/es/DCAIndexService/bin.
      5. Enter one of the following commands:
         • If you have Internet access:
           plugin -install mobz/elasticsearch-head

         • If you do not have Internet access:
           plugin --url file:<local_directory>/elasticsearch-head.zip --install mobz/elasticsearch-head 
           where <local_directory> is the location where you placed the downloaded ZIP file in step 1.

      6. On the application server, from the command prompt, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and restart the TrueSight Vulnerability Management services:

         python truesight-sm.py stop --deployment application
         python truesight-sm.py start --deployment application

   2. Check the front end to confirm that the correct number of Elasticsearch servers have been replicated.

Where to go next

If you have not installed the application earlier, install the TrueSight Vulnerability Management application.

To disable an unwanted Elasticsearch server, use the following command: python truesight-sm.py stop --deployment elasticsearch