Indexer cluster

This topic provides information about what an Indexer cluster means and how it functions.

The Indexer cluster can contain one or more Indexers. The Indexer that you install for the first time (on the primary node) must be the master. All other Indexers are considered as peers and they must be provided with details of the master. If you upgrade to the new version of the product, the initial Indexer is automatically considered as the master.

Within the Indexer cluster, the master plays an important role of the coordinator. The master Indexer (installed on the primary node) communicates and coordinates with the peer Indexers.

The Indexer cluster is responsible for performing the following two functions:

• Indexing data collected as per indexing requests received from the data collectors.
• Returning search results as per search requests received.

When the Collection Station starts up, it contacts the master for the indexing activity. At this time the Collection Station receives all the details about the Indexer cluster, it directly communicates with the specific Indexers for the indexing requests sent. The indexing activity for the data collected is distributed across all the Indexers in the cluster. The Search component works in the same way as the Collection Station; the Search component gets details about the cluster from the master on starting up and sends requests directly to the specific Indexers in the cluster.

The master keeps details about the peers that exist in the cluster, it coordinates with the peers, and maintains the cluster state by acting each time a peer joins or leaves the cluster. The master also allocates shards to the peers in the cluster. If the master goes down, the indexing and search functions stop working and you might experience data loss. If any of the peers go down then the indexing and search functions continue to work, but you might experience partial data loss.

When a peer goes offline temporarily, the data held by the peer is not available for search, for the time when the peer is offline. The peer can go offline due to various reasons, for example, network issues. After the peer comes online, the past data held by the peer becomes available for search. But during the time when the peer is offline, the data collectors are unable to connect and send data (for indexing) to the peer. Data sent to the peer in the time when it was offline is lost.

Each time you want to shut down an Indexer node, use the shutdown CLI command to stop the whole Indexer cluster, instead of stopping individual Indexer nodes. This is important to avoid data loss and performance degradation.