Understanding Indexers, Indexes and Index Blocks


Use this topic to understand how data is indexed in Index Blocks. For that, the following terms need to be understood:

Indexer: An Indexer is a component of IT Data Analytics that acts as an internal data store for storing data collected by Collection Stations and Collection Agents. When you search for data, it is the Indexer component that provides the data input. Indexers provide indexing capability for all the log data collected.

Index: An Index is the way data is organized within an Indexer component. 

Index Block: An Index Block is a conceptual group of indexes whose data have the same properties such as retention period and archive configurations. A group of data collectors mapped to the same Index Block would have the same archive and retention properties.

Related topics

Collecting-data-into-the-system

Setting-up-data-collection

Creating-data-collectors

Assigning-the-data-pattern-and-date-format-to-a-data-collector

Understanding-data-retention-and-deletion

How an Indexer stores data

An Indexer stores data organised into groups of indexes called Index Blocks. Generally there are a group of data collectors whose retention is same and they have the same archive status. Data from this group of data collectors is ingested in the same Index Block. For more information about Index Blocks, see About Index Blocks.

There are three types of Indexer nodes, Live, Archive, and Restore nodes. Data from all three types of Indexer nodes is searchable.

  • Live node: Data that is current is written into the Live node in Index Blocks. 
  • Archive node: Data is moved from the Live node to the Archive node after 2 days unless configured otherwise. 
  • Restore node: Data from the Archives (stored as snapshots) is restored to the Restore node.

You can configure an Indexer node type during the process of installation. 

Recommendation

It is recommended that an Archive node should be configured in the following instances:

  • There is a high CPU utilization.
  • The search response is slow.
  • When archive is enabled.

For more information on how to configure an Indexer during the process of installation, see Installing TrueSight IT Data Analytics in a multiple server environment using the wizard.

 
About Index Blocks

Marking different data collectors in Index Blocks of same retention period and archive properties enables faster management of data purging as data will not have to be deleted at individual data collector levels, but as blocks of data. You can associate a data collector to one of the various Index Blocks, each having a configurable retention period by one of the following ways:

  • You can select an Index Block while creating a data collector from  the Advanced Options section of the Administration> Data Collectors­­­ page in the product. For more information, see Creating Data Collectors.
  • You can select an Index Block through Collection Profiles.

You can select an Index Block from one of the default Index Blocks defined in the product or create custom Index blocks. If you select an ITDA Metrics data pattern when you create a data collector, the Metrics Index Block is automatically associated with the data collector and you cannot choose from an out-of-the-box Index Block. 

When you select an Index Block, the properties of that Index Block are displayed below it. The properties that are displayed are:

  • Archive: This indicates whether the data that you index using the selected Index Block will be archived.
  • Retention Days: This indicates the retention days associated with the Index Block.

Select an Index Block as per your requirements of retention days for the data that is collected and whether you want to archive the collected data. The retention days and archive enable settings are configured by the Administrator. The following retention days are associated with the out-of-the-box Index Blocks by default. These numbers can vary for an upgraded setup.

For more information on modifying Index Block settings, see Changing-system-settings.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*