Collecting data into the system
The easiest way of collecting data is to create a data collector from the Administration > Data Collectors tab. Data collectors collect data and send it to the Indexer for indexing. When you perform a search, the indexed data is made available as a series of individual records (or search results).
When creating a data collector, you need to provide various inputs such as details of the target host from where you plan to collect data, and other parameters such as the data pattern (matching the data that you want to collect), the date format to use for indexing the date and time string, the file encoding to use, and other advanced (but optional) settings. Before you create a data collector, you need to understand the kind of data that you want to collect and collate all the inputs required for creating the particular data collector type.
The following information can help you understand the data collector creation process and associated best practices.
Preparing for data collector creation
As a best practice, before you start creating data collectors, you must create a deployment plan. If you plan to configure a large number of data collectors for multiple applications, you must plan for your needs from various aspects to help you save time and make your data-collection process as efficient as possible. You can start by creating a list of sources of all the data that you want to collect and then fill in key properties about each data source. For example, the contains a list of sample inputs required while creating data collectors.
Icons and associated functions on the Data Collectors tab
The Data Collectors tab allows you to manage data collectors. To access this tab, navigate to Administration > Data Collectors.
The Data Collectors tab displays a default data collector that collects data available in the Collection_metrics.log file. The Dashboards tab displays a graph summarizing the data collected by this data collector. For more information, see Creating-and-managing-dashboards. For more information about collecting data from the product metrics files, see Monitoring-the-product-metric-files.
You can perform the following actions on the Data Collectors tab.
The Data Collectors tab provides the following information:
Best practices for using common data collector settings
The following table lists the best practices that you can apply while using the common data collector settings (at the time of creating a data collector).
Scenarios when data is not collected
The following table provides scenarios in which data might not be collected and therefore is not searchable.
Scenario | Description |
|---|---|
When a data collector is stopped and then started. | The time for which the data collector remains down. |
When a new Collection Station is added to the pool. | This involves restart of all the Collection Agents. Data is not be collected for the time taken by the Collection Agents to restart. |
When the configuremasters-CLI-command is run. | This involves the restart of the Collection Station. Data is not be collected for the time taken by the Collection Station to restart. |
When the movecomponents-CLI-command is run. | This involves the restart of all the Collection Agents. Data is not be collected for the time taken by the Collection Agents to restart. |
When the data collector is started, past data is not collected. | This is applicable for all data collectors except Monitor using External Configuration and Monitor Remote Windows Events data collectors. |
If the Payload channel is full. | This can occur due to a number of reasons. For example, if the Collection Station is not reachable or there is a sudden burst in incoming data. |