Understanding fields

Fields can be identified in the following ways:

• Automatically identified by the product
• Defined by a user while creating a data pattern
• Defined by a user at search-time (virtual fields)
• Defined in the data patterns included in the content packs that you are using
  

The product automatically identifies portions of the data that appear in the name=value format and displays them as fields. To effectively filter the indexed data and to perform advanced search operations (by using search commands), it is recommended that you define fields. Additionally, if you plan to analyze certain portions of your data, then it is recommended that you identify those portions and define fields for those portions of the data. You can define fields, at the time of creating a data pattern or at search-time by using search commands.

The following information provides an overview about how fields can be extracted and used:

About field extraction

At the time of data indexing, fields are automatically extracted. This process is known as field extraction.

Fields can be extracted in the following ways:

• Field extraction performed during the time of data collection and indexing.
• Field extraction performed during search time.

During data collection, the product automatically discovers and extracts name=value pairs from the data and displays it as fields in your search results. In addition to this, for every data entry that is indexed, the product assigns certain fields based on the inputs specified at the the time of creating a data collector or by certain default settings. These fields are treated as default fields and are displayed under the Filters panel > Fields section, on the Search page (after you perform a search).

When you assign a data pattern to a data collector, the tokens used in the data pattern are also extracted as fields and are searchable as name=value pairs. If the data pattern contains the details token, the product looks for the equals sign (=) to use as a delimiter to extract the name=value pairs.

Fields can also be extracted during search-time by using search commands such as extract and extractkv. Fields extracted during search-time are virtual fields that cannot be added to the Filters panel on the Search page.

For more information about the various ways in which fields are extracted, see About field extraction.

Learning how to identify fields in the data

Before you begin to extract fields by creating a data pattern, it is important that you learn how to identify fields that might be useful for performing effective searches.

Begin with analyzing your data file to see if the file follows any patterns that can be captured by means of fields. After finding the pattern, you need to identify if this pattern can be divided into small logical portions and grouped. For every group identified, you can create a field. This ensures that all information that can be categorized into a group is indexed and is available for search.

Suppose you want to extract fields from the following data:

In the preceding lines, every new line starts with the time stamp. And you will notice that the file follows a consistent pattern.

The following information (groups) appears in the preceding lines from left to right:

1. Time stamp
2. Component name
3. Debug information
4. Application message

For each of the preceding groups, you can assign a field.

Searching with fields

Fields are displayed on the Search page, under the Filters panel, and in the Fields section; fields with multiple values are displayed as a comma-separated list. For every field, a count of occurrences is displayed in parentheses () next to the field name. If the number of occurrences is too large, an approximate count is displayed with a plus sign (+). When you expand such a field, the values show the approximate count with an asterisk (*) next to them. The plus sign next to the field name and the asterisk next to the field values indicate that the count for those fields or values is an approximate number, not an exact number. If you select one of the field values to add it to the search criteria and click Search, the accurate count is displayed next to that value.

You can delete all fields that you added manually to the Fields section under the Filters panel, except for the defaults.

In addition to the default fields, you can specify other custom fields to display under the Fields section. You can use these fields in your search query for narrowing down results. For more information, see Filtering your search results.

When you search for name=value pairs, note that the name is limited to the following characters:

• Letters (irrespective of case)
• Numbers (0 to 9)
• Underscore (_)
• Hyphen (-)
• Period (.)