Page tree

Fields represent small portions of your data displayed as name=value pairings, such as Source=<host-name>.

Fields add meaning to the data and help you search more effectively. They help you classify particular portions in your data that might otherwise go unnoticed. Fields act as the building blocks for running search commands and creating dashboards.

Related topics

Fields can be identified in the following ways:

  • Automatically identified by the product
  • Defined by a user while creating a data pattern
  • Defined by a user at search-time (virtual fields)
  • Defined in the data patterns included in the content packs that you are using

The product automatically identifies portions of the data that appear in the name=value format and displays them as fields. To effectively filter the indexed data and to perform advanced search operations (by using search commands), it is recommended that you define fields. Additionally, if you plan to analyze certain portions of your data, then it is recommended that you identify those portions and define fields for those portions of the data. You can define fields, at the time of creating a data pattern or at search-time by using search commands.

The following information provides an overview about how fields can be extracted and used:

About field extraction

When you assign a data pattern to a data collector, the tokens used in the data pattern are also extracted as fields and are searchable as name=value pairs. If the data pattern contains the details token, the product looks for the equals sign (=) to use as a delimiter to extract the name=value pairs.


If you want to capture and extract particular portions in your data that are important to your needs, but which are not automatically discovered and extracted, then you need to define fields for such portions. You can define fields by performing a data pattern based field extraction while creating a data pattern or you can perform a search-time field extraction by using particular search commands.

BMC recommends you to plan your fields beforehand and perform a data pattern based field extraction.

Before you begin to extract fields by creating a data pattern, it is important that you learn how to identify fields that might be useful for performing effective searches.

Begin with analyzing your data file to see if the file follows any patterns that can be captured by means of fields. After finding the pattern, you need to identify if this pattern can be divided into small logical portions and grouped. For every group identified, you can create a field. This ensures that all information that can be categorized into a group is indexed and is available for search.

Suppose you want to extract fields from the following data:

Apr 24, 2014 03:16:40 PM configservice WARN: No configuration found.

Apr 24, 2014 03:16:44 PM dbservice INFO: Starting Schema Apr 24, 2014 03:16:44 PM
dbservice INFO: CONFIGURATIONS table exists in the database.

Apr 24, 2014 03:16:44 PM dbservice INFO: Executing Query to check init property:
select * from CONFIGURATIONS where userName = 'admin' and propertyName ='init'
Apr 24, 2014 03:16:44 PM dbservice INFO: init property exists in CONFIGURATIONS table.

In the preceding lines, every new line starts with the time stamp. And you will notice that the file follows a consistent pattern.

The following information (groups) appears in the preceding lines from left to right:

  1. Time stamp
  2. Component name
  3. Debug information
  4. Application message

For each of the preceding groups, you can assign a field.


If your data contains miscellaneous details that cannot be categorized under a specific field, then you can assign the details field for such miscellaneous details. At the time of indexing, the details field is ignored, but all name=value pairs appearing in the section to which this field is applied are automatically extracted as fields.

Additional information

In the preceding sample lines, suppose you think that extracting the word "CONFIGURATIONS" as a field might be useful. But this word does not appear consistently on each line.

If you decide to extract this portion as a field, the product might skip the other lines that do not contain the word, in which case those lines will not be available for searching. In this scenario, you can use one of the following options:

  • Extract this field at search-time by using search commands such as extract.
  • Extract this field while creating the data pattern. And then while creating the data collector, ensure that you enable the Best Effort Collection setting under the advanced options.

Searching with fields

Fields are displayed on the Search page, under the Filters panel, and in the Fields section; fields with multiple values are displayed as a comma-separated list. For every field, a count of occurrences is displayed in parentheses () next to the field name. If the number of occurrences is too large, an approximate count is displayed with a plus sign (+). When you expand such a field, the values show the approximate count with an asterisk (*) next to them. The plus sign next to the field name and the asterisk next to the field values indicate that the count for those fields or values is an approximate number, not an exact number. If you select one of the field values to add it to the search criteria and click Search, the accurate count is displayed next to that value.

You can delete all fields that you added manually to the Fields section under the Filters panel, except for the defaults.

In addition to the default fields, you can specify other custom fields to display under the Fields section. You can use these fields in your search query for narrowing down results. For more information, see Filtering your search results.

When you search for name=value pairs, note that the name is limited to the following characters:

  • Letters (irrespective of case)
  • Numbers (0 to 9)
  • Underscore (_)
  • Hyphen (-)
  • Period (.)