Adding a new data pattern
This topic provides instructions for adding a data pattern.
For more information about the prerequisites, procedure for adding a data pattern, and notes to keep in mind, see:
Before you begin
- Ensure that you have knowledge of Java regular expressions for the purpose of adding data patterns.
- Read the Notes related to data pattern creation.
- Use the following topics as a guide for adding a new data pattern:
- Examples-of-creating-a-data-pattern for an end-to-end use case
- Sample-data-patterns for sample data patterns
- Sample subpatterns
- Sample-date-formats
Adding a new data pattern
To add a data pattern, navigate to the Administration > Data Patterns tab, click 
Add Data Pattern, provide the following information, and click Create.
Notes about creating data patterns
The following notes are important to keep in mind while adding a new data pattern and will help you understand the impact on the search capabilities:
| Action | Description |
|---|---|---|
1 | Creating a custom date format | If you create a custom date format, then you must create a corresponding subpattern and use it in the primary pattern that you are constructing. ImpactWithout this, you cannot collect data using the particular data pattern. |
2 | Using internal fields | The following fields are internal fields and might not be available for previewing to validate the sample data entries.
ImpactThese fields are not searchable. |
3 | Using more than one subpattern for defining the time stamp field | While constructing a primary pattern, you cannot assign more than one subpattern for extracting the time stamp (field). Instead of using more than one subpattern in the primary pattern, you can create a more complex subpattern that provides the unified value that you were trying to achieve with multiple subpatterns. ImpactA data pattern containing such a primary pattern is invalid and is not usable for data-collection purposes. |
Example of an invalid primary pattern | ||
%{Data:_ignore}\s* | ||
Example of a valid pattern example | ||
Primary pattern: %{Mytimestamp:timestamp} \[%{Data:debuglevel}\] | ||
Supporting subpattern: Mytimestamp: %{DigitDay:day}\s+%{Month:month}\s+ | ||
4 | Using the details field for categorizing miscellaneous information in your data file. | You can assign the details field for miscellaneous information that you do not want to categorize with a specific field. All name=value pairs in the section to which this field is applied are extracted as fields. ImpactAt the time of indexing, the details field is ignored. If you do not specify the details field in your primary pattern, then the product looks for name=value pairs in the entire raw data record and extracts them as fields. |
5 | Using the _ignore field for ignoring certain portions of data in your data file | You can assign the _ignore field to the the portion of your data that you want to ignore and not categorize with a specific field. For example, if you want to ignore the extra digits (the milliseconds) in the custom date and time stamp 2014 Thu May 14 05:25:14.12321, you can assign this field to the extra digits. In this case, you can use the following subpattern to ignore the last two digits: %{extraDigits:_ignore} where, extraDigits = \d{2} ImpactThe portion of data to which this field is applied is not categorized with a field. |
6 | Using the letter X while creating a custom date format. | For a custom date format, the letter X that indicates the ISO 8601 time zone is not supported. To enable you to capture the time zone, when you create a data collector, select an option in the Time Zone field. ImpactYou cannot collect data. |
Related topic