Creating data patterns manually
This topic provides information about creating a data pattern manually.
The following video helps you understand when and how to create a data pattern by using the clone feature.
The following video helps you create a data pattern with a custom date format and corresponding custom subpattern.
This topic contains the following information:
Related topics
Before you begin
- Ensure that you have knowledge of Java regular expressions for the purpose of adding data patterns.
- Read the Notes related to data pattern creation.
- Use the following topics as a guide for adding a new data pattern:
- Examples-of-creating-a-data-pattern for an end-to-end use case
- Sample-data-patterns for sample data patterns
- Sample subpatterns
- Sample-date-formats
Creating a new data pattern manually
To create a data pattern manually, navigate to the Administration > Data Patterns tab, click Add Data Pattern 
, provide the following information, and click Create.
Notes about creating data patterns
The following notes are important to keep in mind while adding a new data pattern and will help you understand the impact on the search capabilities:
| Action | Description |
|---|---|---|
1 | Creating a custom date format | If you create a custom date format, then you must create a corresponding subpattern and use it in the primary pattern that you are constructing. Impact: Without this, you cannot collect data using the particular data pattern. |
2 | Using internal fields | The following fields are internal fields and might not be available for previewing to validate the sample data entries.
Impact: These fields are not searchable. |
3 | Using more than one subpattern for defining the timestamp field | While constructing a primary pattern, you cannot assign more than one subpattern for extracting the timestamp (field). Instead of using more than one subpattern in the primary pattern, you can create a more complex subpattern that provides the unified value that you were trying to achieve with multiple subpatterns. Impact: A data pattern containing such a primary pattern is invalid and is not usable for data-collection purposes. |
Example of an invalid primary pattern | ||
%{Data:_ignore}\s* | ||
Example of a valid pattern example | ||
Primary pattern: %{Mytimestamp:timestamp} \[%{Data:debuglevel}\] | ||
Supporting subpattern: Mytimestamp: %{DigitDay:day}\s+%{Month:month}\s+ | ||
4 | Using the details field for categorizing miscellaneous information in your data file. | You can assign the details field for miscellaneous information that you do not want to categorize with a specific field. All name=value pairs in the section to which this field is applied are extracted as fields. Impact: At the time of indexing, the details field is ignored. If you do not specify the details field in your primary pattern, then the product looks for name=value pairs in the entire raw data record and extracts them as fields. |
5 | Using the _ignore field for ignoring certain portions of data in your data file | You can assign the _ignore field to the the portion of your data that you want to ignore and not categorize with a specific field. For example, if you want to ignore the extra digits (the milliseconds) in the custom date and timestamp 2014 Thu May 14 05:25:14.12321, you can assign this field to the extra digits. In this case, you can use the following subpattern to ignore the last two digits: %{extraDigits:_ignore} where, extraDigits = \d{2} Impact: The portion of data to which this field is applied is not categorized with a field. |
6 | Using the letter X while creating a custom date format. | For a custom date format, the letter X that indicates the ISO 8601 time zone is not supported. To enable you to capture the time zone, when you create a data collector, select an option in the Time Zone field. Impact: You cannot collect data. |