Filtering logs

After the logs are parsed, you can filter the logs to include relevant log data and exclude data that you do not require. For example, you set up the following grep configurations

Sample logs:
The value of the message field contains cool.
The value of the hostname field matches web<INTEGER>.example.com.
The value of the message field does NOT contain uncool.
The following logs are collected:
{"message":"It's cool outside today", "hostname":"web001.example.com"}
{"message":"That's not cool", "hostname":"web1337.example.com"}
The following logs are excluded:
{"message":"I am cool but you are uncool", "hostname":"db001.example.com"}
{"hostname":"web001.example.com"}
{"message":"It's cool outside today"}

To filter logs

1. From the Log Filter list, select Grep.
2. From the Directive field, select Regex (to include logs) or Exclude.

3. In the Key field, enter the key from the log expression.
   Get the keys from the log expression. For example, in the Apache expression, host, user, time, method, path, code, size, refer, and agent are keys.
   For CSV type of parser, keys are the field names that you entered for the columns in the CSV file.

   Important

   Do not configure same key more than one time for same directive. 

4. In the Pattern field, enter the value to be included or excluded.
   For Apache, Apache Error, Nginx, and Regexp, enclose the values within forward slashes (//).

5. Click + to add another grep expression.

   Examples

   Example for Apache, Apache Error, Nginx, and Regexp

   
   Sample logs:
   The value of the message field contains cool.
   The value of the hostname field matches web<INTEGER>.example.com.
   The value of the message field does NOT contain uncool.
   The following logs are collected:
   {"message":"It's cool outside today", "hostname":"web001.example.com"}
   {"message":"That's not cool", "hostname":"web1337.example.com"}
   The following logs are excluded:
   {"message":"I am cool but you are uncool", "hostname":"db001.example.com"}
   {"hostname":"web001.example.com"}
   {"message":"It's cool outside today"}

   Example for Java multiline

   
   Sample logs:
   The value of the message field contains cool.
   The value of the message field does NOT contain uncool.
   The following logs are collected:
   {"message":"It's cool outside today"}
   The following logs are excluded:
   {"message":"I am cool but you are uncool"}
   

   Example for Json

   
   Sample log:
   {"time":1362020400,"host":"111.111.0.1","size":777,"method":"PUT"}
   {"time":1362020400,"host":"111.111.0.1","size":777,"method":"POST"}
   {"time":1362020400,"host":"111.111.0.1","size":777,"method":"GET"}
   
   The following logs are collected:
   {"time":1362020400,"host":"111.111.0.1","size":777,"method":"GET"}
   The following logs are excluded:
   {"time":1362020400,"host":"111.111.0.1","size":777,"method":"PUT"}
   {"time":1362020400,"host":"111.111.0.1","size":777,"method":"POST"}

   Example for CSV

   
   Sample CSV format:

   2013/02/28 12:00:00,111.111.0.1,111,user1
   2013/02/28 12:00:00,111.111.0.1,111,user2
   2013/02/28 12:00:00,111.111.0.1,111,user3

   The following logs are collected:
   2013/02/28 12:00:00,111.111.0.1,111,user2
   2013/02/28 12:00:00,111.111.0.1,111,user3
   The following logs are excluded:
   2013/02/28 12:00:00,111.111.0.1,111,user1