Deriving insights from logs

Get to the root cause of an issue by using out-of-the-box options such as queries, time range, fields, and so on. For example, you observed multiple log entries with 401 status in your Apache logs. A few of such entries might mean that users forgot their passwords. However, multiple entries from same IP address in a short time span might mean a security threat. Use the search options and narrow down the results to find the root cause. For quick references, add the filtered logs to dashboards.

The following video (2:53) illustrates how to analyze and visualize logs:


https://youtu.be/fggAxALVs0w

Search and analyze logs on the Explorer > Discover tab. The following figure highlights the important features of the page that help you in getting to the root cause of an issue.


Let's explore these features in detail. 

Index patterns overview

By default, an index pattern is created for your tennant. All the logs are collected under this index pattern. You can neither delete this index pattern nor create a new one.

Using the Search field


Adding filters

Filters will help you further refine the search results. Add filters on the basis of field values. For example, filter all log records for which @timestamp field is "xxxxxxxxx".

<image>

Using the time range and dates

You get the following choices to set the date to narrow down your search results:

Specify days or hours since when you want to search results. For example, search results for last 15 minutes or last 7 days.

<image>

Set specific date and time (absolute or specific). For example, search results for Jul 18, 2022 18:00 hours till Jul 19, 2022 18:00 hours.

<image>

Using selected and available fields to filter logs


Deriving meaning from the search results


Saving the search


Adding the saved search to a visualization


As an administrator or operator, analyze logs and keep your systems (or environments) healthy:

ActionDetailsSteps
Search logs

Narrow down the search results by using a combination of queries, time range, and fields. For example, by searching logs you found that the 401 status code was returned only a few times in a particular time period, say an hour.

The following image shows the options that will help you search logs and narrow down results to the actual issue:

  1. To view the collected logs, go to Log Explorer > Discover.
  2. To filter data, select the required fields from the Available fields list.

    Tip

    In place of the data type icon of a field, if you see the '?' sign, refresh the index on the index pattern page (Stack Management > Index pattern > index pattern name).

  3. Select a date range to filter data.
  4. (Optional) You can also use Kibana queries to filter data. For more information, see  Kibana documentation .
Save search

Save the search that you might need in the future. Share the saved searches with other users in your organization.

To save the filter criteria for later use as a saved search, click Save.

Add visualizations and dashboards

View log data in the form of charts and other graphical representations (tables, region map, heat map, and so on) by creating visualizations and adding them to dashboards. You can monitor log entries through these dashboards. Filter data by using the filtering options, such as query and time range. For example, add a visualization for all 401 status codes where count is more than 5 in one hour to a dashboard.

  1. Click Visualize > Create new visualization.
  2. Select the type of visualization that you want to use.
    For example, a line chart.
  3. Select the search that you have saved.
  4. Apply additional filters to the data and save the visualization.
  5. To add the visualization to a dashboard:
    1. Click Dashboard.
    2. You can create a new dashboard or edit an existing one.
    3. Click Add and select the visualization.
Was this page helpful? Yes No Submitting... Thank you

Comments