Collecting syslog messages

System logging protocol (syslog) is an event-logging standard that enables devices and applications to send data about status, events, diagnostics, and so on. Syslog contains information about the operations, errors, and status of devices. Monitoring syslog messages helps you to keep track of your system and network events, detect security threats, and troubleshoot problems.

The following image shows how syslog messages are collected by  in a local setup:

The following image shows how syslog messages are collected by  in a remote setup:

The following video (2:19) provides you an overview of monitoring syslog messages in . 

Watch the YouTube video about overview of monitoring syslog messages in BMC Helix Log Analytics.

Before you begin

• Forward all syslog messages to a computer within your IP network from where  will collect them.
• Download and install a Linux (RHEL and CentOS) or Windows connector. For more information, see Installing-and-managing-connectors.
• Create a parsing rule of type Syslogs. For more information, see Creating-a-parsing-rule.
• Create a filtering rule. For more information, see Creating-a-filtering-rule.

To collect syslog messages

In , use the Collection > Collection Policies > Create button and perform the following steps:

1. Add the policy information by performing the following steps:
   1. Enter a unique name and description.
   2. From the Collection type list, select Syslog.
2. In the Connector configurations section, perform the following steps:
   1. From the Connector Type list, select Linux Connector (RHEL8), Linux Connector (CentOS 7.9), or Windows Connector.

   2. In Connector Selection Criteria, create the connector selection criteria to identify connectors for collection.
      When you click in the box, you are prompted to make a selection. Each time you make a selection, you are progressively prompted to make another selection. 
      The selection criteria consist of an opening parenthesis, followed by the slot name, the operator, the slot value (which can be a string based on the type of slot selected), and the closing parenthesis. You can optionally select the logical operator AND or OR to add additional conditions. Specifying the opening and closing parentheses is optional.
      The connector fields that are available to create the selection criteria are status, name, version, host_name, ip, and tags. 

      Important

      • The values that you enter for a field in the selection criteria are case-sensitive. For example, if the host name is WebServer.example.com, add the selection criteria as ( host_name Equals WebServer.example.com ). If you enter, ( host_name Equals webserver.example.com ), the connector is not selected. To add case-insensitive values, use the Equals ignore case operator.

      • When you ingest logs into BMC Helix Log Analytics, do not include the following metadata fields to prevent data corruption:
        • _index
        • _id 
        • _version 
        • _score 
        • _source 
        • _size

      Elasticsearch reserves the _id field for internal operations. If you include the _id field in logs while ingesting them, it can lead to failures, such as the search_after query failure. To prevent such failures, BMC Helix Log Analytics automatically renames the _id field in the incoming logs to bmc_modified_field_id field.

3. In the Configuration section, perform the following steps:
   1. Click Configure.
   2. In the Customize Log Data panel, from the Protocol list, select the protocol that you want to select to communicate with the Linux computer.
   3. Enter the bind address (the IP address of the computer where you have forwarded all syslog messages of your IP network) and port that the protocol will use to establish the connection.
   4. In the Message Length Limit field, enter the maximum length of the syslog messages that you are collecting (in bytes).
      The maximum supported length is 5120 bytes.
   5. Select the Ignore Unparsed Syslogs to ignore the collection of logs that are not parsed by the parsing rule.
   6. Click Save.
4. In the Parsing Rule step, select the parsing rule that you have created.
5. If you have not created a parsing rule, see Creating-a-parsing-rule for instructions to create a parsing rule of type Syslog.
6. From the Filtering Rule list, select the filtering rule that you have created.
   If you have not created a filtering rule, see Creating-a-filtering-rule for instructions.
7. From the User group list, select one or more user groups to assign to the collection policy. 
   Users associated with this user group can see the data collected by this collection policy.
8. Enable and save the policy by performing the following steps:
   1. To start collecting logs, select the Enable Collection Policy check box.
      You can choose to enable the collection policy later.
   2. Click Save.
      The created policy is shown on the Collection Policies page. Use the Actions menu to edit, enable (or disable), and delete the policy.

To verify the log collection

To verify whether the log collection has started, select Explorer > Discover and use the tags or time range to view the collected logs.

To view the the out-of-the-box Syslog dashboard in 

1. Click the Dashboards menu.
2. Select Manage Dashboards.
   The Log Analytics folder is displayed.
3. Click the Syslog dashboard.

Learn more

Read the following blog to learn how you can gain network visibility and performance by monitoring syslog messages: Gain Network Visibility and Performance with Syslog Monitoring.