Collecting Windows event logs

As an administrator, you want to collect Windows event logs and analyze them to troubleshoot issues. 

Use the open source td-agent to collect the Windows event logs. Here are the high-level steps that you need to perform:

To get the API key and tenant URL

1. In the BMC Helix Operations Management UI, select Administration > Repository.
2. Click Copy API Key and paste it in a notepad file.
3. Copy the URL of your BMC Helix Log Analytics tenant and paste it in the notepad file.
   For example: https://tenanttrial.bmc.com.

To download and install td-agent

Download and install the open-source td-agent (Windows-based Fluentd). For more information, see td-agent MSI packages.

After the td-agent is installed, it runs as the Fluentd Windows Service.

To configure the td-agent.conf file

Add the source plugin of type windows_eventlog2 to the td-agent.conf file that helps to collect the following Windows event logs:

• Application
• System
• Setup
• Security

In the channel parameter, you can define which type of logs you want to collect. If you do not specify a value in the channel parameter, the application logs are collected by default.

1. Go to the <td-agent installation folder>\opt\td-agent\etc\td-agent folder.
2. Open the td-agent.conf file.

3. Add the source plugin of type: windows_eventlog2.
   For more information, see Github windows_eventlog2.

   Sample source plugin of type windows_eventlog2

   <source>
     @type windows_eventlog2
     @id windows_eventlog2
     channels application,system # Also be able to use `<subscribe>` directive.
     read_existing_events false
     read_interval 10
     tag WinEventLog_logingest
     render_as_xml false       # default is false.
     rate_limit 200            # default is -1(Winevt::EventLog::Subscribe::RATE_INFINITE).
     # preserve_qualifiers_on_hash true # default is false.
     # read_all_channels false # default is false.
     # description_locale en_US # default is nil. It means that system locale is used for obtaining description.
     # refresh_subscription_interval 10m # default is nil. It specifies refresh interval for channel subscriptions.
     # event_query "Event/System[EventID!=1001]" # default is "*".
     <storage>
       @type local             # @type local is the default.
       persistent true         # default is true. Set to false to use in-memory storage.
       path C:/tdagent/BMC-DevTools/winevt1.pos
     </storage>
     # <parse> # Note: parsing is only available when render_as_xml true
     #  @type winevt_xml # @type winevt_xml is the default. winevt_xml and none parsers are supported for now.
       # When set up it as true, this plugin preserves "Qualifiers" and "EventID" keys.
       # When set up it as false, this plugin calculates actual "EventID" from "Qualifiers" and removing "Qualifiers".
       # With the following equation:
       # (EventID & 0xffff) | (Qualifiers & 0xffff) << 16
       # preserve_qualifiers true # preserve_qualifiers_on_hash can be used as a setting outside <parse> if render_as_xml is false
     # </parse>
     # <subscribe>
     #   channels application, system
     #   read_existing_events false # read_existing_events should be applied each of subscribe directive(s)
     #   remote_server 127.0.0.1 # Remote server ip/fqdn
     #   remote_domain WORKGROUP # Domain name
     #   remote_username fluentd # Remoting access account name
     #   remote_password changeme! # Remoting access account password
     # </subscribe>
     parse_description true
   </source>

4. (Optional) Add the filter plugin.
   Use this plugin to enrich logs with meaningful information. For example, add the hostname to logs that will help you in filtering logs in the Explorer tab.

   Sample filter plugin

   <filter WinEventLog_logingest>
     @type record_transformer
     <record>
       hostname ${hostname}
     </record>
   </filter>

5. To send the collected logs to BMC Helix Log Analytics, add the match plugin.
   You will need the API key and URL of your BMC Helix Log Analytics tenant.

   Sample match plugin

   <match WinEventLog_logingest>
       @type http
       @id OOB_TD-agent-WinEventLog_logingest
       endpoint https://<tenant_URL>/log-service/api/v1.0/logs
       headers {"Authorization":"apiKey <apiKeyoftenant>","Content-Type":"application/json"}
       json_array true
       <format>
         @type json
       </format>
   </match>

6. Save the file.
   To troubleshoot issues with the td-agent configuration, refer to the td-agent.log file in the C:\opt\td-agent\td-agent.log folder.

To restart the Fluentd Windows Service

1. In Windows, go to Services.
2. Right-click Fluentd Windows Service.
   
3. Select Restart.

To view the collected logs

1. In BMC Helix Log Analytics, click the Explorer tab.
2. Search the Windows event logs by searching for a key:value pair that is present in the Windows event logs.
   For example, search the logs for the hostname that you added to the filter plugin.