Collecting application logs

To start collecting logs, you add all the collection-related configurations and other details to a collection policy. You save time by reusing these configurations in multiple collection policies.

The following image shows how logs are collected by using the Linux (RHEL/CentOS) and Windows connectors:

Before you begin

• Make sure that you have downloaded and installed a connector. For more information, see Installing-and-managing-connectors.
• Create a parsing rule. For more information, see Creating-a-parsing-rule.
• Create a filtering rule. For more information, see Creating-a-filtering-rule.

To collect application logs

1. Click the Collection menu and select Collection Policies.
2. On the Collection Policies page, click Create.
3. Enter a unique name and description.
4. From the Connector type list, select the connector type that you have installed.

5. In Connector Selection Criteria, create the connector selection criteria to identify connectors for collection.
   When you click in the box, you are prompted to make a selection. Each time you make a selection, you are progressively prompted to make another selection. 
   The selection criteria consist of an opening parenthesis, followed by the slot name, the operator, the slot value (which can be a string based on the type of slot selected), and the closing parenthesis. You can optionally select the logical operator AND or OR to add additional conditions. Specifying the opening and closing parentheses is optional.
   The connector fields available to create the selection criteria are status, name, version, host_name, ip, and tags. 

   Important

   The values that you enter for a field in the selection criteria are case-sensitive. For example, if the host name is WebServer.example.com, add the selection criteria as ( host_name Equals WebServer.example.com ). If you enter ( host_name Equals webserver.example.com ), the connector is not selected. To add case-insensitive values, use the Equals ignore case operator.

6. In the Configuration step, click Configure.

7. In the Log Collection File Path field, enter the path of the log files that you want to collect.
   For example: /opt/tomcat/apache.log (Linux) or C:/app1/logs/app.log (Windows).
   When you enter folder locations, sub-folders and files present in the folder are shown. 

   Best practices

   • Enter only directory paths and an absolute file name with the path.
   • Separate multiple entries with a comma.
   • Make sure that all log files have the same format so that a single parsing rule parses all the logs.
   • To collect logs from Windows-based applications, ensure that you enter the path of the computer where you have installed the connector.
   • In Windows file path, replace back slashes (\) with forward slashes (/). For example, if your file path is C:\app1\logs\app.log, change it to C:/app1/logs/app.log.  
   • If your log files are created on the basis of size, enter the name of the file where the latest logs are written and do not enter * in the file path.
8. (Optional) If you have entered a path with multiple folders and you want to exclude some folders from collection, in the Exclude Paths field, remove those folders.
   For example, you have entered the log collection path as /opt/bmc/connectors/<connector_name>/logs/applicationLogs and this folder contains the following folders: app1, app2, app3. The app1, app2, and app3 folders are shown in the Exclude Paths field. To prevent log collection from the app3 folder, remove the app3 folder from the field.
9. To start log collection when the policy is enabled, clear the Read Files from Beginning check box.
   By default, all logs present in a log file, including older logs, are collected. If your log files are created (and rotated) on the basis of time, you have given wildcard (*) in the file path, and the Read Files from Beginning check box is cleared, all logs created after the policy is enabled are collected.
10. Click Save.
11. In the Tags field, enter the tags to identify the policy with the collected logs.
    The values that you enter in this field are added to the bmc_tags field that is present in the collected logs. You can use the field or tags to search and analyze logs in Explorer.
12. In Fields, enter the custom information that you want to add to collected logs in the form of key-value pairs.
    Use these fields to search and analyze the logs in Explorer. For example, Key: applicationContext; Value: Apache. Use applicationContext:Apache as a search string to search and analyze the collected logs.
13. In the Parsing Rule step, select the parsing rule that you have created.
    If you have not created a parsing rule, see Creating-a-parsing-rule for instructions.
14. From the Filtering Rule list, select the filtering rule that you have created.
    If you have not created a filtering rule, see Creating-a-filtering-rule for instructions.
15. From the User group list, select one or more user groups to assign to the collection policy. 
    Users associated with this user group can see the data collected by this collection policy.
16. To start collecting logs, select the Enable Collection Policy check box.
17. Click Save.
    The created policy is shown on the Collection Policies page. Use the Actions menu to edit, enable (or disable), and delete the policy.

To verify log collection

1. Click the Explorer tab.
2. Search the logs based on a unique field value.
   For example, you can use the tags that you have added to the collection policy. Let's say you added the tag apache_logs. Search for logs by using bmc_tags:apache_logs or applicationContext:Apache.

Where to go from here

Configuring-logs

Generating-alerts-from-logs

Deriving-insights-from-logs