Troubleshooting log collection from files
No logs are ingested
Issue scope
This issue might occur because incorrect key or pattern are configured in the parsing or filtering rule. This issue might also occur if the format expression that you configured in the parsing rule does not match the the logs that you are collecting.
Resolution
- Correct the key and pattern configured in the parsing and filtering rules.
- Make sure that the format expression configured in the parsing rule matches the logs that you are collecting.
Logs are not being filtered while collection
Resolution
Check if key and pattern are configured correctly for Regexp and Exclude for the grep filter.
Configured integration is in a disconnected state
Resolution
Ensure that the docker container is up and running.
Verify parameters are correctly populated in the fluentd pipeline
Resolution
Verify that the configurations entered on the UI are saved successfully in the configuration file.
- Docker connector: Go to /opt/bmc/connectors/<connector_name>/<integration_name>/pipeline. Open the file_log_pipeline.conf file by running the cat file_log_pipeline.conf command.
- Windows connector: Go to <connector download folder>\opt\td-agent\file_log_pipeline.conf. Verify the configurations in this file.
config error .... error="Duplicate key: <key_Name>" in connector logs
Issue symptom
Integration is shown in the Configured state, but this error is shown in the connector td-agent logs.
Issue scope
Issue might occur with all connectors and integrations.
Resolution
In the integration configuration for log collection, when you configure the grep filter, ensure that for the same directive, keys are unique. For example, do not repeat the msgType key for multiple Regexp directives.
Then, verify that the configurations entered on the UI are saved successfully in the configuration file:
- Docker connector: Go to /opt/bmc/connectors/<connector_name>/<integration_name>/pipeline. Open the file_log_pipeline.conf file by running the cat file_log_pipeline.conf command.
- Windows connector: Go to <connector download folder>\opt\td-agent\file_log_pipeline.conf. Verify the configurations in this file.
- Linux connector: Open the file_log_pipeline.conf file located in the opt/td-agent/etc/data/<Integration_ID>/pipeline/ folder and modify the the file by removing the duplicate entry. Then, restart the td-agent service.
Unable to run the batch script to install the Windows connector
Resolution
Confirm that you have the permission to execute the script. Ensure that you launch the command prompt with the Run as Administrator option.
Unable to run the shell script to install the Linux connector
Resolution
Ensure that you have the root permissions to run a script file.
Batch script to install the Windows connector failed
Resolution
Check the logs for error. The installation script execution logs are displayed on the command prompt and are also added to the installation log file, which is located at <folder where you extracted the downloaded zip file>\BMC-DevTools\opt\<connector_Name> with name <connector_name>_install.log.
Path too long error while extracting the Windows connector .zip file
Resolution
You get this error when you use Windows out-of-the-box utility to extract a .zip file. Resolve the error by following either of the following:
- Ensure that the destination folder name is short.
- Use 7-zip or any other utility to extract the file.
System cannot find the specified path error while installing the Windows connector
Resolution
You get the BMC-DevTools folder after extracting the tdc-connector-windows-<build_number>.zip file. The BMC-DevTools folder and the .bat file to install the connector must be at the same level in the folder hierarchy. For example, if you extract the downloaded connector file in the C:\Connector folder, ensure that the downloaded .bat file is also present inside the Connector folder.
Not able to execute install script error while installing the Linux connector
Resolution
Ensure that you have extracted the .tar.gz file in the /opt directory.
/opt/td-agent already exist error while running the Linux connector install script
Resolution
On the Linux server, either the connector is already installed or a td-agent is currently running. Remove the existing installation and run install script again.
/opt/bmc-devtools/td-agent does not exist error while running the Linux connector install script
Resolution
Extract the installation .tar.gz file in the /opt directory only.
Permission denied error
When you are installing the Windows connector, you get the permission denied error even though you are installing as an administrator.
Issue symptom
Windows connector installation fails.
Issue scope
Issue might occur with Windows connector installation.
Resolution
Make sure that there are no spaces in the name of the folder where you are installing the connector.
Connector state is suspended
Issue symptom
The following error is shown in the fluentd.log/td-agent.log file (located at /opt/td-agent for Linux and <connector download folder>\BMC-DevTools\opt\td-agent for Windows):
config error file="/fluentd/etc/fluent.conf" error_class=Fluent::ConfigError error="format should be Regexp //
Issue scope
This issue might occur in file-based log collection if you update and enter incorrect regular expression in Apache, Apache Error, Nginx, Regexp, or Java Multiline formats in the parsing rule.
Resolution
- Open the log file:
- Windows connector: <installation-directory>\BMC-DevTools\opt\td-agent\etc\data\<integration_id>\pipeline\file_log_pipeline.conf
- Linux connector: /opt/td-agent/etc/data/<integration_id>/pipeline/file_log_pipeline.conf
- Search for the following line in file_log_pipeline.conf file:
@type <format>
For example, @type regexp - Make sure that the regular expression is enclosed in back slashes (/) and the syntax of the expression is correct:
expression /<regex_expression>/ - Restart the connector service.