Skip to Content

Troubleshooting PATROL for Linux KM

This section provides you information to troubleshoot the KM:

Error messages

This section describes some common error messages that you might encounter while running PATROL for Linux, and provide suggested resolutions for the errors. The messages in the KM use a prefix to identify which application or collector sent the message. Find the prefix you are interested in for a list of the messages associated with that application or collector.

Filesystems-related issues

Issue

Solution

Filesystem parameters still collect data even if the monitored filesystem has been unmounted from server 

Any of the following solution might resolve your issue:

  1. Check the statvfs permission as statvfs system API is used for filesystem data collection. The statvfs binary must be owned by root user and configure setuid by running the following commands:
    cd $PATROL_HOME/../unix/Linux-2-6-x86-64-nptl/bin
    ls -l statvfs
    -rwsr-xr-x 1 root patrol 9568 May 9 2011 statvfs
  2. This API does not have the capability of identifying if the provided path is a normal directory or there is any filesystem actually mounted on that path. If the any filesystem gets unmounted, the API returns the details for directory on which the filesystem was mounted. For example, if /sftp was earlier mounted on /home/sftp, after unmounting the filesystem, the API would return data for /home/sftp.
  3. PATROL Agent uses the "mount -v" command to get the information of mounted filesystems and update the monilist Ruleset.
    1. Delete the "monilist" configuration variable from PATROL Agent configuration value by:
    2. Go to PATROL Console and open "System Output Window"(SOW).
    3. In the prompt that opens, tpe the following command:
      OS> %PSL pconfig ("DELETE","/NUK/NUK_FileSystem_Container/moniList");
    4. Restart the PATROL Agent.
      This PSL command deletes the existing filesystem and allows PATROL Agent to discover the existing filesystems. It also helps to stop events that are generating on the FSMountStatus attribute. If you do not have PATROL Console, apply the following configuration variable and restart the PATROL Agent: "/NUK/NUK_FileSystem_Container/moniList" = { DELETE = "" }  

If the issue is not resolved, contact BMC Support with the following logs:

  • Patrol Agent configuration
  • Mount command output

Filesystem instance label name is incorrect 

If the FILESYSTEM instance label consists of more than 20 characters, the display name of the instance label is truncated by default.

The /NUK/NUK_FileSystem_Container/dispFullName pconfig variable enables you to display the complete FILESYSTEM instance label. You can assign the following values to this variable:

  • 0: Displays the truncated FILESYSTEM instance label if the characters are more than 20.
  • 1: Displays the complete FILESYSTEM instance label.

Example

/NUK/NUK_FileSystem_Container/dispFullName" = { REPLACE = "1" }

 If the issue is not resolved, contact BMC Support with the following logs:

  • Patrol Agent configuration
  • Mount command output

Filesystem exclusion does not work or how to disable alerts for unmounted filesystem

Check the filesystems type.

For example, the /run/user/* filesystems have type tmpfs.

To know the filesystem type, run the following command: mount | grep <filesystemName> (example - mount | grep /run/user/*)

When these filesystems get unmounted, mount command does not show entries for such file systems . Therefore, "type" of such instances is not available.

Check if the below rules are present. If these rules are not present, the exclusion is not successful:

"/ConfigData/NUK_FileSystem_Container/customFilterEnabled" = { REPLACE = "1" }, 

"/ConfigData/NUK_FileSystem_Container/customFsType" = { REPLACE= "tmpfs" },

To fix, see Configuring-FileSystems and restart PATROL Agent.

If the issue is not resolved, contact BMC Support with the following logs:

  • PATROL Agent configuration
  • Mount command output

How to enable default monitoring of all filesystems using Linux KM?

How to include specific file system monitoring from TrueSight Policy?

By default, only the following filesystems are monitored:

  • ^/$ (root)
  • ^/tmp$ (tmp)
  • ^/usr$ (usr)
  • ^/home$ (home)

To monitor all filesystems, add the following regular expression to the Include field “.*” while configuring the Filesystems monitor type. For more information, see Configuring-FileSystems.

Including all filesystems for monitoring puts unnecessary load on PATROL Agent, but you can add regular expressions to include filesystems to monitor.

You can include or exclude a filesystem from monitoring by adding comma-separated regular expressions in the Include and Exclude fields. For example, ^/scripts, ^/mnt, ^/local/utils.

 If the issue is not resolved, contact BMC Support with the following logs:

  • PATROL Agent configuration
  • Mount command output

Filesystem is present in df -h but is not monitored

Perform the following actions:

  1. Delete the content of the following pconfig variable - /NUK/NUK_FileSystem_Container/moniList
  2. Add the following to the pconfig variable - /NUK/NUK_FileSystem_Container/moniList= {REPLACE = “ “ }
  3. Reload the variable.
  4. Restart PATROL Agent.

How to exclude a specific filesystem monitoring from TrueSight policy?

If you are trying to exclude /run/user/* filesystems and still getting events, note that /run/user/* filesystems have type "tmpfs". When these filesystems get unmounted, mount command does not show entries for such file systems hence "type" of such instances is not available on server. As "type" is no more available, the following rules would not work:

 "/ConfigData/NUK_FileSystem_Container/customFilterEnabled" = { REPLACE = "1" }, 

"/ConfigData/NUK_FileSystem_Container/customFsType" = { REPLACE= "tmpfs" },

To fix this scenario, perform the following actions while configuring the FileSystems monitor type. For more information, see Configuring-FileSystems.

  1. In the Exclude by type section, select Custom and add tmpfs to the Custom type list field.
  2. In the Unmounted filesystems handling > Remove by type field, select Custom types and add tmpfs to the Custom types list text box.
  3. Restart the PATROL Agent. 

If the issue is not resolved, contact BMC Support with the following log: File system Debug (to check if the filesystem is discovered by the agent and KM).

How to enable default monitoring of all filesystems using Linux KM?

How to include specific file system monitoring from TrueSight Policy? 

By default, only the following filesystems are monitored:

  • ^/$ (root)
  • ^/tmp$ (tmp)
  • ^/usr$ (usr)
  • ^/home$ (home)

 To monitor all filesystems, add the following regular expression to the Include field “.*” while configuring the Filesystems monitor type. For more information, see Configuring-FileSystems.

Including all filesystems for monitoring puts unnecessary load on PATROL Agent, but you can add regular expressions to include filesystems to monitor.

You can include or exclude a filesystem from monitoring by adding comma-separated regular expressions in the Include and Exclude fields. For example, ^/scripts, ^/mnt, ^/local/utils.

 If the issue is not resolved, contact BMC Support with the following logs:

  • PATROL Agent configuration
  • Mount command output

Process-related issues

Issue

Solution

Process monitor unable to start a process (Process Monitoring for Linux) 

This issue has been fixed in PATROL for Linux version 1.2.00.04. Upgrade to the latest version of the KM to resolve the issue.

For more information, see Upgrading.

Debug

Issue

Solution

How to enable debug?

The Linux Debug monitor profile enables you to configure the KM debugging. You can enable debugging for various monitor types for a particular host. For more information, see Configuring-Linux-Debug-monitor-profile.

You can also perform the following actions and enable debug:

  1. In Truesight, open an Agent Query window for the PATROL Agent server for which you want to enable logs.
  2. Run the “%PSLPS” command.
    The PSL process list along with PID is displayed.
  3. In the output, search for the line like the following.
    67 NUK_FileSystem HALTED DISCOV NUK_FileSystem -
    Note: Ensure that you are observing the DISCOV line.
  4. Note the PID for that collector.
    In the example, PID is 67. On your environment it would be different.
  5. Run the following command on the "Agent Query":
    “%PSL trace_psl_process("PID", <pid number collected in #4 step>,-1);”
    Example: %PSL trace_psl_process("PID", 67,-1);
  6. Tracing for the filesystem starts.
  7. Restart the PATROL Agent.
  8. Wait for 5 minutes.
  9. Stop tracing using “%PSL trace_psl_process("PID", <pid number collected in #4 step>,0);” command on "Query Agent".
    Example: %PSL trace_psl_process("PID", 67,0);
    For debugging an issue for you, BMC Support might need the files from - /opt/bmc/Patrol3/log/trace/hostname/3181/, PATROL Agent configuration, Mount command output.

Remote monitoring-related issues

Issue

Solution

Remote monitoring prerequisites

To establish a stable connection between the PATROL Agent and a remote host, complete the following validation and configuration steps:

Verify basic connectivity and access

Perform the following validations from the PATROL Agent server:

  • Make sure your credentials are valid and verify that you can establish an SSH connection from the PATROL Agent server to the remote host without disconnections.

  • Confirm that an SSH2 server is installed and running on the remote host.

  • Check hostname resolution by running: nslookup <remote-host>

Validate SSH configuration on the remote host

Verify the following configuration settings in /etc/ssh/sshd_config:

PAM Setting

The UsePAM value must align with your security policy. To configure the UsePAM value, do the following:

  1. Open /etc/ssh/sshd_config file.
  2. Set it to either UsePAM yes or UsePAM no as appropriate.
  3. Save the file.

Session Limits

The MaxSessions parameter is appropriately configured. To configure the MaxSessions value, do the following:

  1. Open /etc/ssh/sshd_config file.
  2. Set the MaxSessions to 10. 
  3. Save the file.

Review remote host system logs

If connectivity fails, inspect the SSH logs on the remote host at /var/log/secure.
Example error observed during failure scenarios: fatal mm_request_receive_expect: read: rtype 125 != type 115
This error indicates issues related to SSH session handling, Cipher negotiation mismatches, and Privilege separation conflicts.

(Optional) Disable privilege separation (SSH workarounds for remote host)

Apply the following configuration only if all previous validations fail to resolve the issue:

  1. Open /etc/ssh/sshd_config file.
  2. Add UsePrivilegeSeparation no.
  3. Save the file.

This may help resolve session issues caused by privilege separation.

Password‑based authentication requirements

If the monitoring policy uses password-based authentication, you must explicitly enable password authentication on the remote host as follows:

  1. Open /etc/ssh/sshd_config file.
  2. Set PasswordAuthentication yes.
  3. Restart the SSH service for the change to take effect using the command: systemctl restart sshd

Contact BMC Support

If the issue persists after completing the above steps, please collect and provide the following to BMC Support:

  • Complete PATROL Agent configuration
  • Full PATROL Agent log folder
  • Any SSH logs related to the disconnection issue from remote host

Remote monitoring is not working on the Linux OS

Resolve "Broken pipe" error

 The occurrence of the message nukremotexec.xpc: XPC error 32 -- Broken pipe in the PATROL Agent error logs indicates permission or dependency issues.

Perform the following steps on the PATROL Agent host:

  1. Identify any missing dependencies by running the following command:
    ldd nukremotexec.xpc
  2. A common cause of this error is the absence of the libnsl library. Depending on your distribution, install it using the appropriate package manager:
    • For RHEL/CentOS, run the following command: yum install libnsl
    • For Ubuntu/Debian run the following command: apt-get install libnsl
  3. Restart the PATROL Agent to apply the changes.

Cipher compatibility

For remote monitoring to work, the PATROL Agent and the remote host must support compatible SSH ciphers.  Mismatched or deprecated ciphers can cause unexpected disconnects.

Remote monitoring is not working on Windows OS.

This issue may happen if the PATROL Agent default account directory is missing in c:\Users directory.

To create this folder user can run the below command from the command prompt:
runas /user:<mymachinename>\<patrol default account> cmd

For example:

runas /user:win-host-name\patroluser cmd

Enter the password for patroluser:

Attempting to start cmd as user "win-host-name\patroluser" ...

ssh client (Linux KM) will create .ssh directory under C:\Users\patroluser directory. .ssh directory is required for SSH key management.

MountStatus is showing Unknown status for filesystems. 

This will be observed when SUDO is configured in the remote host configuration policy.

Check if ‘Defaults requiretty’ is configured in /etc/sudoers file on the remote server.

Add below configuration in the /etc/sudoers file to fix this issue.
Defaults:<remote user> !requiretty

For example:

Defaults:patroluser !requiretty

Data collection-related issues

Issue

Solution

Data is not collected for CPU and Memory

Higher CPU or Memory utilization by PATROL for Linux KM 

CPU Utilization

  • Check the OS support
  • Start the PATROL Agent on a different port.
  • Check the process consuming high CPU using Agent profiling
    $ ./PatrolAgent -profiling/tmp/agentprof -p <port>
  • Agent writes all profiling data after termination; hence stop PATROL Agent with pconfig +KILL OR kill -15 (SIGTERM). Kill -9 (SIGKILL) is never recommended.
  • Profiling output is a binary file; use the following command to get text output of Agent profiling.
    $ ppv /tmp/agentprof > /tmp/agent_profiling.txt
  • If profiling does not give a legitimate output, run PATROL Agent without any KMs and then load KMs one-by-one to check when CPU utilization rises to determine which KM is causing the issue. 

Memory Utilization

  • Check OS support.
  • Check the process consuming high memory.
  • Start the PATROL Agent on a different port.
  • Verify if any old configurations exist and follow the following actions:
    1. Stop the PATROL Agent.
    2. Take the backup of config and log folders.
    3. Purge the Agent.
    4. Start the PATROL Agent.
    5. Check the PATROL Agent CPU consumption.

 If the issue is not resolved, contact BMC Support with the following logs:

Here are the steps to obtain PATROL Agent debug logs:

  1. Stop the PATROL Agent.
  2. Start the PATROL Agent in debug mode by running the following command
    • Linux: ./PatrolAgent -debug ALL,file="/some_filesystem/PAdebug.txt",count="10000000"
    • Windows: double-click the PatrolAgent service, stop the service. When the service is stopped, enter the following line (with appropriate modifications to path, filename, and count, as needed) in the 'Start Parameters' field: -debug ALL,file=C:\\patrol_agent_debug_output.txt,count=250000000

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*

BMC PATROL for Linux 25.3