Frequently asked questions for agent troubleshooting

Resolution: Installation of an RSCD agent requires approximately 200 MB of space on Windows platforms and 450 MB of space on Linux and UNIX platforms. When running, an RSCD agent on any platform requires approximately 60 MB of available disk space on 32-bit platforms and 140 MB on 64-bit platforms. 
See Minimum hardware requirements.

Resolution: Please refer to Knowledge Article ID: 000090421, or to the following pages in the documentation:

• User accounts
• Considerations for automation principals and Windows user mapping

When pushing ACLs you might observe that extra roles are added to the users file.  This is because the ACL Push and ACL Preview resolve the permissions for the server object as well as any Component objects linked to the server. Roles that have permission on the associated Component will be resolved in the users file and pushed to the Server. For more information, see Creating or modifying ACL Push Jobs.

Resolution: Change the umask in /etc/bashrc on the Linux server to 022 and restart the agent before the copy operation takes place. The idea is to change the umask value in the environment of the agent process.

Resolution: The -profile switch of blcred cred command overrides the profile set using environment variable BL_AUTH_PROFILE_NAME: This override applies only to the process of issuing the credential.

However, it does not act as a profile selection during authentication against the NSH-Proxy. This selection still has to be made separately, either by setting the environment variable BL_AUTH_PROFILE_NAME, or by specifying the profile in the secure file.

Resolution: Certain RSCD Agent directories (typically within /opt/bmc/bladelogic/RSCD) have world-writable (777) permissions, with a sticky bit. The following table provides information about why these directories (which all reside within the RSCD Agent installation directory) require world-writable permissions. In general, the directories are world-writable so that when roles are mapped to different local user accounts, these accounts are able to write various data into the following directories:

The application of the umask setting through the RSCD Agent varies, depending on a number of factors:

• In the case of a login process (nexec –i –l . . .), profiles that create the login environment are executed, and these profiles set the umask
• In the case of a non-login process (for example, cp //<rscd-host>/ . . . ), umask is inherited when the agent is started. If the agent is started from the login shell by the root user (/etc/init.d/rscd start or agentctl start/restart from the root user's bash shell command prompt), it inherits the umask that was set by the login shell (or any subshells), if any. Umask can also be set explicitly in /etc/init.d/rscd. This also includes scriptutil executions and "Type 3" NSH Jobs.
• If the RSCD Agent is started during the system start or reboot, /etc/init.d/functions sets the umask.

Resolution: The files below are used by Magnicomp's sysinfo executable, which is used by the Hardware Information CO to gather information about the system. This information is typically included in the Inventory Snapshot Jobs that populate the out-of-the-box inventory reports in BDSSA. If BSA roles that gather this information are mapped to root or if the information is not needed, then the suid bit is not required. If the BSA roles that gather the information are mapped to a non-root account, then the suid bit is required for the executables to run correctly.

• <RSCD installDir>/nativetool/bin/.mcsiwrapper
• <RSCD installDir>/nativetool/bin/mcsysinfo (symlink to .mcsiwrapper)
• <RSCD installDir>/nativetool/bin/sdi (symlink to .mcsiwrapper)
  
• <RSCD installDir>/nativetool/bin/sysinfo (symlink to .mcsiwrapper)
  

The following additional files in the RSCD Agent install directory are also granted the setuid permissions. Similar to the group of files listed above, if the BSA roles that execute these functions are mapped to root, then the suid bit is not required.

• <RSCD installDir>/bin/vsh — a virtual shell that provides keystroke logging
• <RSCD installDir>/bin/lsof — used to set up x-windows forwarding through the NSH>RSCD connection

The scenario: After caching the credentials at the console startup time, when a domain user runs an NSH script as a custom command (which sets up the authorization profile and RBAC-role using the blcli_setoption command), the user is prompted for the RBAC-role, even though the RBAC-role is cached by setting the blcli_setoption, and the environment variable BL_RBAC_ROLE is set.

Resolution: Delete the user and create a new one.

Resolution: Ensure <user> belongs to the administrator group.

Resolution: Modify the target servers exports file and change * to the IP addresses of targets from which you wish to NSH to the target.
For example:
Anyip1 rw
Anyip2 rw

Resolution: Use one of the following methods:

• Upgrade the server to 8.1 SP3.
• Delete the BladeLogicRSCD user and restart the agent service.

Resolution: On a Windows machine, BMC Server Automation uses a technique called user privilege mapping, which allows the agent to temporarily grant the local user's group privileges to an unprivileged user account called BladeLogicRSCD (or BladeLogicRSCDDC, in a domain controller environment). This privilege mapping mechanism allows the agent to acquire the mapped local user's group privileges without having to access that user's Windows credentials (user name and password).
The BladeLogicRSCD user is created on the Windows during the installation of RSCD, to obtain the local user privileges on the target server, based on the mapping policies defined in exports, users, and user.local files. 

If entries in the configuration files map the client user to a local user, the agent uses a NetUserGetInfo API call to obtain detailed information about this local user and temporarily acquires the group privileges that the managed server's operating system grants to this local user. In this way BMC Server Automation takes advantage of the access control mechanisms provided by the remote server's operating system.

By default, the SeBatchLogonRight and SeDenyInteractiveLogonRight privileges are granted to BladelogicRSCD user.
Additionally, the SeSecurityPrivilege privilege is added when BladelogicRSCD user impersonates a local user.
When the incoming request gets mapped to a local user, to be impersonated, its privileges are granted to the BladelogicRSCD user. 

At the time of installation, a random password is generated for the BladelogicRSCD user account.
The password can be modified using the nsh command "chapw".

Resolution: By default, primary communication port from Application Server to each managed host is 4750. 
Apart from the above mention port, the following ports are important in BBSA environment. These ports should be opened based upon the configuration on need basis: 

• 9840 - Authentication Serivce
• 9841 - Application Server
• 9842 - NSH Proxy Server

Resolution: 4750 

A complex password of up to 16 characters, consisting of alphanumeric and special characters. 

Yes. Customers can use chapw -p to create a password of up to 80 characters, after the initial password creation.

On a domain controller, you can also use the agentctl utility to change the password of the BladeLogicRSCDDC user account, as discussed in Changing the BladeLogicRSCDDC account password on domain controllers.

No. chapw -r creates a random password of with a length of 15 characters, that contains only alphanumeric characters.

Resolution: The ESX agent on EPD has been certified only till ESX 4 – 32 bits. It is not certified on ESX 64 bit. The Linux 64 bit agent should work, though.
VMware does not allow any third party agents on ESX, if found they will not provide support to the customer on his/her ESX machine.

Resolution: No. It is not a supported platform.

Resolution: The registry needs to be read to detect if it is a VC Agent. For this, the user mapping must be to an Administrator user. Check the user mapping with the agentinfo command.

Resolution: NSH is a shell similar to ZSH, BASH, etc. An NSH script can be written using a combination of commands of UNIX-like shell script and BMC Server Automation commands like cp, agentinfo, blcli, blquery, nexec, etc. 

Sample script:  

#!/bin/nsh 

If test "`uname --s`" != "WindowsNT"

then 
UID_GID=`agentinfo 10.20.32.252 | egrep "User Permissions" | awk -F ':' '{print $2}' | awk '{print $1}'`
UID=`echo $UID_GID | cut --d'/' --f1`
GID=` echo $UID_GID | cut --d'/' --f2' 
echo "Mapped User Id = $UID"
echo "Mapped Group Id = $GID" 
fi 

exit 0

Resolution: RSCD Agent & NSH are supported on AIX71 from version 8.1.02

Resolution: The details are as follows:

• BL version 8.2: 8.0.50727.762
• BL version 8.1: 8.0.50727.762
• BL version 8.0: 8.0.50727.762

Resolution: No. The RSCD Agent will not work if the following runtime llibraryversions are missing:

• BL version 8.2: 8.0.50727.762
• BL version 8.1: 8.0.50727.762
• BL version 8.0: 8.0.50727.762 

Resolution: Multiple RSCD Agent installation is supported on UNIX host with an exception of HPUX (using -local option with installer).
On UNIX hosts, you will have to use additional -local option with installer.
For more information, see How to configure two RSCD agents on a single host (user contribution). 

Resolution:The following applications may have an impact on the smooth running of RSCD Agent:

• VC++ 2005 libraries bundled with the agent, which may be a problem if 2008 is the only version authorized on a system by customer's internal policy.
• Applications configured to use the same port as the BMC Server Automation RSCD agent listening port (4750): The listening port of the agent can be customized if necessary.
• Anti-virus software which prevents the agent from accessing its own files, or communicating with the application server.

Resolution: nousers is pushed if the server property PUSH_ACL_NOUSERS_FLAG is set to true (Flag default value = true). 
nousers denies authorization to access the host to any user not specified in the users/users.local file.

Resolution: This is one of the most common error messages which specify that you are not allowed to access the remote host. This error message is given in multiple scenarios, so for the exact cause of the error, you will have to see the rscd.log on the remote host. 

Resolution: This error is seen in the rscd.log if the host from which you are trying to connect to a remote host doesn't have permission on that remote host.
This will usually happen if you don't have entry for "*" in the exports file.
For more information, see Setting up configuration files.
 

Example:

 On HOST2, if the exports file contains:

 HOST3rw 

 When we try to access HOST2 from HOST1:

 [root@HOST1]$ agentinfo HOST2
 

Can't access host "HOST2": No authorization to access host
 

[root@HOST1]$ 
 

Whereas in the rscd.log on HOST2 we will see:

 05/15/12 04:11:40.802 WARN rscd - HOST1 24646 0/0 (root): agentinfo: Host not authorized 

Resolution: The tcptunnel command is allowed to be executed only by a RBAC user.
If you are trying to run it with native OS user like "Administrator", you will see this error in the rscd.log file. In short tcptunnel must be executed by an authenticated user (RBAC user). This RBAC user must be granted the Server.TCPTunnel authorization.

Example:

On HOST2, if the users file contains:

 rootrw,commands=tcptunnel 

Now when you run tcptunnel from HOST1 as "Administrator" user

 [Administrator@HOST1] > tcptunnel -c 12345 -s HOST2
Can't access host "HOST2": No authorization to access host
[Administrator@HOST1] > 

 Whereas in rscd.log on HOST2 we will see:

 05/15/12 16:04:22.926 INFO1 rscd - HOST1 12160 SYSTEM (Administrator): tcptunnel: User Administrator not authorized to create TCP Tunnel 

Resolution: By using the rootdir option in the configuration file (exports, users or users.local) we can restrict the access to the directory tree on the RSCD Agent. If the RSCD agent can't change the root directory to the configured value, we would see this warning.
For more information, see Setting up configuration files .

Example:

On HOST2, the /opt/test directory doesn't exist and if the users file contains:

 rootrw,rootdir=/opt/test

Now when we try to access HOST2 from HOST1 like:

 [root@HOST1]$ agentinfo HOST2
Can't access host "HOST2": No authorization to access host
[root@HOST1]$ 

 Whereas in the rscd.log on HOST2 we will see:

 05/15/12 04:24:45.782 WARN rscd - HOST1 25287 0/0 (root): agentinfo: Failed to change the root directory

Resolution: If we want to execute a command on a remote host with the user permissions other than the mapped user on remote host, then we need to use the NSH rsu command.
For more information, see Setting up configuration files . 

This error is primarily seen in the following cases:

• The alternate user doesn't exist on the remote host
• The alternate user exists, but the wrong password is entered.

Example:

On HOST2, if the user named USER1 doesn't exist, and we try to run the command from HOST1 as:

[root@HOST1]$ rsu USER1 ls //HOST2/opt
ls: //HOST2/opt: No authorization to access host
[root@HOST1]$ 

On HOST2, we see following warning in the rscd.log

05/15/12 04:51:37.411 WARN rscd - HOST1 26560 0/0 (root): ls: Failed to change to alternate user

Resolution: On the remote host command execution can be limited by using the commands option in the configuration files (exports, users or users.local). If this option is not specified, then all NSH commands are allowed. If the command client is trying to execute on the remote host is not available in this commands  option, the command is rejected and we see the error in the rscd.log file.
For more information, see Setting up configuration files . 

Example:

 On HOST2, if the users file contains

 rootrw,commands=ls 

Now from HOST1, if you run a NSH command like:

 [root@HOST1 ]$ du //HOST2/tmp
du: Unable to access file //HOST2/tmp/: No authorization to access host
[root@HOST1 ]$ 

 You will see following error in the rscd.log on HOST2:

 05/15/12 04:03:38.585 WARN rscd - HOST1 24288 0/0 (root): du: command: "du" not authorized 

Resolution: The configuration files (users or users.local) provides options (map) to force all incoming client connections to run with permission of local user. If the specified local user doesn't exist on the remote host, we see this error in rscd.log file. For more information, see Setting up configuration files . 

Example:

On HOST2, the local user "USER1" doesn't exist and if the users file contains:

rootrw, map=USER1 

Now from HOST1, if you run a command like:

[root@HOST1] $ agentinfo HOST2
Can't access host "HOST2": No authorization to access host
[root@HOST1] $ 

You will see following error in the rscd.log on HOST2:

05/15/12 05:04:46.073 WARN rscd - HOST1 27173 65534/65533 (root): agentinfo: Failed to map user to local user

Resolution: This error is reported when the product is not installed correctly or secure file gets corrupted. Reinstallation of RSCD Agent may resolve the issue.

Resolution: The configuration files (exports, users, users.local) provides options ( allowed, validusers) to restrict the access to remote host based on the incoming user.
If the user with which you are trying to connect to remote host is not allowed, you see this error in the rscd.log file.
For more information, see Setting up configuration files. 

 Example:

 On HOST2, if the local user USER1 doesn't exist, and if the exports file contains:

 *rw,allowed=USER1 

 Now from HOST1, if you try to access HOST2 like:

[root@HOST1]$ agentinfo HOST2
Can't access host "HOST2": No authorization to access host
[root@HOST1] $ 

 You will see following error in the rscd.log on HOST2:

 05/15/12 05:14:55.330 WARN rscd - HOST1 27633 0/0 (root): agentinfo: Failed to map the user to allowed/validuser

 Resolution: Check if non-existent/non-reachable servers are listed in the secure file. If yes, remove them.

Resolution : 'TARGET.NAME' and SCAN_FOLDER are properties that come from the User interface (UI) to scriptutil, when scriptutil is called from within the GUI. 
So it would be something like the example:  

FOO=server1
Scriptutil -h $FOO ...
Scriptutil -h $FOO -s <script> -x arg1 arg2 arg3 

Where arg1, arg2, arg3 are input parameters to script. 

If x is set to -1 then, all directories are searched.

Resolution: nnet retrieves information of interfaces installed on the system from the kernel, so the relevant information would be present irrespective of the state of interface.

Resolution: limits.conf only applies to a shell session. These configuration settings do not apply at boot time, because no shell is started at that time.
Thus, if you start the agent at boot time, you will have to define the limits in the agent startup script.
This is standard UNIX practice, nothing to do with BMC Server Automation.

For more information, see Ubuntu forums entry

Resolution: See Add NSH Script - Parameters and Add NSH Script - Script Options for details.

The values that you pass in the Value box get passed to the scripts as arguments. If you pass 3 values, inside the script, you can access them with $1, $2 and $3.

Resolution: If you are unable to access the servers after an IP Address change, perform the following tests:

1. From the application server run a ping name, where name is the name of the server object in BSA. Confirm that the Application Server can resolve the correct IP Address. It is OK if the ping itself fails.
2. If the ping correctly resolves the IP Address, confirm the DNS cache settings for the Application Server, as configured in the java.security file (<install directory>/NSH/jre/lib/security/java.security  on Windows;  <install directory>/NSH/br/java/lib/security/java.security on Linux).
   In this file, if the networkaddress.cache.ttl value is set to -1 (cache forever), this means that the Application Server has cached the old IP and will not find the new IP until the cache is cleared (appserver restart). Since a restart will be required to pick up the new IP and the cache ttl, this is also a good time to adjust the ttl setting so that the Application Server does not cache lookups forever, so that this problem does not happen in the future. We recommend a value of 600 seconds (10 minutes) for the networkaddress.cache.ttl setting. 
3.  After the correct IP address is resolving, check if a Verify action or an Update Server Property (USP) Job successfully runs against the target, communicates with the target, and updates the IP_ADDRESS property for the server object with the new IP address.
4. If the Verify action or USP Job fails, perform a few basic connectivity checks:
   1. Can you run agentinfo from the Application Server (that is, directly from the Application Server, not from NSH Here or NSH launched from your console) against the target. Do you see connections come through to the agent?
      
   2. Confirm whether there are any firewall rules that might be blocking access to the new IP (for example, if the new IP is in a different VLAN compared to the old one).
   3. If there is a SOCKS Proxy in use to communicate with this target, confirm whether the correct SOCKS rules are being used for the new IP. Take this opportunity to check repeater routing rules, as well.
      
5. If you receive an error such as "No authorization to access host: Hostname," then check the exports file on the target and confirm that it is allowing the Application Server(s) to connect. If the appserver hostnames are listed, confirm that the target can correctly resolve the appserver name to IP. The reason for this test is that perhaps along with the IP Address change on the target, the DNS servers that it uses were changed and they cannot resolve the appserver name.
   

Resolution: If the user decommissions the target server by opting for deregister license from the licensing portal, then that target not counted as licensed.
From the 8.2 release, there is no concept of Automatically License/Deregister License 

Resolution: Post install, the RSCD agent will get evaluation license of two weeks, provided the RSCD agent installed on the target server for the very first time.
From 8.2, there is no concept of temporary license. Post installation, the target server is automatically licensed. 

Resolution: RHEL 5.0 comes with a firewall ON by default. Firewall could block the communication between the agent running on the RHEL server and BMC Server Automation client/servers. Reconfiguration of firewall to allow communication on agent port could resolve the issue. 

 Resolution: Please refer to Creating rules to determine the repeater used for target servers

Resolution: Perform the following procedure:

1. Stop the RSCD Agent.
2. Delete certificate.pem.
3. Start the Agent.

Resolution: Delete the BladeLogicRSCD user and restart the RSCD service. This may occur if the product is not installed appropriately: Reinstallation of the RSCD Agent may be helpful in such situations. 

Resolution: This issue may arise after incomplete installation or upgrade of the RSCD agent. Reinstalling the RSCD Agent may resolve the issue.

Resolution: Starting with version 8.2.00, we can deploy RSCD agent on Windows vio GPO.

Use the following installers:

•  RSCD82-WIN32.msi
•  RSCD82-WIN64.msi

Resolution: When an agent is uninstalled, the license file is removed. When the user installs the new agent, because the agent is not licensed, the RSCD_DIR is empty.
Decommission and add the server back by opting for auto-licensing, to populate the RSCD Agent installation directory. 

Resolution: One of the causes for this situation may be that the additional hostname of the RSCD host is not getting resolved to its IP.
Adding the hostname and IP address in the hosts database (for example /etc/hosts) may resolve this issue. 

Resolution: There is no method by which exports file can be edited on the go during the silent install.
Once the installation is complete, it can be edited to have desired entries/user mappings in it. 

Resolution:

1. Verify the system event logs (on Windows)
2. Set the agent log level to debug by one of the following methods:
   • Manually, by editing log4crc.txt, and then restarting the agent.
   • From version 8.2 SP1 and later: Use agentctl toggle, to switch between the debug level and the current level.

Resolution: Use --local flag while installing/upgrading RSCD agent on Solaris 10 sparse root zone.
For example, RSCD82-SOL10-SPARC.sh --local

Resolution: First, upgrade NSH on the server, and then the RSCD Agent. Another way is upgrade both the console and NSH, and then to upgrade the RSCD Agent.

Resolution: As of now there is no way by which the rscd log custom path can be specified during the silent install. Though, after the silent install is complete the user defined rscd log can be specified in the log4crc.txt file which is located at /etc/rsc or /usr/lib/rsc.

Resolution: Follow the steps:
UNIX:

1. Log on as root to the server where the RSCD agent installation program resides.
2. Copy <RSCD version>-<platform>.sh (the RSCD agent installation program) to a temporary directory (for example: /tmp).
3. Grant executable permissions to <RSCD version>-<platform>.sh.
4. Use the console to browse to the server to which you copied the RSCD Agent installation program.
5. Select <RSCD version>-<platform>.sh, right click & choose Deploy files.
6. For destination directory, enter /tmp.
7. Choose the servers with agents which needs to be upgraded

Windows:

1. Copy the RSCD agent installation program (rscd<version>.exe) to <WINDIR>\Temp, where <WINDIR> is typically C:\winnt or C:\Windows.
2. From the command line, connect to <WINDIR>\Temp.
3. From the command line, enter rscd<version> -a -r -f1<WINDIR> \Temp\Version-install_type.iss.
   The installation program for the RSCD agent opens. 
4. Perform an initial RSCD agent installation.
5. Using the CM Console, browse to the <WINDIR>\Temp directory.
6. Select and deploy the following files:
   • rscd<version>.exe
   • <version>-upgrade.iss        

The files are deployed to target servers and the installation runs silently.

Resolution:

1. Log on as root to the server where the RSCD agent installation program resides.
2. Copy RSCDversion-platform.sh (the RSCD agent installation program) to /tmp.
3. Make RSCDversion-platform.sh executable.
4. Use the console to browse to the server to which you copied the RSCD Agent installation program.
5. Select RSCDversion-platform.sh, right click & choose Deploy files.
6. For destination directory, enter /tmp.
7. Choose the servers with agents being upgraded.

Resolution:

If you've just installed an RSCD Agent and are experiencing the following issues:

• The RSCD Agent doesn't start.
• Logs are not showing up in the rscd.log file, or the file is not created. 

It may be that the RSCD Agent is not able to resolve your hostname to an IP address. To validate the issue, perform the following procedure:

1. Run the hostname command.
2. Run ping <output from above command>.
   

If the ping command is not able to resolve the hostname to IP address, then it is an issue.

BMC recommends that you follow the policy set by your organization for hostname to IP resolution. However, as a short term solution, you may add an entry in the /etc/hosts file for this hostname and manually restart the RSCD Agent.  

1. Include the following text:
   127.0.0.1localhost
   <IP address><server name>
2. Manually restart the RSCD Agent.

Example: For a server with the following credentials:

• Server name: myserver
• IP address: 192.168.0.9

You would use:

127.0.0.1localhost
192.168.0.9myserver

Resolution: If the registry value for the HKEY_LOCAL_MACHINE\SOFTWARE\BladeLogic\RSCD Agent\AgentHome value is set incorrectly, then there may be the issue while starting RSCD Agent.
This can be easily obtained by looking at the properties of the RSCD service.

Resolution: You can restart the remote RSCD Agent using following command from NSH:
nexec <HOSTNAME> agentctl -b restart

Resolution: nativetool_installer should not be removed. If removed, the package manager will report the files as missing when package is verified. If anyone doesn't care about the package verification they can feel free to delete the folder.

Resolution: After upgrade, execute the Update Server Properties job against all the upgraded targets. Use the agentinfo command to verify that the targets show the correct agent version.

Resolution: The platform default locations are as follows:

• Platform: Solaris/Linux/AIX/HPUX Location: /usr/lib/rsc
• Platform: Windows Location: WINDIR\rsc (WINDIR can be \windows or \winnt.)

This information is covered in Administering.

drwxrwxrwt root|root /exec/products/bladelogic/NSH/Transactions
drwxrwxrwt bin|bin /exec/products/bladelogic/NSH/snapshot 

Resolution: The owner, group,and permissions for both directories are correct.

•  The Transactions folder stores data related to deploy jobs (logs, roll-back data, and so on).
• The snapshot folder is used for temporary storage of snapshot data when snapshot jobs are run against a server.

Resolution: The root user should be the owner of these files.

• drwxr-xr-x 5117|man /exec/products/bladelogic/NSH/blyum/sqlite
• drwxr-xr-x 5117|man /exec/products/bladelogic/NSH/blyum/python
• drwxr-xr-x 5117|man /exec/products/bladelogic/NSH/blyum/yum
• drwxr-xr-x 5117|man /exec/products/bladelogic/NSH/blyum/glib2 

Resolution: Path for RSCD agent configuration files (users, users.local, and so on) is /usr/lib/rsc and /etc/rsc from product version 8.2 and later. 

Resolution: You can use any rule which you will specify for the repeater routing rule. Here you can select any property, such as Location, Name, and so on.

 Resolution: No, you can assign any other property value in Repeater rule definition to let the system know which repeater assigned to it.