Troubleshooting container issues

This topic describes the issues that you might encounter with containers. 

Communication issues with the application server host

Issue

After installing TrueSight Vulnerability Management, you cannot access the application server due to IP subnet conflict.

Resolution

The default IP (172.xx.0.0/16) for the docker0 (docker network adapter/ bridge) conflicts with your datacenter's subnet CIDR. As a result, you cannot install the application.

Resolution

Perform the following steps to resolve the issue:

1. Ensure that you have downloaded and extracted the Stack Manager tool (TSVM<versionNo>-SM-LIN64.zip) from the BMC Electronic Product Distribution (EPD).

2. On the application server computer, navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory, and run the following command to stop the application services:

   python truesight-sm.py stop --deployment application

3. Stop the docker service by running the  following command:
   systemctl stop docker
4. On the machine where you want to install the application, navigate to the /etc/docker directory.
5. Open the daemon.json file with a text editor (create the file if it does not exist).
6. Add the following lines with the updated IP prefix (replace xxx.xx with the required subnet prefix) so that it doesn’t conflict with your dataceneter's subnet prefix:
   

   {

     "bip": "xxx.xx.0.1/16"

   }

7. Remove the docker0 network by using the following command:
   ip link del docker0
8. Start the docker service by running the following command:
   systemctl start docker

9. Navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and run the following command to start the application services:

   python truesight-sm.py start --deployment application

Communication issues with endpoint managers 

Issue

• Container is not able to communicate with the endpoint managers within the network or requests to ping the endpoint managers time out.

• Sometimes, when the docker container starts, the host machine where you want to install the application or the machine where the application is already installed, is inaccessible from the local machine due to docker subnet conflict between the host machine and the local machine.

Resolution

The most common reason for these issues is IP subnet conflict. That means, one of the docker's subnet is conflicting with the endpoint manager's subnet or subnet of the machine from where you are accessing the docker host.

To resolve the issue, perform the following steps:

1. Ensure that you have downloaded the Stack Manager tool from the BMC EPD.

2. Identify the container networks by running the following command. This command shows the network name and subnet.

   python truesight-sm.py network show --deployment application

3. From the output of the above command, identify the subnet that is conflicting with the local machine's IP address. 

4. After identifying the subnet, decide whether you want to change the subnet only for the component which is conflicting or change the subnet for all the components.

   • Follow these steps if you want to change the subnet only for the component which is conflicting:

   1. Stop the component of which subnet is conflicting. For example, if the workmanager subnet is conflicting, use the following command:

      python truesight-sm.py stop --components workmanager --deployment application

   2. Remove the conflicting docker network (for example, workmanager_data-net).

      python truesight-sm.py network delete --name workmanager_data-net --deployment application

   3. Create the subnet by specifying a different subnet and gateway, which is not conflicting with the local machine's IP address (as shown in the following example). Make sure to use the exact name as observed in the output of the docker network list earlier (for example, workmanager_data-net).

      python truesight-sm.py network create --name workmanager_data-net --subnet 172.27.0.0/16 --gateway 172.27.0.1  --deployment application

   4. Start the service by going to the service directory and starting it (for example, components workmanager). 

      python truesight-sm.py start --components workmanager --deployment application

   • Follow these steps if you want to change the network assigned to all the components:

     1. Navigate to the TSVM<versionNo>-SM-LIN64/truesight-sm directory and run the following command to stop the application services:

        python truesight-sm.py stop --deployment application

     2. Reassign the network for all the components:

        python truesight-sm.py network reassign --prefix <newIPAddress> --deployment application

        Replace newIPAddress with the new subnet prefix for the application containers' network in one of the following formats: x, xx. Here x represents an octet. For example, 138 or 138.10.

     3. Run the following command to start the application services:

        python truesight-sm.py start --deployment application

Application container cannot connect to the Elasticsearch server

Issue

The application container cannot connect to the Elasticsearch server and the bsm.log file shows the NoNodeAvailableException.

Resolution

This issue can happen because of the following reason:

• Reason 1: The DNS resolution from the application node is resolving to a different IP than what is specified for the Elasticsearch node (publish_host  entry in the elasticsearch.yml file).

• Resolution 1: To resolve this issue, ensure that both bmc-config.json and elasticsearch.yml files have the same host name, as follows.

  For example, the following bmc-config.json file shows the host name as es-qa.bmc.com.

  "com.bmc.dcaportal.index.service.core.ElasticSearchConfigManager":
  {
   ...
  "elasticsearch.hosts":
  "es-qa.bmc.com",  
  ...
  }

  The elasticsearch.yml file must show the same host name, as follows:

   # Set the address other nodes will use to communicate with this node. If not
   # set, it is automatically derived. It must point to an actual IP address.
   #
   network.publish_host: es-qa.bmc.com

• Reason 2: The cluster name mentioned in bmc-config.json does not match the name in the elasticsearch.yml file. 

• Resolution 2: To resolve this issue, ensure that both bmc-config.json and elasticsearch.yml files have the same cluster name, as follows.
  For example, the following bmc-config.json file shows the cluster name as esCluster.

  "com.bmc.dcaportal.index.service.core.ElasticSearchConfigManager": {
   ...
   "cluster.name": "esCluster",
   ... }

  The elasticsearch.yml file must show the same cluster name, as follows:

  # Cluster name identifies your cluster for auto-discovery. If you're running
  # multiple clusters on the same network, make sure you're using unique names.
  #
  cluster.name:esCluster