Skip to Content
Information

This site will undergo a brief period of maintenance on Friday, 18 December at 12:30 AM Central/12:00 PM IST. During a 30 minute window, site availability may be intermittent.

Information
Documentation update To provide a better user experience, we have now created a separate documentation space for BMC Helix Automation Console (previously called BMC Helix Vulnerability Management). Users of BMC Helix Automation Console can find the latest documentation at BMC Helix Automation Console..

Troubleshooting EFK logging issues

Consult this topic for information about troubleshooting the EFK logging issues.


The Kibana URL is not accessible

This issue occurs because no external IP is assigned to Kibana, and Kibana cannot be accessed from outside the cluster in the same network. Perform the following steps:

  • Verify that all pods and services are running.

  • Verify that an external IP is assigned to the Kibana service. Use the following command:

    kubectl get svc -n bmc-helix-logging

  • If no external IP is assigned, use the following command with any master-node IP:

    kubectl patch service elasticsearch-logging-kibana -n ade-logging -p '{"spec":{"externalIPs":["'10.129.111.192'"]}}'


The Kibana pod is in CrashLoopBackOff

This issue might occur if the network settings of multiple hosts are different. Perform the following steps to specify the host of the back-end server:

  1. Edit the Kibana config map and set the following value:

    "kubectl edit cm -n ade-logging elasticsearch-logging-kibana-conf"
          server.host: "0.0.0.0"

  2. Delete the Kibana pod by using the following command:

    kubectl delete pod <<podname>> -n <<namespace>>


The Fluentd daemon set pods are not visible

This issue occurs if the rbac or psp values are not set correctly in the chart_value.yaml file. Perform the following steps:

  • Ensure that the tsac-deployment-manager/bmc-helix-logging/efk/fluentd/chart_value.yaml file has the following setting depending on the Kubernetes management platform:
    • (For Rancher Kubernetes) rbac=true, psp=true
  • Ensure that the fluentd-privileged-binding role binding is present in the logging namespace.


Logs are not displayed in Kibana

This issue occurs when the forwarder runs the container as a non-root user. In the tsac-deployment-manager/bmc-helix-logging/efk/fluentd/chart_value.yaml file, verify that the securityContext of the forwarder has the following values:

securityContext:
enabled: true
runAsUser: 0
runAsGroup: 0
fsGroup: 0


EFK pods restart

This occurs because the Fluentd Deaemonset checks the health of the nodes. The pods restart until the Fluentd Daemonset receives the healthy status of the nodes.

If the installer displays the following message, it means that Fluentd needs more time than the default timeout duration in receiving the health status of the nodes:

ERROR: Failed to install helm chart: fluentd.
ERROR: Failed to install EFK-Fluentd.

Workaround

  • Wait till the Fluentd pods start.
  • Manually restart the nodes or restart the docker service.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*

TrueSight Automation Console 23.1