Preparing to collect logs from external log sources

Before you begin

• BMC Helix IT Operations Management version 25.1.00 or later must be installed.

• Make sure that the following ports are available in cluster nodes for Elasticsearch and Kibana services: 

  The [confluence_table-plus] macro is a standalone macro and it cannot be used inline. Click on this message for details.
  This macro generates standalone content. As a consequence you need to make sure to use a syntax that separates your macro from the content before and after it so that it's on a line by itself. For example in XWiki Syntax 2.0+ this means having 2 newline characters (a.k.a line breaks) separating your macro from the content before and after it.
  

  Important

  The traffic to Kibana ports is routed from the Load balancer on port 443 to the Ingress controller. As the Ingress controller decides the internal routing of the traffic, the Kibana ports should be open in the Kubernetes cluster.

To prepare to collect logs

1. Download the bmc-helix-logging-25.1.00-45.tar file from EPD.
   
2. Extract the tar file to the utilities folder.

3. Perform the following prerequisites that are relevant to your deployment.

   Deployment	Procedure
   Kubernetes	Use the default namespace or create a namespace in Kubernetes: Example: bmc-helix-logging. Navigate to helix-on-prem-deployment-manager/utilities/bmc-helix-logging/efk/fluent-bit/ and replace the efk-fluent-bit-clusterrole.yaml namespace with the namespace that created in step a. Use the cluster admin permission and apply the privileged policy to the new namespace. kubectl label namespace <namespace-name> pod-security.kubernetes.io/enforce=privileged kubectl label namespace <namespace-name> pod-security.kubernetes.io/enforce-version=latest kubectl label namespace <namespace-name> pod-security.kubernetes.io/audit=privileged kubectl label namespace <namespace-name> pod-security.kubernetes.io/audit-version=latest kubectl label namespace <namespace-name> pod-security.kubernetes.io/warn=privileged kubectl label namespace <namespace-name> pod-security.kubernetes.io/warn-version=latest To use a different namespace, go to bmc-helix-logging.config and set the variable BMC_HELIX_LOGGING_NAMESPACE to a new namespace.
   OpenShift	Use the default namespace or create a namespace in Openshift: Example: bmc-helix-logging. Using admin privileges, navigate to helix-on-prem-deployment-manager/utilities/bmc-helix-logging/efk/fluent-bit/ and run the following commands: oc apply -f efk-fluent-bit-scc.yaml to replace the efk-fluent-bit-scc.yaml namespace. oc apply -f efk-fluent-bit-clusterrole.yaml to replace the efk-fluent-bit-clusterrole.yaml namespace. oc adm policy add-scc-to-user efk-fluent-bit -z efk-fluent-bit -n <namespace> to add adm policy scc to the service account in the namespace. Add bmc-helix-logging GID in bmc-helix-logging.config. Example: LOGGING_RUN_AS_USER=1000750000 LOGGING_RUN_AS_GROUP=1000750000 LOGGING_FS_GROUP=1000750000
   Splunk	You have the Spunk deployed in your cluster. Splunk is running on a different namespace in the cluster. (Optional) If you are not using Splunk for logs collection, you must have the Fluentbit or Fluentd running in your cluster.

4. Go to helix-on-prem-deployment-manager/utilities/bmc-helix-logging/bmc-helix-logging.config and set the following values:
   • To use IPv4, set ESKIBANA_SERVER_HOST ="0.0.0.0"
   • To use IPv6, set ESKIBANA_SERVER_HOST ="::"
5. Run the ./bmc-helix-logging-deployer.sh script.
   The BMC Helix Logging deployer deploys EFK in the bmc-helix-logging namespace.

6. Perform the relevant post deployment steps:

   Action	Desctiption
   To access the Kibana URL	Open the bmc-helix-logging.config file. Find the KIBANA_LB_HOST parameter. Specify a URL to create a Kibana load balancer host.  The BMC Helix Logging Ingress uses the value of this parameter. Example: KIBANA_LB_HOST= kibana-private-poc.mydomain.com. Viewing-logs-on-Kibana.
   For Splunk	Add the Splunk plugin in Fluentbit configmap.  Kubectl edit cm fluent-bit -n <namespace>  Remove the output plugins other than Splunk from the Fluentbit configmap. Update the parameters: [OUTPUT]\n Name splunk\n Match kube.*\n Host <SPLUNK_HOST>\n Port <SPLUNK_PORT>\n Splunk_Token <SPLUNK_TOKEN>\n TLS On\n TLS.Verify Off\n Splunk host is the service name of Splunk and the namespace. <svc name>.<Splunk deployed namespace>Example: splunk-enterprise.splunk.svc.cluster.local [OUTPUT]        Name splunk        Match kube.*        Host splunk-enterprise.splunk.svc.cluster.local        Port <port>        Splunk_Token <token>        TLS On        TLS.verify Off Restart the Fluentbit pods. Go to Splunk to see the streaming logs.