Configuring the KM to work with Kerberos Kafka server environment

PATROL for Apache Kafka supports the Kafka server that uses Java Authentication and Authorization Service (JAAS) for SASL configuration. 

To be able to work with the Kafka KM and BMC Producer to connect to the Kerberos Kafka server, those clients will authenticate to the cluster with their own principal (usually with the same name as the user running the client), so obtain or create these principals as needed. Then configure the JAAS configuration file.

A principal is divided into three parts, the primary, the instance, and, the realm.

The format of a typical principal is primary/instance@REALM

The primary is the first part of the principal. In the case of a user, it is the same as your username. For a host, the primary is the word host.

The instance is an optional string that qualifies the primary. The instance is separated from the primary by a slash (/). In the case of a user, the instance is usually null, but a user might also have an additional principal, with an instance called admin, which he/she uses to administrate a database.

The realm is your Kerberos realm. In most cases, your Kerberos realm is your domain name, in upper-case letters. For example, the machine daffodil.example.com would be in the realm EXAMPLE.COM.

Prerequisites

Creating Kerberos Principals

If you are using the organization's Kerberos or Active Directory server, ask your Kerberos administrator for a principal for your Apache Kafka KM and BMC Producer.

If you have installed your own Kerberos, create these principals by using the following commands:

sudo /usr/sbin/kadmin.local -q 'addprinc -randkey kafka/{hostname}@{REALM}'
sudo /usr/sbin/kadmin.local -q "ktadd -k /etc/security/keyTabs/{keyTabname}.keyTab kafka/{hostname}@{REALM}"

Note

If you are using a Microsoft Windows path in the keyTab variable, the path must contain one forward slash or two backward slashes.

For example, the path can be:

keyTab="C:/security/keyTabs/kafka_client.keyTab"

OR

keyTab="C:\\security\\keyTabs\\kafka_client.keyTab"

The KafkaClient section describes how the client can connect to the Kerberos Kafka cluster. Following is an example configuration of a client using the keyTab (recommended for long-running processes).

KafkaClient {
        com.sun.security.auth.module.Krb5LoginModule required
        usekeyTab=true
        storeKey=true
        keyTab="/etc/security/keyTabs/kafka_client.keyTab"
        principal="kafka-client-1@EXAMPLE.COM";
    };

Configuring the BMC Producer 

Before you begin

  • Ensure that you have the Kerberos krb5.conf configuration file. This file includes the locations of KDCs and admin servers for the Kerberos realms of interest.
  • Ensure that the krb5.conf file is located at /etc/krb5.conf on the Kerberos server computer.

To configure the BMC Producer 

  1. In the security directory, open the producer_jaas.conf file in edit mode

  2. Enter the keys in the KafkaClient section:

    1. keyTab="<kafka client keyTab location>"
    2. principal="{service name}/{hostname}@{REALM}";

    The producer_jaas.conf file provides the following information:

    • Informs the Broker which principal must be used
    • Location of the keyTab where the principal is stored
    • Ensures that the {hostname} is reachable using hostnames (this is a Kerberos requirement that all your hosts can be resolved with their FQDNs)
  3. Save the file.
  4. Ensure that the keyTab file points to the correct location. The keyTabs configured in the JAAS configuration must be readable by the operating system user who is starting the Kafka client. 
  5. In the conf directory, open the producer.properties file in edit mode.
  6. Uncomment the following lines:
    1. security.protocol=SASL_PLAINTEXT
    2. sasl.kerberos.service.name=<The Kerberos principal service name that Kafka runs as. Can be found in the producer_jaas.conf file in the principal section>
  7. Save the file.
  8. Set or export the following environment variable
    1. Unix: export JVM_ARGS="-Djava.security.krb5.conf=/etc/kafka/krb5.conf -Djava.security.auth.login.config=/home/bmc/BMCProducer/security/producer_jaas.conf"
    2. Microsoft Windows: set JVM_ARGS=-Djava.security.krb5.conf="c:\\kerebors\\krb5.conf" -Djava.security.auth.login.config="C:\\bmc\\BMCProducer\\security\\producer_jaas.conf"
  9. You can now run the bmc_producer tool.

Configuring PATROL for Apache Kafka

Before you begin

Ensure that you have the krb5.conf file and create the JAAS configuration file. Follow the same procedure as documented in the BMC Producer section.

You can use the same JAAS config file.

  1. In the conf directory, open the consumer_jaas.conf file in edit mode 
  2. Enter the keys in the KafkaClient and the Client section:
    1. keyTab="<kafka client keyTab location>"

    2. principal="{service name}/{hostname}@{REALM}"; 

      The consumer_jaas.conf file provides the following information:

      • Informs the Broker which principal must be used
      • Location of the keyTab where the principal is stored
      • Ensures that the {hostname} is reachable using hostnames (this is a Kerberos requirement that all your hosts can be resolved with their FQDNs)
  3. Save the file.
  4. Ensure that the keyTab file points to the correct location. The keyTabs configured in the JAAS configuration must be readable by the operating system user who is starting the Kafka client. 

In the TrueSight policy configuration, enter the details as described. 

kerberos_security_details.png

Field

Description

Security protocol

Enter SASL_PLAINTEXT

SASL Kerberos service name

Enter the service name described in the JAAS file

Krb5 config path

Enter the path to the krb5.conf file

JAAS config path

Enter the path to the JAAS configuration file that you have created

zookeeper_connection_details.png

If you have entered an FQDN in the JAAS file, the same format must be used for the Host/IP in the Zookeeper connection details.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*