Apache Kafka

This topic was edited by a BMC Contributor and has not been approved.  More information.

Related topics
Product name
Publisher page
  • [Apache|Apache]
Distibuted Messaging System
TKU 2024-Mar-1
More information
Publisher link

Product Description

Apache Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system.

Software Pattern Summary

Product ComponentOS TypeVersioningPattern Depth

Apache Kafka Server


Active, Path, Package, File name


Apache Kafka Cluster


Software Instance


Platforms Supported by the Pattern

The pattern discovers Apache Kafka Server deployed on both UNIX and Windows systems.


Software Instance Triggers

Apache Kafka pattern

The Apache Kafka pattern triggers on a java process which arguments match at least one of the following:

  • libs/kafka_[^/\s]+\.jar
  • Kafka
  • confluent.support.metrics.SupportedKafka

If the java process contains org.apache.zookeeper.server in the arguments, the pattern is not executed.

Apache Kafka Cluster pattern

The pattern triggers on a SoftwareInstance that matches the following criteria:

  • Type is one of “Apache Kafka Server”, “Confluent Kafka Server” or “Cloudera Kafka Server”
  • Zoo_family attribute matches the regular expression \S+\:\d+

Simple Identification Mappings


Apache Kafka

unix_cmd 'java'

regex '/libs/kafka_[^/\s]+\.jar'

Apache Kafka

windows_cmd 'java' or unix_cmd ‘java’

regex 'kafka\.Kafka'

Confluent Kafka Server

unix_cmd 'java'

regex 'io\.confluent\.support\.metrics\.SupportedKafka'


Find installation root

The pattern finds the installation root by parsing the trigger process arguments with one of the following regular expressions:

Regular expressionNotes




UNIX only


Classpath only


Classpath only

The pattern also checks the path to the trigger process command for the install root.

If multiple candidates are found, the pattern runs active versioning against all of the candidates and selects the first successful installation root. If no install roots are successful, the pattern selects the first installation root.

Active Versioning

To obtain version information, the pattern attempts to run the following commands: 

  • (UNIX only) <install root>/bin/kafka-topics.sh –version 
  • (UNIX only) <install root>/bin/kafka-topics –version
  • (Windows only) <install root>\bin\windows\kafka-topics.bat –version 

Path Versioning

The pattern checks for path versioning by parsing the trigger process arguments with the following regular expressions:

  • \bkafka_\d(?:\.\d+){1,2}\-(\d+(?:\.\d+)*)
  • \bkafka_\d(?:\.\d+|){1,2}\-(\d+(?:\.\d+)*)

Package Versioning

The pattern attempts to find versioning from all packages which name starts with confluent-kafka-

File name Versioning

If the process arguments refer to “cloudera”, the file name versioning is not attempted. 

The pattern lists the following searches for filenames that match the common\-config\-(\d+(?:\.\d+)*)\.jar  regular expression in the following directories:

  • <installation root>/current/share/java/confluent-common
  • <installation root>/share/java/confluent-common

 If the pattern finds the filenames, it obtains the full version.

Additional Attributes

Finding the configuration file

(Windows) The pattern finds the path to the configuration file by parsing the trigger process arguments with the following regular expression:

  • kafka\.Kafka\s*"?(\w\:.+[^'"])

(UNIX) The pattern attempts to find the path to the configuration file by parsing the trigger process argument with the following regular expression:

  • (?i)(?:kafka\.Kafka|io\.confluent\.support\.metrics\.SupportedKafka)\s(/\S+)

If this approach fails, the pattern attempts to find the configuration filename with the following regular expression:

  • (?:kafka\.Kafka|io\.confluent\.support\.metrics\.SupportedKafka)\s(?:\S+/)*(\S+)

In the third approach, the path to the file is assumed to be <installation root>/config/<config file name>


The pattern obtains the instance information by parsing the configuration file with the following regular expression:

  • Broker.id=(\d+)


The pattern obtains the port information by parsing the configuration file with one of the following regular expressions:

  • (?m)^\s*port=(\d+)
  • (?m)^\s*listeners\s*=.*?PLAINTEXT://.*?:(\d+)

If the port information is still missing, the default port is 9092.


The pattern sets the following publisher attributes:

  • If the process arguments refer to Cloudera, "Cloudera".
  • If the process arguments refer to Confluent or file name versioning is successful, — "Confluent". 
  • If the process arguments refer to none of Cloudera or Confluent, "Apache".

Application Model Produced By Pattern

The pattern produces an instance-based SoftwareInstance with a key based on a configuration file path, type, and host key.

Relationship Creation

The pattern creates a dependency relationship with the Apache Zookeeper Server. The pattern detects if the configuration file contains zookeeper.connect=(\S+). If this attribute is detected, its information is used to find what Zookeeper server to connect to.


The pattern creates a SoftwareCluster node that consists of all Kafka SoftwareInstances that share the same Apache Zookeeper Server.


The pattern models Kafka Topics if the relevant option is enabled. By default it is

The pattern gets details of the Topics by running one of the following commands


<install root>\bin\windows\kafka-topics.bat –zookeeper <zookeeper address> --describe

<install root>\bin\windows\kafka-topics.bat –bootstrap-server<port> --describe


<install root>/bin/kafka-topics.sh –zookeeper <zookeeper address> --describe

<install root>/bin/kafka-topics.sh –bootstrap-server<port> --describe

This command lists the names of all the Topics, along with details of their partition count and replication factor. 

The pattern gets details of what consumer groups are running on each Topic by running the following command:

  • %install_root%\\bin\\windows\\kafka-consumer-groups --bootstrap-server localhost:%port% --describe –all-groups
  • %install_root%/bin/kafka-consumer-groups.sh --bootstrap-server localhost:%port% --describe –all-groups

The pattern creates a Detail node for each topic and links it to either a standalone SI or a cluster.


The following image illustrates a typical installation of Apache Kafka:

Subject Matter Expertise

Subject Matter Expert input is welcome on any approaches towards improvement of the Apache Kafka model.

Information Sources

  • Apache Kafka documentation Open link
  • How to deploy and configure Apache Kafka in Cloudera Enterprise Open link
  • Running a Multi-Broker Apache Kafka 0.8 Cluster on a Single Node Open link

Open Issues

There are no known open issues with this pattern.

Was this page helpful? Yes No Submitting... Thank you