Product Description

Apache Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system.

Software Pattern Summary

Platforms Supported by the Pattern

The pattern discovers Apache Kafka Server deployed on both Unix and Windows systems.

Identification

Software Instance Triggers

Simple Identification Mappings

The trigger process is given a simple identification mapping:

Obtain Install root

The pattern tries to retrieve root from following regular expression:

For Unix based operating systems:

• regex '(?i)kafka\.Kafka\s(\S+)/config\S+\.properties', in case of failure, pattern retrieve installation root from log-files directory:
  
• regex '-Xloggc:(/\S+)/logs/kafkaServer-gc\.log'
  

For Windows pattern uses next RegEx:

• 'Kafka\s*"?(\w\:.+)\\config\\'

The pattern also attempts to fetch zookeeper_chroot by following regular expression:

•  regex '(/\S+)'

Obtaining the version information 

The pattern obtains version information through active

• "<install_root>\\bin\\windows\\kafka-topics.bat" --version

• <install_root>/bin/kafka-topics.sh --version

and path versioning by following one of the regular expressions:

• regex '\bkafka_\d(?:\.\d+){1,2}\-(\d+(?:\.\d+)*)'

Obtaining Configuration file

The pattern obtains configuration information by following regular expression:

•  regex 'kafka\.Kafka\s(?:\S+/(\S+\.properties))' - for Unix
• 'Kafka\s*"?(\w\:.+\\config\\server.properties)' - for Windows

Obtaining Port information

The pattern attempts to retrieve port information for zoo_keeper port using following regular expression:

•  regex ':(\d+)'

Application Model Produced by Software Pattern

Software Pattern Model

Pattern Trigger

The Apache Kafka server pattern triggers on java process which has '/opt/kafka_2.10-0.8.2.0/bin/../core/build/libs/kafka_2.10*.jar' or '"C:\Program Files (x86)\MicroStrategy\Messaging Services\Kafka\kafka_2.11-0.10.1.0\libs*" kafka.Kafka' in arguments.

The Apache Kafka Cluster patterns triggers on Software Instance which depends on zookeeper host.

SI Depth

The Apache Kafka Server pattern models Software Instance which key is based on location of config file for each instance.

The Apache Kafka Cluster models Software Instance which key is based on zookeeper_chroot, SI type and zookeper service key.

Software Instance Modeling

This is version for SI for each instance found:

Updated

Software Instance-Cluster

Cluster member

Kafka Topics Details

Kafka Topics are obtained using active command - 'kafka-topics --bootstrap-server 127.0.0.1:<port> --list' and modeled as Details of SoftwareInstance or SoftwareCluster:

Relationship Creation

The pattern creates the following relationship between Apache Kafka and Apache Zookeeper.

Subject Matter Expertise

Subject Matter Expert input will be welcome on any other potential approaches not discussed to improving product versioning coverage and depth of Apache Kafka model.

Information Sources

• http://kafka.apache.org/documentation.html
• https://kafka.apache.org/08/ops.html
• http://kafka.apache.org/downloads.html
• http://blog.cloudera.com/blog/2015/02/how-to-deploy-and-configure-apache-kafka-in-cloudera-enterprise/
• http://www.michael-noll.com/blog/2013/03/13/running-a-multi-broker-apache-kafka-cluster-on-a-single-node/

Open Issues

There are no known open issues with this pattern.