Configuring the Kafka actor and monitor adapters

This topic describes the procedure to configure the Kafka actor and monitor adapters and provides sample XML configurations.

You configure an adapter in Grid Manager. Adapter configuration determines how the adapter interacts with Apache Kafka. You can configure multiple adapters of the same type, however, each adapter must have a unique name. You configure the Kafka actor and monitor adapters using the XML view only. 

To configure the Kafka actor adapter

The actor adapter uses Kafka Producer API. 

1. On Grid Manager, go to Manage > Adapters.
2. From the Adapters in Repository list, select the bmc-adapter-kafka-actor adapter, and click Add to Grid to add it in the Adapters on Grid list.
3. Click Configure.
4. Enter a unique name and an optional description. 

5. Provide the following information for configuring the adapter:

   XML element	Description	Required
   <url>	Specifies the URL where the Kafka broker is located in a hostname:port or IPAddress:port format. This input element is used to set the connection property called bootstrap.servers in Kafka. Do not specify the bootstrap.servers property in the connection properties input element. For more information, seeApache Kafka documentation..	Conditional; required if not specified during adapter request
   <topic>	Specifies the topic to which the messages are published. If a topic is not created, a new topic will be created when the adapter request is run. This element is required if the URL is provided.	Conditional; required if not specified during adapter request
   <partition>	Specifies the partition number to be used for the topic while publishing messages. This element is required if the URL and topic are provided. If not specified, a default value of 0 is considered and the messages are published without a partition.	Conditional; required if not specified during adapter request
   <connection-properties>	Contains the connection properties required to connect to Kafka. Contains the <key.serializer> and <value.serializer> elements. You can specify other properties within the <connection-properties> element that are supported by the Apache Kafka interface. See For more information, seeApache Kafka documentation..	Conditional; required if not specified during adapter request
   <key.serializer>	Specifies the serializer class for a key that implements the Kafka interface. This is a required element for publishing a message. However, if no value is provided, a default value is provided by the adapter. Default value: org.apache.kafka.common.serialization.StringSerializer	Conditional; required if not specified during adapter request
   <value.serializer>	Specifies the serializer class for a key that implements the Kafka interface. This is a required element for publishing a message. However, if no value is provided, a default value is provided by the adapter. Default value: org.apache.kafka.common.serialization.StringSerializer	Conditional; required if not specified during adapter request

6. (Optional) To specify the adapter configuration in an XML, click Switch to XML view and provide the XML configuration that matches your environment. 

   Sample empty adapter configuration

   <config />

   Warning

   Note

   You can choose not to specify any values in the adapter configuration, (use empty <config/> element) and specify these elements in the adapter request. The input values specified in the adapter request override the values specified in the adapter configuration.

   Sample adapter configuration with required elements only

   <config>
      <url>[host]:[port]</url>
      <topic>adapterTest</topic>
      <connection-properties />
   </config>
   

   Sample adapter configuration with all supported elements

   <config>
      <url>[host]:[port]</url>
      <producer-class-name>org.apache.kafka.clients.producer.KafkaProducer</producer-class-name>
      <topic>adapterTest</topic>
      <partition>0</partition>
      <connection-properties>
          <key.serializer>org.apache.kafka.common.serialization.StringSerializer</key.serializer>
          <value.serializer>org.apache.kafka.common.serialization.StringSerializer</value.serializer>
      </connection-properties>
   </config>
   

   Sample adapter configuration with additional producer configuration property values

   <config>
      <url>[host]:[port]</url>
      <producer-class-name>org.apache.kafka.clients.producer.KafkaProducer</producer-class-name>
      <topic>adapterTest</topic>
      <partition>0</partition>
      <connection-properties>
          <key.serializer>org.apache.kafka.common.serialization.StringSerializer</key.serializer>
          <value.serializer>org.apache.kafka.common.serialization.StringSerializer</value.serializer>
          <acks>all</acks>
          <retries>1</retries>
          <batch.size>16384</batch.size>
          <linger.ms>1</linger.ms>
      </connection-properties>
   </config>
   

To configure the Kafka monitor adapter

The monitor adapter can receive messages from a topic. 

1. On Grid Manager, go to Manage > Adapters.
2. From the Adapters in Repository list, select the bmc-adapter-kafka-monitor adapter, and click Add to Grid to add it in the Adapters on Grid list.
3. Click Configure.
4. Enter a unique name and an optional description. 

5. Provide the following information for configuring the adapter:

   XML element	Description	Required
   <url>	Specifies the value for the bootstrap.servers property in Kafka. This is a list of host and port pairs that are used to establish the initial connection to the Kafka cluster. For more information, seeApache Kafka documentation..	Yes
   <topic>	Specifies the topic to from which the messages are consumed.	Yes
   <wait-for-subscriber-time-seconds>	Specifies the time, in seconds, before the adapter starts polling for a specified topic using the subscriber. If the subscriber is not ready, it will loop back to wait again. Polling for new messages only starts when the subscriber has successfully subscribed to a topic. Default value: 10	No
   <consumer-poll-timeout-ms>	Specifies the time, in milliseconds, for which the consumer waits for before exiting the polling function. However, the adapter goes back to polling in a while loop repeatedly until the adapter is shut down. Default value: 1000	Yes

   <config>
     <url>[host]:[port]</url>
     <topic>adapterTest</topic>
     <wait-for-subscriber-time-seconds>10</wait-for-subscriber-time-seconds>
     <consumer-poll-timeout-ms>100</consumer-poll-timeout-ms>
     <log-file-name>tso-adapter-monitor-kafka.log</log-file-name>
     <log-file-size>4MB</log-file-size>
     <log-file-max-backup-index>10</log-file-max-backup-index>
     <log-file-append>true</log-file-append>
     <log-level>info</log-level>
   </config>