Apache Kafka Connector¶

Summary¶

The Apache Kafka connector establishes access to Apache Kafka.

The Apache Kafka connector provides an interface for creating an Apache Kafka connection, the foundation used for generating instances of Apache Kafka activities. These activities, once configured, interact with Apache Kafka through the connection.

The Apache Kafka connector is accessed from the design component palette's Connections tab (see Design Component Palette).

Connector Overview¶

This connector is used to first configure an Apache Kafka connection. Activity types associated with that connection are then used to create instances of activities that are intended to be used as sources (to provide data in an operation) or targets (to consume data in an operation).

Together, a specific Apache Kafka connection and its activities are referred to as an Apache Kafka endpoint:

Apache Kafka activity types

Send Message: Sends streams of data into a topic in an Apache Kafka cluster and is intended to be used as a target in an operation.
Get Messages: Retrieves streams of data from a topic in an Apache Kafka cluster and is intended to be used as a source in an operation.

Note

This connector is a Connector SDK-based connector, which may be referred to by Jitterbit when communicating changes made to connectors built with the Connector SDK.

Prerequisites and Supported API Versions¶

The Apache Kafka connector requires the use of an agent version 10.1 or later. These agent versions automatically download the latest version of the connector when required.

The Apache Kafka connector uses the Apache Kafka Core API. Refer to the API documentation for information on the schema nodes and fields.

Troubleshooting¶

If you experience issues with the Apache Kafka connector, these troubleshooting steps are recommended:

Click the Test button in the connection configuration to ensure the connection is successful and to ensure the latest version of the connector is downloaded to the agent (unless using the Disable Auto Connector Update organization policy).
Check the operation logs for any information written during execution of the operation.
Enable operation debug logging (for Cloud Agents or for Private Agents) to generate additional log files and data.
If using Private Agents, you can enable connector verbose logging for this connector using this specific configuration entry of logger name and level:
```
<logger name="org.jitterbit.connector.apache.kafka" level="DEBUG"/>
```
If using Private Agents, you can check the agent logs for more information.