Kafka integration

TBMQ Kafka Integration enables communication with Apache Kafka, allowing TBMQ to publish messages to external Kafka clusters. This is useful for the following scenarios:

• Streaming IoT Data — forwarding device telemetry, logs, or events to Kafka for processing and storage.
• Event-Driven Architectures — publishing messages to Kafka topics for real-time analytics and monitoring.
• Decoupled System Communication — using Kafka as a buffer between TBMQ and downstream applications.

1. Device (client) publishes an MQTT message to a topic matching the integration’s topic filters.
2. TBMQ broker receives the message and forwards it to the TBMQ Integration Executor.
3. Integration Executor processes the message and sends it to a configured Kafka topic.
4. Kafka consumers process the message in downstream systems.

• A running TBMQ instance.
• An external Kafka cluster ready to receive messages (e.g., Confluent Cloud).
• A client capable of publishing MQTT messages (e.g., TBMQ WebSocket Client).

1. Go to the Integrations page and click the ”+” button.
2. Select Kafka as the integration type and click Next.
3. On the Topic Filters step, click Next to subscribe to the default topic tbmq/#.
4. In the Configuration step, enter the Bootstrap servers (Kafka broker addresses). See below for common and Confluent Cloud configurations.
5. Click Add to save the integration.

Navigate to Integrations page and click the plus icon Select Kafka as the integration type and click Next Click Next to subscribe to default topic tbmq/#

• Common / Docker
• Confluent Cloud

Specify the bootstrap server address (e.g., localhost:9092 for a local broker). The screenshot below shows the basic configuration for establishing a connection between TBMQ and a Kafka broker.

Topic filters define MQTT-based subscriptions that trigger the integration. When TBMQ receives a message matching a configured topic filter, the integration processes it and forwards the data to the Kafka cluster.

For example, with the topic filter tbmq/devices/+/status, any of the following messages will trigger the integration:

tbmq/devices/device-01/status
tbmq/devices/gateway-01/status

TBMQ logs integration-related events for debugging and troubleshooting:

• Lifecycle Events — logs events such as Started, Created, Updated, Stopped.
• Statistics — insights into integration performance, including processed message counts and error rates.
• Errors — captures failures related to authentication, timeouts, payload formatting, or connectivity issues.

Lifecycle events for the Kafka integration Statistics events showing processed message counts Error events capturing integration failures

1. Navigate to the WebSocket Client page.
2. Select WebSocket Default Connection (or any working connection) and click Connect. Verify the connection status shows Connected.
3. Set the Topic field to tbmq/kafka-integration to match the integration’s topic filter tbmq/#.
4. Click the Send icon to publish the message.

Select a working connection and click Connect Set topic to tbmq/kafka-integration and click Send In Confluent Cloud open Topics and click on the topic tbmq.messages The message from TBMQ Kafka integration is available in the messages table

If successful, the message should be available in your Kafka service under the topic tbmq.messages:

{
    "payload": "eyJ0ZW1wZXJhdHVyZSI6MjV9",
    "topicName": "tbmq/kafka-integration",
    "clientId": "tbmq_df52bNUQ",
    "eventType": "PUBLISH_MSG",
    "qos": 1,
    "retain": false,
    "tbmqIeNode": "tbmq_ie_node",
    "tbmqNode": "tbmq_node",
    "ts": 1742554969254,
    "props": {},
    "metadata": {
        "integrationName": "Kafka integration"
    }
}

Message field descriptions: