Send IoT Device Data to Kafka

Who it’s for

Platform and DevOps engineers asking “how do I make device telemetry available to my Kafka consumers?”, “how do I feed analytics, ML, or data lake pipelines from ThingsBoard without polling?”, “how do I bridge IoT events into our existing event-driven architecture?” — when the goal is to forward device telemetry into an existing Kafka topic as it arrives.

What it does

Extends the default ThingsBoard root chain with the Kafka-publishing flow on the Post telemetry path — an enrichment node that tags each message with device identity, feeding an action node that publishes it to Kafka. The record value is the message data, the partition key is deviceId for per-device ordering, and message metadata fields become prefixed headers.

• Fetch Device Info writes deviceName, deviceType, and deviceId into the message data, so each Kafka record carries the identity of the device it came from — enabling consumers to filter, group, and correlate telemetry by device.
• Send to Kafka transforms the message into a Kafka record and publishes it.

Requirements

• Apache Kafka cluster is reachable from the ThingsBoard server
• Kafka topic created or auto-creation enabled on the broker side

How to set up

1. Click Install to open the install dialog. The dialog offers two paths:

   • Bind to a profile — enable Set as profile default rule chain and pick the target device profile. The imported chain becomes that profile’s default rule chain.
   • Install only — leave the toggle off. After install, either set it as the root — it already bundles the default routing plus the Kafka-publishing flow — or extract the Kafka-publishing flow into a nested rule chain and connect it to the Post telemetry output of Message Type Switch.

2. Open the Send to Kafka node and set:

   • Bootstrap servers — all broker addresses (e.g., kafka-1:9092,kafka-2:9092,kafka-3:9092). Default: localhost:9092.
   • Topic pattern — target topic. Default: iot.telemetry.

3. The chain runs on post telemetry uploads. Use Check connectivity on any device whose telemetry routes through the installed chain to confirm the record arrives in your Kafka topic.

How to customize

• To route different device types to different topics — use templatization in Topic pattern on the Send to Kafka node (e.g. iot.telemetry.$[deviceType]). Kafka topic names accept only alphanumerics, dots, dashes, and underscores — add a Script node upfront to sanitize values when device types contain other characters.
• To turn off the metadata headers — disable Add Message metadata key-value pairs to Kafka record headers on the Send to Kafka node. Headers are prefixed tb_msg_md_ and let consumers route or filter without parsing the record value.
• To tune producer reliability and throughput — adjust the Producer settings on the Send to Kafka node (retries, batch size, linger time, buffer memory, acks). Defaults work for most deployments. See the Kafka producer settings for what each field controls.
• To enable compression, SSL/TLS, or any other Kafka producer property — add a key-value entry under Other properties in the Advanced section of the Send to Kafka node (e.g. compression.type → lz4). Property names and values follow the standard Apache Kafka producer configuration.
• To drop unwanted messages before they reach Kafka — add a Filter node before Send to Kafka (e.g. a filter script on data or metadata fields).
• To add more context to the Kafka record — add an Enrichment node before Send to Kafka (e.g. customer, tenant, asset, or related-entity data).