Platform Integrations

ThingsBoard PE Feature

Only Professional Edition supports Platform Integrations feature.
Use ThingsBoard Cloud or install your own platform instance.

• How it works?
• Uplink data converter
  • Generic Uplink data converter
  • Typed Uplink data converter
  • Test decoder function
    • Payload
    • Metadata
    • Converter output
• Downlink data converter
  • Converter output
  • Synchronous vs asynchronous downlinks
  • Example
• Converters library
• Debug mode
• Deployment options
  • Embedded integrations
  • Remote integrations
• Platform Integrations vs IoT Gateway
• Feature Roadmap
  • Usage statistics
  • More integrations and protocols
  • More data converters
• See also
• Video tutorial
• Next steps

ThingsBoard Platform integrations feature was designed for two primary use cases / deployment options:

• Connect existing NB IoT, LoRaWAN, SigFox and other devices with specific payload formats directly to ThingsBoard platform.
• Stream data from devices connected to existing IoT Platforms to enable real-time interactive dashboards and efficient data processing.

Both use cases have few things in common. There is a server-side component in the deployment topology that prevents direct access to device and provides set of APIs to interact with the device in the field instead. The payload format of the device is not well-defined. Often two devices that have similar sensors have different payload formats depending on a vendor or even software version.

The job of ThingsBoard Integration is to provide secure and reliable API bridge between core platform features (telemetry collection, attributes and RPC calls) and specific third-party platform APIs.

How it works?

At the moment ThingsBoard supports various integration protocols. Most popular are HTTP, MQTT and OPC-UA. Platform also support integration with specific LoRaWAN Network servers, Sigfox backend, various NB IoT devices using raw UDP and TCP integrations. AWS IoT, IBM Watson and Azure Event Hub allows to subscribe to the data feed from devices via MQTT or AMQP.

The list of platform integrations is constantly growing, however, the general integration concepts are the same and explained below.

Once message arrives from External Platform to ThingsBoard it passes validation according to platform specific payload format and security rules. Once message is validated ThingsBoard Integration invokes assigned Uplink data converter to extract sub-set of meaningful information out of the incoming message. The message is basically transformed from device and platform specific payload to the format that ThingsBoard uses.

Since TB PE v2.0, Rule Engine is also able to push Downlink messages to the integrations. The example of such message may be:

• notification about shared attribute (configuration) update;
• notification about oneway RPC call to trigger some action on the device;
• any custom message from the rule engine.

The most common use cases are:

• changing data upload frequency based on shared attribute value change
• triggering firmware update procedure based on shared attribute value change
• changing device state based on rpc call;

Once message is pushed by the rule engine, ThingsBoard invokes assigned Downlink data converter and transforms the rule engine message to the specific data format that is used by the Integration.

Uplink data converter

The primary function of the Uplink data converter is to parse the payload of incoming messages from devices (e.g., MQTT, HTTP, CoAP, or other protocols) and convert it into a format that ThingsBoard can process.

Starting from ThingsBoard version 4.0, we have simplified the process of writing converters for certain integrations that receive payload messages with the same structure. You can now easily choose where the message fields from the integration should go (attributes or telemetry) without manually defining this in the decoder function.

This feature applies to uplink converters for the following integrations:

• ChirpStack
• Loriot
• The Things Stack Community
• The Things Stack Industries
• ThingPark
• ThingPark Enterprise

Converters created before the release of ThingsBoard 4.0 will still be available and will continue to function properly.

Generic Uplink data converter

To create a generic Uplink data converter suitable for all integration types, follow these steps:

• Navigate to the “Data converters” section in the “Integration center”, click the “plus” icon button, and In the dropdown menu, select “Create new converter”.
• In the new window:
  • Make sure the converter type selector is set to “Uplink”.
  • Select the integration type where this converter will be used (e.g., MQTT, OPC-UA, etc.), or leave it set to “All” if it will be used across multiple integrations.
  • Enter a name for the converter.
  • (Optional) Enable Debug mode if you need to troubleshoot the converter during development or testing.

Debug mode is extremely useful for development and troubleshooting. However, having it on all the time can significantly increase the disk space used by the database since all the debug data is stored there.
Therefore, starting from version 3.9, ThingsBoard stores all debug events for uplink converter only during the first 15 minutes. After that, only failure events are retained. These settings can be combined or completely disabled.

• In the Main decoding configuration section, define the decoding logic: use the provided default script or enter your custom script to parse and transform the incoming data.

One can use either TBEL (ThingsBoard expression language) or JavaScript to develop user defined functions. We recommend utilizing TBEL as it’s execution in ThingsBoard is much more efficient compared to JS.

• Configure advanced decoding parameters:
  • To prevent unnecessary updates for telemetry or attribute values, you can use the “Update only keys list” field. Keys specified in this list will only be updated if their values have changed compared to the previous ones. If a listed key appears in the converted message but its value remains the same, the update will be skipped.

    Please note that in a cluster setup, values, associated with keys specified in the “Update only keys list” field may be updated more than once if a message is received by a different integration executor nodes. The same behavior is expected if the converter configuration has been updated.

• Once everything is set up, click the “Add” button to create and save the new converter.

Typed Uplink data converter

The typed Uplink data converter is designed for use with a specific integration type. To create the Uplink data converter for an integration that supports the new uplink converter functionality (ChirpStack, Loriot, The Things Stack Community, The Things Stack Industries, ThingPark, ThingPark Enterprise), follow these steps:

• The “Integration type” field lists all supported integrations. For this example, I'll use Loriot.
• Specify a name for the converter.
• Go to the “Main decoding configuration” section:
  • Select the entity type (Device or Asset) that will be created as a result of the integration, and specify the entity name.

    The Entity name field is mandatory, so it cannot be left empty. By default, the entity name follows the pattern Device $eui (or Asset $eui). Here you need to underline the pattern $ in the device name. As you type $, a list of available keys from the Loriot message will appear. This list is standardized and does not depend on the specific message content. The $eui pattern will dynamically fetch the device's unique identifier from the Loriot message.

• In the Main decoding configuration section, define the decoding logic: use the provided default script or enter your custom script to parse and transform the incoming data.

One can use either TBEL (ThingsBoard expression language) or JavaScript to develop user defined functions. We recommend utilizing TBEL as it’s execution in ThingsBoard is much more efficient compared to JS.

• Configure advanced decoding parameters:
  • The Device profile, Device label, Customer name, and Device group name fields are not mandatory, and you can also use the $ pattern to populate them dynamically. If the Device profile field is left empty, the device profile will be set to “default”.
  • In the Attributes and Telemetry sections specify the keys that should be interpreted as attributes and telemetry, respectively. If a specified key is not present in the incoming message, it will be ignored by the converter.
  • To prevent unnecessary updates for telemetry or attribute values, you can use the “Update only keys list” field. Keys specified in this list will only be updated if their values have changed compared to the previous ones. If a listed key appears in the converted message but its value remains the same, the update will be skipped.

    Please note that in a cluster setup, values, associated with keys specified in the “Update only keys list” field may be updated more than once if a message is received by a different integration executor nodes. The same behavior is expected if the converter configuration has been updated.

• Once everything is set up, click the “Add” button to create and save the new converter.

Test decoder function

When creating the Uplink data converter, you can validate the decoding logic used to process incoming device data. To get started, click the “Test payload decoder” button. This allows you to simulate how your converter will handle real payloads and fine-tune your decoding function accordingly.

Payload

The Payload - this represents the encoded input data received from your device.

Note: Each supported integration has a pre-filled sample payload. You can use this as a starting point, or modify it to match the specific message format used by your integration.

The format can vary depending on the integration type and may be:

• JSON
• TEXT
• Binary (Base64)

  For the Binary (Base64) payload content type, the payload is automatically decoded into a byte array, so there's no need to decode it manually in your script.

You can access the payload in your decoder script using the payload keyword.

Metadata

The Metadata is a key-value map that contains special fields provided by the integration. You can reference these values in your decoder script using the metadata keyword.

Additionally, you can configure custom metadata parameters for each integration in its settings. For example, you may define a metadata parameter such as deviceType and then use it in your script to automatically assign the appropriate device type when new devices are created.

Converter output

The Converter output displays the result in JSON format returned by your decoder function. This includes device value arrays (such as telemetry or attributes) and may also contain timestamps in telemetry values. This allows you to compare and verify how the decoded data is ultimately interpreted and processed by ThingsBoard.

The only mandatory parameters in the output JSON are deviceName and deviceType.

NOTE: Starting version 2.4.2, ThingsBoard also supports assetName and assetType instead of deviceName and deviceType.

NOTE: Starting version 2.4.2, ThingsBoard also support optional customerName and groupName. Those parameters will cause ThingsBoard to automatically create customer and/or entity group and assign those entities to the customer and/or group.

Example of converter output data:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
{
  "entityType": "DEVICE",
  "name": "Device 1000000000000001",
  "profile": "default",
  "telemetry": [{
    "ts": 1684478801936,
    "values": {
      "battery": 95,
      "temperature": 36.6,
      "saturation": 99
    }
  }, {
    "rssi": -21,
    "data": "01755e030001040001",
    "battery": 94,
    "snr": 10,
    "fСnt": 2
  }],
  "attributes": {
    "serialNumber": "SN-12345678",
    "fPort": 85,
    "dr": "SF9 BW125 4/5",
    "frequency": 867500000,
    "eui": 1000000000000001
  }
}

Downlink data converter

The main function of downlink data converter is to transform the incoming rule engine message and its metadata to the format that is used by corresponding Integration.

To create the Downlink data converter, follow these steps:

• Navigate to the “Data converters” section in the “Integration center”, click the “plus” icon button, and In the dropdown menu, select “Create new converter”.
• In the new window:
  • Switch the converter type selector to “Downlink”.
  • Select the integration type where this converter will be used (e.g., MQTT, OPC-UA, etc.), or leave it set to “All” if it will be used across multiple integrations.
  • Enter a name for the converter.
  • (Optional) Enable Debug mode if you need to troubleshoot the converter during development or testing.

Debug mode is extremely useful for development and troubleshooting. However, having it on all the time can significantly increase the disk space used by the database since all the debug data is stored there.
Therefore, starting from version 3.9, ThingsBoard stores all debug events for downlink converter only during the first 15 minutes. After that, only failure events are retained. These settings can be combined or completely disabled.

• In the Main encoding configuration section, define the decoding logic: enter script for parsing and transforming the output data.

One can use either TBEL (ThingsBoard expression language) or JavaScript to develop user defined functions. We recommend utilizing TBEL as it’s execution in ThingsBoard is much more efficient compared to JS.

• Click the “Add” button to create and save your new converter.

Downlink Converter is basically a user defined function with the following signature:

1
function encoder(msg, metadata, msgType, integrationMetadata);

where

ㅤ* msg - JSON with rule engine msg
ㅤ* metadata - list of key-value pairs with additional data about the message (produced by the rule engine)
ㅤ* msgType - Rule Engine message type. See predefined message types for more details
ㅤ* integrationMetadata - key-value map with some integration specific fields. You can configure additional metadata for each integration in the integration details

Converter output

Converter output should be a valid JSON document with the following structure:

1
2
3
4
5
6
7
{
    "contentType": "JSON",
    "data": "{\"tempFreq\":60,\"firmwareVersion\":\"1.2.3\"}",
    "metadata": {
        "topic": "temp-sensor/sensorA/upload"
    }
}

where

ㅤ* contentType - JSON, TEXT or BINARY (Base64 string) and is specific to your Integration type
ㅤ* data - data string according to the content type
ㅤ* metadata - list of key-value pairs with additional data about the message. For example, topic to use for MQTT integration, etc.

Synchronous vs asynchronous downlinks

Most of the integrations are able to process downlink messages to devices asynchronously. For example, each message pushed by the rule engine to MQTT based integration is immediately pushed to the corresponding external MQTT broker.

However, some integrations, like SigFox or generic HTTP integration are not able to push message asynchroniously. These integrations, due to the nature of underlying HTTP protocol, are only able to push downlink information synchronously in reply to uplink message request. In this case, the last downlink message originated by rule engine will be stored in the queue until the new uplink message arrives for particular device.

Example

Let's assume an example where temperature and humidity upload frequency attributes are updated via ThingsBoard REST API and you would like to push this update to an external MQTT broker (TTN, Mosquitto, AWS IoT, etc.). You may also want to include the “firmwareVersion” attribute value that was configured long time ago and is not present in this particular request. The topic to push the update should contain the device name.

One can use either TBEL (ThingsBoard expression language) or JavaScript to develop user defined functions. We recommend utilizing TBEL as it’s execution in ThingsBoard is much more efficient compared to JS.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
// Encode downlink data from incoming Rule Engine message

// msg - JSON message payload downlink message json
// msgType - type of message, for ex. 'ATTRIBUTES_UPDATED', 'POST_TELEMETRY_REQUEST', etc.
// metadata - list of key-value pairs with additional data about the message
// integrationMetadata - list of key-value pairs with additional data defined in Integration executing this converter

/** Encoder **/

var data = {};

// Process data from incoming message and metadata

data.tempFreq = msg.temperatureUploadFrequency;
data.humFreq = msg.humidityUploadFrequency;

data.firmwareVersion = metadata['ss_firmwareVersion'];

// Result object with encoded downlink payload
var result = {

    // downlink data content type: JSON, TEXT or BINARY (base64 format)
    contentType: "JSON",

    // downlink data
    data: JSON.stringify(data),

    // Optional metadata object presented in key/value format
    metadata: {
        topic: metadata['deviceType']+'/'+metadata['deviceName']+'/upload'
    }

};

return result;

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
// Encode downlink data from incoming Rule Engine message

// msg - JSON message payload downlink message json
// msgType - type of message, for ex. 'ATTRIBUTES_UPDATED', 'POST_TELEMETRY_REQUEST', etc.
// metadata - list of key-value pairs with additional data about the message
// integrationMetadata - list of key-value pairs with additional data defined in Integration executing this converter

/** Encoder **/

var data = {};

// Process data from incoming message and metadata

data.tempFreq = msg.temperatureUploadFrequency;
data.humFreq = msg.humidityUploadFrequency;

data.firmwareVersion = metadata['ss_firmwareVersion'];

// Result object with encoded downlink payload
var result = {

    // downlink data content type: JSON, TEXT or BINARY (base64 format)
    contentType: "JSON",

    // downlink data
    data: JSON.stringify(data),

    // Optional metadata object presented in key/value format
    metadata: {
        topic: metadata['deviceType']+'/'+metadata['deviceName']+'/upload'
    }

};

return result;

In order to invoke the downlink processing by the integration, tenant administrator should configure the rule chain similar to the one below:

The full rule chain configuration is available here.

Converters library

ThingsBoard Converters library is a built-in collection of ready-to-use uplink decoder functions for over 100 devices, supporting six popular LoRaWAN network servers.
It significantly simplifies integration setup with various sensors and vendors.

Currently, the converters library is supported by the following integrations: ChirpStack, Loriot, The Things Stack Community, The Things Stack Industries, ThingPark, and ThingPark Enterprise.
In the future, the list of integrations that support the converters library will be expanded.

To access the Converters library:

• Navigate to the “Integrations” page under the “Integrations center” section. Click the “plus” icon button to start creating a new integration.
  • Select one of the integrations that currently support the Converters library.
  • Click “Next” to proceed.
• In the “Uplink data converter” tab:
  • Switch to the “Library” option.
  • From the dropdown menu, choose the device vendor.
  • Then select the specific sensor model you are using. The corresponding decoder function will be automatically loaded into the editor.
  • If needed, configure advanced decoding options.
  • Click “Next”.
• Continue the process to finalize the integration setup.

The Converters library is open-source and actively maintained by the ThingsBoard team. It is hosted on GitHub at the following link.

Debug mode

Debug mode is extremely useful for development and troubleshooting. However, having it on all the time can significantly increase the disk space used by the database since all the debug data is stored there. Therefore, starting from version 3.9, ThingsBoard stores all debug events only during the first 15 minutes. After that, only failure events are retained. These settings can be combined or completely disabled.

Deployment options

ThingsBoard Integration has two deployment options: embedded and remote. See details and architecture diagrams below.

Embedded integrations

Embedded integration is running in the main ThingsBoard server process. Basically it is part of a monolith deployment scenario.

Pros:

• simplifies deployment of new integration (just few clicks on ThingsBoard UI);
• minimize latency for message delivery;

Cons:

• consume resources allocated to main ThingsBoard process: network connections, OS threads and CPU cycles;
• low level of isolation;
• can't access local MQTT brokers or OPC-UA servers if ThingsBoard is deployed in the cloud.

Remote integrations

Remote integration become available since ThingsBoard PE v2.4.1 and enables new deployment scenario. One can install remote integration in the local network and stream data to the cloud.

Let's assume you have local MQTT broker or OPC-UA server deployed on-premises. Those brokers and/or servers don't have dedicated external IP address, so ThingsBoard instance in the cloud can't connect to them directly. However, you can install remote integration close to this server, in the same local network. This integration will connect to the broker/server, pull the data and store it in the local file system. Remote integration will stream the data to the ThingsBoard instance deployed in the cloud once the internet connection is available.

Pros:

• enables integration with servers deployed in the local network;
• isolates the integration process from main ThingsBoard process;

Cons:

• requires installation of a separate package;

Learn how to configure integration to run remotely using this guide.

Platform Integrations vs IoT Gateway

Experienced ThingsBoard users may notice that functionality of Integrations feature partially overlap with functionality of IoT Gateway. However, there are key differences between these two systems/features:

• IoT Gateway is designed for local network deployments, Integrations are designed for server-to-server integrations.
• IoT Gateway is designed to support < 1000 devices, while Integrations are designed for high throughput, scalability and cluster deployments as part of ThingsBoard server.
• Gateway recompilation and restart is required to add custom payload decoder while Integration Converter is a JS function that may be modified in real time.

As you can see, both systems are important and applicable in different use cases.

Feature Roadmap

Usage statistics

We plan to log statistics for amount of messages processed by each integration with possible limitations of messages processed on a tenant / system levels.

More integrations and protocols

We plan to provide specific integrations for different platforms, and also for different communication protocols, like gRPC.

More data converters

We plan to collect and maintain data converters for most popular devices on the market to simplify integration path even more. Please note that you can share your converters with community and send them to us to make part of official ThingsBoard distributive.

Contact us to suggest missing feature for your use case.

See also

Explore guides and video tutorials related to specific integrations:

• HTTP
• MQTT
• AWS IoT
• AWS Kinesis
• IBM Watson IoT
• Azure Event Hub
• Azure IoT Hub
• Actility ThingPark
• TheThingsIndustries
• SigFox
• OceanConnect
• TheThingsStack
• OPC-UA
• TCP
• UDP
• Custom
• LORIOT
• Kafka
• ChirpStack
• CoAP

Video tutorial

See the video tutorial below for step-by-step instructions on how to set up an Uplink data converter.

Next steps

• Getting started guides - These guides provide quick overview of main ThingsBoard features. Designed to be completed in 15-30 minutes.

• Data visualization - These guides contain instructions on how to configure complex ThingsBoard dashboards.

• Data processing & actions - Learn how to use ThingsBoard Rule Engine.

• IoT Data analytics - Learn how to use rule engine to perform basic analytics tasks.

• Hardware samples - Learn how to connect various hardware platforms to ThingsBoard.

• Advanced features - Learn about advanced ThingsBoard features.