Data Filtering and Traffic Reduction

This recipe shows how to reduce the traffic between an Edge and the ThingsBoard server. An in-vehicle monitoring device sends readings from 10 sensors to the Edge: distance, gas consumption, vehicle speed, engine temperature, ambient temperature, tire temperature, and tire pressure for four tires. The Edge stores all readings locally. A script in the Edge root rule chain forwards only the distance readings to the server, where a dashboard shows the distance traveled.

• Completed the Getting Started guide.
• Familiar with Edge rule chain templates.
• ThingsBoard server is running and ThingsBoard Edge is connected to it.

Create a device named In-vehicle monitoring system on the Edge.

1. Log in to the Edge and go to Devices & assets > Devices.
2. Click + in the top-right corner, then click Add new device.
3. Enter the name In-vehicle monitoring system and click Add.

The device appears in the devices table. The Edge also sends it to the server: on the server, go to Devices & assets > Devices and check that In-vehicle monitoring system is in the list.

The Edge receives its root rule chain from the server as a rule chain template. You edit the Edge Root Rule Chain template on the server, and the server sends the updated chain to every Edge that uses it.

In the template, telemetry goes from the Save Timeseries node straight to the Push to cloud node. You add a script transformation node between them. The script replaces the message with a new one that contains only the distance field:

var newMsg = {};
newMsg.distance = msg.distance;
return { msg: newMsg, metadata: metadata, msgType: msgType };

1. Log in to the ThingsBoard server. Go to Edge management > Templates, open the Rule chain tab, and open Edge Root Rule Chain.
2. In the node search, type script. Drag the script node (Transformation section) onto the canvas.
3. Enter the name Transform Incoming Message, replace the default code with the script above, and click Add.
4. Select the Success link between Save Timeseries and Push to cloud and click X to delete it. Leave the link from Save Client Attributes to Push to cloud as it is: it is not part of this change.
5. Drag a link from Save Timeseries to Transform Incoming Message. Select Success and click Add.
6. Drag a link from Transform Incoming Message to Push to cloud. Select Success and click Add.
7. Click Apply changes.

After the change, telemetry takes this path:

To check that the Edge received the update, log in to the Edge, go to Data processing > Rule chains, and open Edge Root Rule Chain. It shows the new node and links.

Get the device access token from the Edge, then run the telemetry generator script.

1. In the Edge UI, go to Devices & assets > Devices.
2. Click In-vehicle monitoring system to open the device details.
3. Click Copy access token. The token is copied to your clipboard.

Install the MQTT Python library:

pip install paho-mqtt

Download the telemetry generator script, update the three placeholders at the top, then run it:

import paho.mqtt.client as mqtt
import json
import random
import time

ACCESS_TOKEN = YOUR_ACCESS_TOKEN # REPLACE with your device access token, e.g. 'edge_vehicle'
THINGSBOARD_EDGE_HOST = YOUR_TB_EDGE_HOST # REPLACE with your ThingsBoard Edge host, e.g. 'localhost'
THINGSBOARD_EDGE_MQTT_PORT = YOUR_TB_EDGE_MQTT_PORT # REPLACE with your Edge MQTT port, e.g. 11883 or 1883
DISTANCE = 0

def on_connect(client, userdata, rc, *extra_params):
    client.subscribe('v1/devices/me/rpc/request/+')
    client.publish('v1/devices/me/telemetry', get_telemetry(), 1)
    print(get_telemetry())

def get_telemetry():
    return json.dumps(device)

while(True):
    device = {
        'distance': DISTANCE,
        'gas_consumption': random.randint(10, 15),
        'vehicle_speed': random.randint(60, 70),
        'engine_temperature': random.randint(195, 220),
        'ambient_temperature': random.randint(65, 80),
        'tire_temperature': random.randint(90, 110),
        'tire_pressure_front_left': random.randint(30, 35),
        'tire_pressure_front_right': random.randint(30, 35),
        'tire_pressure_rear_left': random.randint(30, 35),
        'tire_pressure_rear_right': random.randint(30, 35),
    }
    DISTANCE = DISTANCE + random.randint(0, 100)

    client = mqtt.Client()
    client.on_connect = on_connect
    client.username_pw_set(ACCESS_TOKEN)
    client.connect(THINGSBOARD_EDGE_HOST, THINGSBOARD_EDGE_MQTT_PORT, 60)
    try:
        time.sleep(5)
        client.loop()
    except KeyboardInterrupt:
        client.loop_stop()
        print("Exited!")

python mqtt-generator.py

The script sends a new set of readings every 5 seconds.

Open the device on both sides and compare the Latest telemetry tab:

If the server shows all 10 keys, the Edge is still using the old rule chain. Check that the template changes were applied and that the Edge received them (see the previous section).

Create a dashboard on the ThingsBoard server with a gauge that shows the distance traveled. For details on dashboards, aliases, and widgets, see Dashboards.

1. Log in to the ThingsBoard server and go to Monitor > Dashboards.
2. Click +, select Create new dashboard, enter the title Edge Vehicle, and click Add.
3. Open the Edge Vehicle dashboard and click Edit mode.
4. Open Entity aliases, click Add alias, fill in the alias settings from the table below, and click Save.
5. Click Add widget, open the Digital gauges bundle, and select Digital speedometer. In the widget dialog, switch from Basic to Advanced.
6. On the Data tab, fill in the datasource settings from the table below.
7. Open the Appearance tab, set the appearance settings from the table below, and click Add.
8. Click Save to save the dashboard.

The gauge shows the latest distance value from the device. The value grows as the script runs.

You have sent telemetry from the device, saved all sensor readings on the Edge, pushed only the filtered distance data to the server, and visualized it on a dashboard.