Stand with Ukraine flag
Try it now Pricing
Edge
Community Edition Professional Edition Cloud Edge PE Edge IoT Gateway License Server Trendz Analytics Mobile Application PE Mobile Application MQTT Broker
Installation > On premise > Edge Cluster Setup
Getting Started Documentation Devices Library
Architecture API FAQ
On this page

Edge Cluster Setup

Available since TB Version 4.0

Overview

Edge clustering refers to connecting and grouping multiple Edge nodes to work together at the edge of the network. Clustering is useful for large-scale industrial IoT, smart cities, factories, or any scenario with thousands of devices in one region.

Edge-specific considerations:

  • All Edge nodes connect to the same database.
  • Devices can connect to any node based on load balancing.
  • Edge nodes share the workload and maintain local failover.
  • The Edge Cluster syncs aggregated data to ThingsBoard Cloud.

edge-cluster

For more details, see the microservices architecture page.

Prerequisites

ThingsBoard microservices run in a Dockerized environment. Before starting, make sure that Docker CE and Docker Compose are installed on your system.

Install Docker:

doc warn icon

Don’t forget to add your linux user to the docker group. See Manage Docker as a non-root user.

Step 1. Pull the ThingsBoard Edge image

Log in to Docker Hub and use the command to pull the image:

1

docker pull thingsboard/tb-edge:4.0.1EDGE

Step 2. Clone the ThingsBoard Edge CE repository

1
2

git clone -b release-4.0 https://github.com/thingsboard/thingsboard-edge.git --depth 1
cd thingsboard-edge/docker-edge

Step 3. Configure ThingsBoard Edge database and queue service

Before performing the initial installation, configure the type of database to be used with ThingsBoard Edge. To set the database type, change the value of the DATABASE variable in the environment file (.env) file.

ThingsBoard Edge currently supports two messaging systems/brokers for storing the messages:

  • In Memory queue implementation is not suitable for any sort of cluster deployments.
  • Kafka is recommended for production deployments and used by default. This queue is used on most of the ThingsBoard production environments now.

To edit the ThingsBoard Edge .env file, run the following command:

1

nano .env

Check the following lines:

1
2
3
4
5

DATABASE=postgres
TB_QUEUE_TYPE=kafka

CLOUD_ROUTING_KEY=PUT_YOUR_EDGE_KEY_HERE # e.g. 19ea7ee8-5e6d-e642-4f32-05440a529015
CLOUD_ROUTING_SECRET=PUT_YOUR_EDGE_SECRET_HERE # e.g. bztvkvfqsye7omv9uxlp

View the full .env file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

# redis or redis-cluster or redis-sentinel
CACHE=redis

DOCKER_REPO=thingsboard

TB_EDGE_NODE_DOCKER_NAME=tb-edge-node
TB_EDGE_VERSION=latest

# Database used by ThingsBoard, can be either postgres (PostgreSQL) or hybrid (PostgreSQL for entities database and Cassandra for timeseries database).
# According to the database type corresponding docker service will be deployed (see docker-compose.postgres.yml, docker-compose.hybrid.yml for details).

DATABASE=postgres
TB_QUEUE_TYPE=kafka

CLOUD_ROUTING_KEY=PUT_YOUR_EDGE_KEY_HERE # e.g. 19ea7ee8-5e6d-e642-4f32-05440a529015
CLOUD_ROUTING_SECRET=PUT_YOUR_EDGE_SECRET_HERE # e.g. bztvkvfqsye7omv9uxlp

CLOUD_RPC_HOST=demo.thingsboard.io
CLOUD_RPC_PORT=7070


LOAD_BALANCER_NAME=haproxy-certbot

# If enabled Prometheus and Grafana containers are deployed along with other containers
MONITORING_ENABLED=false

# Limit memory usage for each Java application
JAVA_OPTS="-Xmx2048M -Xms2048M -Xss384k -XX:+AlwaysPreTouch"
  • CACHE: Set the cache type:
    • redis: Set the redis value to use the Redis standalone cache (1 node - 1 master).
    • redis-cluster: Set the redis-cluster value to use the Redis cluster cache (6 nodes - 3 masters, 3 slaves).
    • redis-sentinel: Set the redis-sentinel value to use the Redis sentinel cache (3 nodes - 1 master, 1 slave, 1 sentinel).
  • DATABASE: Replace the database value with:
    • postgres: Set the postgres value to use the PostgreSQL database.
    • hybrid: Set the hybrid value to use the PostgreSQL database for the entities database and Cassandra for the time-series database.
  • TB_QUEUE_TYPE: Use kafka as the queue type. The in-memory value is not valid for a cluster deployment.
  • CLOUD_ROUTING_KEY: Put your edge key.

  • CLOUD_ROUTING_SECRET: Put your edge secret.

  • CLOUD_RPC_HOST: Use demo.thingsboard.io if you connect Edge to the ThingsBoard Demo, or an IP address of the machine with the ThingsBoard Platform.

  • MONITORING_ENABLED: To start cluster monitoring with Grafana and/or Prometheus services, set the variable to true.
Doc info icon

Once deployed, you can reach Prometheus at http://localhost:9090 and Grafana at http://localhost:3000

By default:

  • Login: admin
  • Password: foobar

Step 4. Create and check required host volumes

Execute the following command to create log folders for the services and chown of these folders to the docker container users. To be able to change user, chown command is used, which requires sudo permissions (script will request password for a sudo access):

doc warn icon

For Docker Desktop users on MacOS, that utilize Synchronized file shares feature (enabled by default for /Users subdirectories):

Please note that you need to omit changing host volume ownership, since it is resolved automatically by virtualization engine.

Use --skipChown flag for both create and check scripts:

./docker-create-log-folders.sh --skipChown

./docker-check-log-folders.sh --skipChown

1

./docker-create-log-folders.sh

In order to check if all required volume folders are available and have correct ownership:

1

./docker-check-log-folders.sh

Step 5. Install and run ThingsBoard Edge

To run the installation, execute the following command:

1

./docker-install-tb.sh

To start the service, execute the following command:

1

./docker-start-services.sh
Doc info icon

It will take a few minutes to start the services. Once all services are successfully started, open the ThingsBoard Edge service at http://{your-host-ip} in the browser (e.g., http://localhost). To log in, use the credentials from the ThingsBoard account.

Examine edge service logs for errors in case of any issues. To see ThingsBoard Edge node logs, execute the following command:

1

docker-compose logs -f tb-edge1 tb-edge2 tb-edge3

To see the state of all the containers, use:

1

docker-compose ps

To inspect the logs of all running services, use:

1

docker-compose logs --f

See docker-compose logs command reference for details.

Stop and remove docker containers

To stop the services, use:

1

./docker-stop-services.sh

To stop and completely remove the deployed docker containers, run the command:

1

./docker-remove-services.sh

Upgrading

To update a particular or all services (pull newer docker image and rebuild container):

1

./docker-update-service.sh [SERVICE...]

Where:

  • [SERVICE…]: The list of services to update (defined in docker-compose configurations). If not specified, all services will be updated.

To upgrade the database, run the following commands:

1
2
3

./docker-stop-services.sh
./docker-upgrade-tb.sh
./docker-start-services.sh

Generate certificate for HTTPS

We use HAproxy to proxy traffic to containers, and for Web UI we use 80 and 443 ports by default. To use HTTPS with a valid certificate, run these commands:

1
2

docker exec haproxy-certbot certbot-certonly --domain your_domain --email your_email
docker exec haproxy-certbot haproxy-refresh

Valid certificate will only be used if you visit Web UI by domain in URL. If you are using IP address to access UI, this would use self-signed certificate.

Next steps