Stand with Ukraine flag
Try it now Pricing
Community Edition
Community Edition Professional Edition Cloud Edge PE Edge IoT Gateway License Server Trendz Analytics Mobile Application PE Mobile Application MQTT Broker
Getting Started Documentation Devices Library Guides Installation Architecture API FAQ
On this page

Cluster setup with Docker Compose

This guide will help you to setup ThingsBoard in cluster mode using Docker Compose tool.

Prerequisites

ThingsBoard Microservices are running in dockerized environment. Before starting please make sure Docker Engine and Docker Compose are installed in your system.

Install Docker:

doc warn icon

Don’t forget to add your linux user to the docker group. See Manage Docker as a non-root user.

Step 1. Pull ThingsBoard CE Images

Make sure your have logged in to docker hub using command line.

1
2
3
4
5
6
7
8

docker pull thingsboard/tb-node:4.0.1
docker pull thingsboard/tb-web-ui:4.0.1
docker pull thingsboard/tb-js-executor:4.0.1
docker pull thingsboard/tb-http-transport:4.0.1
docker pull thingsboard/tb-mqtt-transport:4.0.1
docker pull thingsboard/tb-coap-transport:4.0.1
docker pull thingsboard/tb-lwm2m-transport:4.0.1
docker pull thingsboard/tb-snmp-transport:4.0.1

Step 2. Review the architecture page

Starting ThingsBoard v2.2, it is possible to install ThingsBoard cluster using new microservices architecture and docker containers. See microservices architecture page for more details.

Step 3. Clone ThingsBoard CE repository

1
2

git clone -b release-4.0 https://github.com/thingsboard/thingsboard.git --depth 1
cd thingsboard/docker

Step 4. Configure ThingsBoard database

Before performing initial installation you can configure the type of database to be used with ThingsBoard. In order to set database type change the value of DATABASE variable in .env file:

1

nano .env

to one of the following:

  • postgres - use PostgreSQL database;
  • hybrid - use PostgreSQL for entities database and Cassandra for time series database;

NOTE: According to the database type corresponding docker service will be deployed (see docker-compose.postgres.yml, docker-compose.hybrid.yml for details).

Step 5. Choose ThingsBoard queue service

ThingsBoard currently supports two messaging systems/brokers for storing the messages: In-memory and Kafka-based systems.

In Memory queue implementation is not suitable for any sort of cluster deployments.

  • Kafka is recommended for production deployments and used by default. This queue is used on the most of ThingsBoard production environments now. It is useful for both on-prem and private cloud deployments. It is also useful if you like to stay independent from your cloud provider. However, some providers also have managed services for Kafka. See AWS MSK for example.

  • Confluent Cloud is a fully managed streaming platform based on Kafka. Useful for a cloud agnostic deployments.

See corresponding architecture page and rule engine page for more details.

Kafka (default, recommended for on-prem, production installations)
Confluent Cloud (Event Streaming Platform based on Kafka)

Apache Kafka is an open-source stream-processing software platform.

Configure ThingsBoard environment file:

1

nano .env

Check following line:

1

TB_QUEUE_TYPE=kafka

Confluent Cloud Configuration

To access Confluent Cloud you should first create an account, then create a Kafka cluster and get your API Key.

Configure ThingsBoard environment file:

1

nano .env

Check following line:

1

TB_QUEUE_TYPE=confluent

Configure Confluent Cloud environment file for ThingsBoard queue service:

1

nano queue-confluent-cloud.env

Don’t forget to replace “CLUSTER_API_KEY”, “CLUSTER_API_SECRET” and “confluent.cloud:9092” with your real Confluent Cloud bootstrap servers:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

TB_QUEUE_TYPE=kafka

TB_KAFKA_SERVERS=confluent.cloud:9092
TB_QUEUE_KAFKA_REPLICATION_FACTOR=3

TB_QUEUE_KAFKA_USE_CONFLUENT_CLOUD=true
TB_QUEUE_KAFKA_CONFLUENT_SSL_ALGORITHM=https
TB_QUEUE_KAFKA_CONFLUENT_SASL_MECHANISM=PLAIN
TB_QUEUE_KAFKA_CONFLUENT_SASL_JAAS_CONFIG='org.apache.kafka.common.security.plain.PlainLoginModule required username="CLUSTER_API_KEY" password="CLUSTER_API_SECRET";'
TB_QUEUE_KAFKA_CONFLUENT_SECURITY_PROTOCOL=SASL_SSL
TB_QUEUE_KAFKA_CONFLUENT_USERNAME=CLUSTER_API_KEY
TB_QUEUE_KAFKA_CONFLUENT_PASSWORD=CLUSTER_API_SECRET

TB_QUEUE_KAFKA_RE_TOPIC_PROPERTIES=retention.ms:604800000;segment.bytes:52428800;retention.bytes:1048576000
TB_QUEUE_KAFKA_CORE_TOPIC_PROPERTIES=retention.ms:604800000;segment.bytes:52428800;retention.bytes:1048576000
TB_QUEUE_KAFKA_TA_TOPIC_PROPERTIES=retention.ms:604800000;segment.bytes:52428800;retention.bytes:1048576000
TB_QUEUE_KAFKA_NOTIFICATIONS_TOPIC_PROPERTIES=retention.ms:604800000;segment.bytes:52428800;retention.bytes:1048576000
TB_QUEUE_KAFKA_JE_TOPIC_PROPERTIES=retention.ms:604800000;segment.bytes:52428800;retention.bytes:104857600

# These params affect the number of requests per second from each partitions per each queue.
# Number of requests to particular Message Queue is calculated based on the formula:
# ((Number of Rule Engine and Core Queues) * (Number of partitions per Queue) + (Number of transport queues)
#  + (Number of microservices) + (Number of JS executors)) * 1000 / POLL_INTERVAL_MS
# For example, number of requests based on default parameters is:

# Rule Engine queues:
# Main 10 partitions + HighPriority 10 partitions + SequentialByOriginator 10 partitions = 30
# Core queue 10 partitions
# Transport request Queue + response Queue = 2
# Rule Engine Transport notifications Queue + Core Transport notifications Queue = 2
# Total = 44
# Number of requests per second = 44 * 1000 / 25 = 1760 requests

# Based on the use case, you can compromise latency and decrease number of partitions/requests to the queue, if the message load is low.
# By UI set the parameters - interval (1000) and partitions (1) for Rule Engine queues.
# Sample parameters to fit into 10 requests per second on a "monolith" deployment: 

TB_QUEUE_CORE_POLL_INTERVAL_MS=1000
TB_QUEUE_CORE_PARTITIONS=2
TB_QUEUE_RULE_ENGINE_POLL_INTERVAL_MS=1000
TB_QUEUE_TRANSPORT_REQUEST_POLL_INTERVAL_MS=1000
TB_QUEUE_TRANSPORT_RESPONSE_POLL_INTERVAL_MS=1000
TB_QUEUE_TRANSPORT_NOTIFICATIONS_POLL_INTERVAL_MS=1000
TB_QUEUE_VC_INTERVAL_MS=1000
TB_QUEUE_VC_PARTITIONS=1

You can update default Rule Engine queues configuration using UI. More about ThingsBoard Rule Engine queues see in documentation.

Step 6. Enable monitoring (optional)

In order to start cluster monitoring - Grafana and Prometheus services, please edit configuration file:

1

nano .env

You’ll need to make sure that MONITORING_ENABLED variable set to true:

1

MONITORING_ENABLED=true

After deployment, you will be able to reach Prometheus at http://localhost:9090 and Grafana at http://localhost:3000 (default login is admin and password foobar).

Step 7. Running

Create and check required host volumes

Execute the following command to create log folders for the services and chown of these folders to the docker container users. To be able to change user, chown command is used, which requires sudo permissions (script will request password for a sudo access):

doc warn icon

For Docker Desktop users on MacOS, that utilize Synchronized file shares feature (enabled by default for /Users subdirectories):

Please note that you need to omit changing host volume ownership, since it is resolved automatically by virtualization engine.

Use --skipChown flag for both create and check scripts:

./docker-create-log-folders.sh --skipChown

./docker-check-log-folders.sh --skipChown

1

./docker-create-log-folders.sh

In order to check if all required volume folders are available and have correct ownership:

1

./docker-check-log-folders.sh

Install and run ThingsBoard

Execute the following command to run installation:

1

./docker-install-tb.sh --loadDemo

Where:

  • --loadDemo - optional argument. Whether to load additional demo data.

Execute the following command to start services:

1

./docker-start-services.sh

After a while when all services will be successfully started you can open http://{your-host-ip} in you browser (for ex. http://localhost). You should see ThingsBoard login page.

Use the following default credentials:

If you installed DataBase with demo data (using --loadDemo flag) you can also use the following credentials:

In case of any issues you can examine service logs for errors. For example to see ThingsBoard node logs execute the following command:

1

docker compose logs -f tb-core1 tb-rule-engine1
Doc info icon

If you still rely on Docker Compose as docker-compose (with a hyphen) execute next command:

docker-compose logs -f tb-core1 tb-rule-engine1

Or use the following command to see the state of all the containers:

1

docker compose ps
Doc info icon

If you still rely on Docker Compose as docker-compose (with a hyphen) execute next command:

docker-compose ps

Use the following command to inspect the logs of all running services:

1

docker compose logs -f
Doc info icon

If you still rely on Docker Compose as docker-compose (with a hyphen) execute next command:

docker-compose logs -f

See docker-compose logs command reference for details.

Execute the following command to stop services:

1

./docker-stop-services.sh

Execute the following command to stop and completely remove deployed docker containers:

1

./docker-remove-services.sh

Execute the following command to update particular or all services (pull newer docker image and rebuild container):

1

./docker-update-service.sh [SERVICE...]

Where:

  • [SERVICE...] - list of services to update (defined in docker-compose configurations). If not specified all services will be updated.

Upgrading

If you update your volumes, or merge your deploy files with GitHub repository files, do not forget to inspect your host volumes - like log folders or etc - using provided check and (if needed) create scripts - they are explained at the beginning of “Running” step.

In case when database upgrade is needed, edit .env file to set “TB_VERSION” to target version (e.g. set it to 4.0.1 if you are upgrading to the latest). Then, execute the following commands:

1
2
3

./docker-stop-services.sh
./docker-upgrade-tb.sh --fromVersion=[FROM_VERSION]
./docker-start-services.sh

Where FROM_VERSION - from which version upgrade should be started. See Upgrade Instructions for valid fromVersion values. Note, that you have to upgrade versions one by one (for example 3.6.1 -> 3.6.2 -> 3.6.3 etc).

Generate certificate for HTTPS

We using HAproxy for proxying traffic to containers and for web UI by default we using 80 and 443 ports. For using HTTPS with a valid certificate, execute these commands:

1
2

docker exec haproxy-certbot certbot-certonly --domain your_domain --email your_email
docker exec haproxy-certbot haproxy-refresh

NOTE: Valid certificate used only, when you visit web UI by domain in URL. If you using IP address for access to UI, this would be used self-signed certificate.

Next steps