- Introduction
- Create device on ThingsBoard
- Install required libraries and tools
- Connect device to ThingsBoard
- Check data on ThingsBoard
- Synchronize device state using client and shared attribute requests
- Control device using shared attributes
- Control device using RPC
- Conclusion
Introduction
ESP-EYE is a development board for image recognition and audio processing, which can be used in various AIoT applications.
It features an ESP32 chip, a 2-Megapixel camera and a microphone. ESP-EYE offers plenty of storage, with an 8 Mbyte PSRAM and a 4 Mbyte flash.
It also supports image transmission via Wi-Fi and debugging through a Micro-USB port.
In this guide, we will learn how to create device on Thingsboard, install required libraries and tools.
After this we will modify our code and upload it to the device, and check the results of our coding and check data on ThingsBoard using imported dashboard.
Our device will synchronize with ThingsBoard using client and shared attributes requests functionality.
Of course, we will control our device using provided functionality like shared attributes or RPC requests.
Prerequisites
To continue with this guide we will need the following:
- ESP-EYE
- ThingsBoard account
Create device on ThingsBoard
For simplicity, we will provide the device manually using the UI.
- Login to your ThingsBoard instance and navigate to the "Entities". Then click the "Devices" page.
- Click on the "+" icon in the top right corner of the table and then select "Add new device".
- Input device name. For example, "My Device". No other changes required at this time. Click "Add" to add the device.
- Your device has been added.
Install required libraries and tools
Install the board for Arduino IDE: Go to File > Preferences and add the following URL to the Additional Boards Manager URLs field.
1
https://dl.espressif.com/dl/package_esp32_index.json
Then go to Tools > Board > Board Manager and install the ESP32 by Espressif Systems board.
After the installation is complete, select the board by Board menu:
Tools > Board > M5Stack > M5Stack-Timer-CAM.
Also, do not forget to specify the port for the device:
Tools > Port > /dev/ttyUSB0.
Port depends on operation system and may be different:
- for Linux/MacOS it will be /dev/ttyUSBX
- for Windows - COMX.
Where X - some number, that was assigned by your system.
To install ThingsBoard Arduino SDK - we will need to do the following steps:
- Go to "Tools" tab and click on "Manage libraries".
- Put "ThingsBoard" into the search box and press "INSTALL" button for the found library.
At this point, we have installed all required libraries and tools.
Connect device to ThingsBoard
To connect the device you need to get the device credentials first. ThingsBoard supports various device credentials. We recommend using default auto-generated credentials, which is an access token for this guide.
- Click on the device row in the table to open device details.
- Click "Copy access token". The token will be copied to your clipboard. Please save it in a safe place.
Now it’s time to program the board to connect to ThingsBoard.
To do this, you can use the code below. It contains all required functionality for this guide.
In the code, replace placeholders with your WiFi network SSID, password, ThingsBoard device access token.
Necessary variables for connection:
| Variable name | Default value | Description |
|---|---|---|
| WIFI_SSID | YOUR_WIFI_SSID | Your WiFi network name. |
| WIFI_PASSWORD | YOUR_WIFI_PASSWORD | Your WiFi network password. |
| TOKEN | YOUR_DEVICE_ACCESS_TOKEN | Access token from device. Obtaining process described in #connect-device-to-thingsboard |
| THINGSBOARD_SERVER | demo.thingsboard.io | Your ThingsBoard host or ip address. |
| THINGSBOARD_PORT | 1883U | ThingsBoard server MQTT port. Can be default for this guide. |
| MAX_MESSAGE_SIZE | 256U | Maximal size of MQTT messages. Can be default for this guide. |
| SERIAL_DEBUG_BAUD | 1883U | Baud rate for serial port. Can be default for this guide. |
1
2
3
4
5
6
7
8
9
10
11
12
13
14
...
constexpr char WIFI_SSID[] = "YOUR_WIFI_SSID";
constexpr char WIFI_PASSWORD[] = "YOUR_WIFI_PASSWORD";
constexpr char TOKEN[] = "YOUR_ACCESS_TOKEN";
constexpr char THINGSBOARD_SERVER[] = "demo.thingsboard.io";
constexpr uint16_t THINGSBOARD_PORT = 1883U;
constexpr uint32_t MAX_MESSAGE_SIZE = 256U;
constexpr uint32_t SERIAL_DEBUG_BAUD = 115200U;
...
Send data part (By default the example sends random value for temperature key and some WiFi information):
1
2
3
4
5
6
7
8
...
tb.sendTelemetryInt("temperature", random(10, 20));
tb.sendAttributeInt("rssi", WiFi.RSSI());
tb.sendAttributeString("bssid", WiFi.BSSIDstr().c_str());
tb.sendAttributeString("localIp", WiFi.localIP().toString().c_str());
tb.sendAttributeString("ssid", WiFi.SSID().c_str());
tb.sendAttributeInt("channel", WiFi.channel());
...
Then upload the code to the device by pressing Upload button or keyboard combination Ctrl+U.
Check data on ThingsBoard
To check the data and get ability to send commands or data to the device we will create the dashboard.
First download the Check and control device data dashboard file.
To add the dashboard to ThingsBoard, we need to import it, and to do this, we have to go through the following steps:
- Go to Dashboards through the main menu on the left side of the screen.
- Click on the “+” button in the upper right corner of the page and choose "Import dashboard".
- Select your dashboard.json file and press import button.
- Now you can see imported dashboard in the table.
After importing we should choose entity alias for our device.
To do this - we need to press the pen icon and select entity aliases, select alias “My device” and open it for editing by pressing the pen icon.
Then choose a device with name My device from dropdown list and save entity alias, now you should be able to see the data from the device.
To check the data from our device we need to open imported dashboard:
- Open dashboard by clicking on it in the table.
- The view of checking data and control our device dashboard.
- Received attributes from device.
- Device information from the ThingsBoard server.
- Widget to see the history of led mode changes.
- Widget to see the history of our emulated temperature.
Synchronize device state using client and shared attribute requests
In order to get the state of the device from ThingsBoard during booting we have functionality to do this in the code.
Responsible parts of the code example:
- Attribute callbacks:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
...
void processSharedAttributes(const Shared_Attribute_Data &data) {
for (auto it = data.begin(); it != data.end(); ++it) {
if (strcmp(it->key().c_str(), BLINKING_INTERVAL_ATTR) == 0) {
const uint16_t new_interval = it->value().as<uint16_t>();
if (new_interval >= BLINKING_INTERVAL_MS_MIN && new_interval <= BLINKING_INTERVAL_MS_MAX) {
blinkingInterval = new_interval;
Serial.print("Updated blinking interval to: ");
Serial.println(new_interval);
}
} else if(strcmp(it->key().c_str(), LED_STATE_ATTR) == 0) {
ledState = it->value().as<bool>();
digitalWrite(LED_BUILTIN, ledState ? HIGH : LOW);
Serial.print("Updated state to: ");
Serial.println(ledState);
}
}
attributesChanged = true;
}
void processClientAttributes(const Shared_Attribute_Data &data) {
for (auto it = data.begin(); it != data.end(); ++it) {
if (strcmp(it->key().c_str(), LED_MODE_ATTR) == 0) {
const uint16_t new_mode = it->value().as<uint16_t>();
ledMode = new_mode;
}
}
}
...
const Attribute_Request_Callback attribute_shared_request_callback(SHARED_ATTRIBUTES_LIST.cbegin(), SHARED_ATTRIBUTES_LIST.cend(), &processSharedAttributes);
const Attribute_Request_Callback attribute_client_request_callback(CLIENT_ATTRIBUTES_LIST.cbegin(), CLIENT_ATTRIBUTES_LIST.cend(), &processClientAttributes);
...
As you can see - we have 2 callbacks, the first one is for shared attributes and the second one is for client attributes.
The first one receives response with blinking Interval to set the correct period for blinking.
The second callback receives mode and state of the LED to save it and set them.
This functionality allows us to keep the actual state after rebooting.
- Attribute requests:
1 2 3 4 5 6 7 8 9 10 11 12 13
... // Request current states of shared attributes if (!tb.Shared_Attributes_Request(attribute_shared_request_callback)) { Serial.println("Failed to request for shared attributes"); return; } // Request current states of client attributes if (!tb.Client_Attributes_Request(attribute_client_request_callback)) { Serial.println("Failed to request for client attributes"); return; } ...
In order for our callbacks to receive the data, we have to send a request to ThingsBoard.
Control device using shared attributes
Also we can change the period of the blinking using shared attribute update functionality.
- To change period of the blinking we just need to change the value on our dashboard.
- After applying by pressing check mark you will see a confirmation message.
In order to change state when blinking is disabled - we can use the switch in the same widget:
- It can be done only when the blinking mode is disabled.
To reach this, we have a variable “blinkingInterval” used in the following parts of the code:
- Callback for shared attributes update:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
...
void processSharedAttributes(const Shared_Attribute_Data &data) {
for (auto it = data.begin(); it != data.end(); ++it) {
if (strcmp(it->key().c_str(), BLINKING_INTERVAL_ATTR) == 0) {
const uint16_t new_interval = it->value().as<uint16_t>();
if (new_interval >= BLINKING_INTERVAL_MS_MIN && new_interval <= BLINKING_INTERVAL_MS_MAX) {
blinkingInterval = new_interval;
Serial.print("Updated blinking interval to: ");
Serial.println(new_interval);
}
} else if(strcmp(it->key().c_str(), LED_STATE_ATTR) == 0) {
ledState = it->value().as<bool>();
digitalWrite(LED_BUILTIN, ledState ? HIGH : LOW);
Serial.print("Updated state to: ");
Serial.println(ledState);
}
}
attributesChanged = true;
}
...
const Shared_Attribute_Callback attributes_callback(SHARED_ATTRIBUTES_LIST.cbegin(), SHARED_ATTRIBUTES_LIST.cend(), &processSharedAttributes);
...
- Subscribing for shared attributes update:
1
2
3
4
5
6
...
if (!tb.Shared_Attributes_Request(attribute_shared_request_callback)) {
Serial.println("Failed to request for shared attributes");
return;
}
...
- Part of code to blink:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
...
if (ledMode == 1 && millis() - previousStateChange > blinkingInterval) {
previousStateChange = millis();
ledState = !ledState;
digitalWrite(LED_BUILTIN, ledState);
tb.sendTelemetryBool(LED_STATE_ATTR, ledState);
tb.sendAttributeBool(LED_STATE_ATTR, ledState);
if (LED_BUILTIN == 99) {
Serial.print("LED state changed to: ");
Serial.println(ledState);
}
}
...
You can change the logic to reach your goals and add processing for your attributes.
Control device using RPC
You can manually change state of the led and change mode between continuous lightning and blinking. To do this, you can use the following parts of our dashboard:
- Change led state using switch widget to continuous lightning.
- Change led state using round switch widget to blinking mode.
Please note that you can change the LED state only if blinking mode is disabled.
In the code example we have functionality to handle RPC commands.
To get ability to control the device we have used the following parts of the code:
- Callback for RPC requests:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
...
RPC_Response processSetLedMode(const RPC_Data &data) {
Serial.println("Received the set led state RPC method");
// Process data
int new_mode = data;
Serial.print("Mode to change: ");
Serial.println(new_mode);
if (new_mode != 0 && new_mode != 1) {
return RPC_Response("error", "Unknown mode!");
}
ledMode = new_mode;
attributesChanged = true;
// Returning current mode
return RPC_Response("newMode", (int)ledMode);
}
...
const std::array<RPC_Callback, 2U> callbacks = {
RPC_Callback{ "setLedMode", processSetLedMode },
RPC_Callback{ "takePicture", processTakePicture }
};
...
- Subscribing for RPC requests:
1
2
3
4
5
6
...
if (!tb.RPC_Subscribe(callbacks.cbegin(), callbacks.cend())) {
Serial.println("Failed to subscribe for RPC");
return;
}
...
Such as the board has included camera we can take a picture and see it on the dashboard.
- You can take a picture from camera module, by pressing the button on ThingsBoard dashboard.
To take a picture we send “takePicture” RPC to the device.
The following part of the code takes a picture.
1
2
3
4
5
6
7
8
9
10
11
12
13
...
bool captureImage() {
camera_fb_t *fb = NULL;
fb = esp_camera_fb_get();
if (!fb) {
return false;
}
encode((uint8_t *)fb->buf, fb->len);
esp_camera_fb_return(fb);
return true;
}
...
We are unable to send a raw bytes array of the photo in JSON, so we are also encoding bytes to Base64:
1
2
3
4
5
6
7
8
9
...
void encode(const uint8_t *data, size_t length) {
size_t size = base64_encode_expected_len(length) + 1;
base64_encodestate _state;
base64_init_encodestate(&_state);
int len = base64_encode_block((char *)&data[0], length, &imageBuffer[0], &_state);
len = base64_encode_blockend((imageBuffer + len), &_state);
}
...
Our encoded picture will be sent in the main loop:
1
2
3
4
5
6
...
if (sendPicture) {
tb.sendTelemetryString(PICTURE_ATTR, imageBuffer);
sendPicture = false;
}
...
You can change the code to reach your goals and add processing for your RPC commands.
Conclusion
With the knowledge outlined in this guide, you can easily connect your ESP-EYE and send data to ThingsBoard.
Explore the platform documentation to learn more about key concepts and features. For example, configure alarm rules or dashboards.
