Stop the war

Stand with Ukraine flag

Support Ukraine
Try it now Pricing
IoT Gateway
Community Edition Professional Edition Cloud Edge PE Edge IoT Gateway License Server Trendz Analytics Mobile Application PE Mobile Application MQTT Broker
Documentation > Configuration guides > FTP Connector
Getting Started
Installation
On this page

FTP Connector Configuration

This guide will help you to get familiar with FTP Connector configuration for ThingsBoard IoT Gateway. Use general configuration to enable this Connector. The purpose of this Connector is to connect to an external FTP server and get data from files from specific paths. The connector is also able to push data to FTP server files based on the updates/commands from ThingsBoard.

This connector is useful when you have some FTP server in your device or some data in an external resource and you would like to push this data to the ThingsBoard.

We will describe the connector configuration file below.

Connector configuration: ftp.json

Connector configuration is a JSON file that contains information about how to connect to an external FTP server, what paths to use when reading data, and how to process the data.
Let’s review the format of the configuration file using the example below.

Example of FTP Connector config file.

The example listed below will connect to FTP server in a local network deployed on server with IP 0.0.0.0. The connector will utilize basic FTP auth using username and password. Then, connector will subscribe to a list of paths from the mapping section. See more info in the description below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65

{
  "host": "0.0.0.0",
  "port": 21,
  "TLSSupport": false,
  "security": {
    "type": "basic",
    "username": "admin",
    "password": "admin"
  },
  "paths": [
    {
      "devicePatternName": "asd",
      "devicePatternType": "Device",
      "delimiter": ",",
      "path": "fol/*_hello*.txt",
      "readMode": "FULL",
      "maxFileSize": 5,
      "pollPeriod": 5,
      "txtFileDataView": "SLICED",
      "withSortingFiles": true,
      "attributes": [
        {
          "key": "temp",
          "value": "[1:]"
        },
        {
          "key": "tmp",
          "value": "[0:1]"
        }
      ],
      "timeseries": [
        {
          "type": "int",
          "key": "[0:1]",
          "value": "[0:1]"
        },
        {
          "type": "int",
          "key": "temp",
          "value": "[1:]"
        }
      ]
    }
  ],
  "attributeUpdates": [
    {
      "path": "fol/hello.json",
      "deviceNameFilter": ".*",
      "writingMode": "WRITE",
      "valueExpression": "{'${attributeKey}':'${attributeValue}'}"
    }
  ],
  "serverSideRpc": [
    {
      "deviceNameFilter": ".*",
      "methodFilter": "read",
      "valueExpression": "${params}"
    },
    {
      "deviceNameFilter": ".*",
      "methodFilter": "write",
      "valueExpression": "${params}"
    }
  ]
}

General section

Parameter Default value Description
host localhost Domain address or ip of the server.
port 21 Port of the server.
TLSSupport true Verify whether TLS support is available on the server.

Subsection “security”

Subsection “security” provides configuration for client authorization at FTP Server.

BasicRecommended
AnonymousNo security

One type of security configuration is basic. For authorization, a combination of username and password provided in this section’s config will be used.

Parameter Default value Description
type basic Type of authorization.
username username Username for authorization.
password password Password for authorization.

Security subsection in configuration file will look like this:

1
2
3
4
5

    "security": {
      "type": "basic",
      "username": "username",
      "password": "password"
    }

Anonymous auth is the most simple option. It is useful for testing on public FTP servers.

Parameter Default value Description
type anonymous Type of authorization.

Security subsection in configuration file will look like this:

1
2
3

    "security": {
      "type": "anonymous"
    }

Section “paths”

This configuration section contains an array of objects with paths that the gateway will try to read after connecting to the server.
Also, this section contains settings for processing incoming messages (converter).

Parameter Default value Description
path data/log.txt Path to file for reading data.

Note Make sure that your file extension is one of the supported extensions by FTP connector (.txt, .json, .csv)

The path supports special symbols, such as ‘*’ to allow subscription to multiple paths and files. Let’s look at another example of using path parameter:

Example Name Path Comments
Example 1 data/log.txt Read data from one file (if it exists)
Example 2 data/*/log.txt Read all log.txt files from all folders that contain data folder
Example 3 data/*/*_log.txt Read all files that name containing _log.txt from all folders that contain data folder
Example 3 data/*.* Read all files with all supported extensions that are included in data folder

Now let’s review another required parameter in path section.

Parameter Available value Description
delimiter , Symbol that will be used to split the data in the file
readMode FULL/PARTIAL If parameter is equal to FULL, the file will be read from the start to the end. If parameter is equal to PARTIAL, the file will be read from the line that was previously being read
maxFileSize 5 The max file size in MB that will be read (if file size is more than 5 MB, it will be skipped)
pollPeriod 60 The period of time (in seconds) during which files in the path will be read
txtFileDataView TABLE/SLICED The txtFileDataView parameter is used only for .txt files
withSortingFiles true/false This is how the found files in path will be appended to the array
attributes   This subsection contains parameters of the incoming message, that will be interpreted as attributes for the device.
timeseries   This subsection contains parameters of the incoming message, that will be interpreted as telemetry for the device.

Let’s look at how we can configure this section for different file extensions:

  1. For .txt with TABLE data view and .csv files 
    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

       "paths": [
     {
       "devicePatternName": "${temp}",
       "devicePatternType": "Device",
       "delimiter": ",",
       "path": "fol/*.*",
       "readMode": "FULL",
       "maxFileSize": 5,
       "pollPeriod": 60,
       "txtFileDataView": "TABLE",
       "withSortingFiles": true,
       "attributes": [
         {
           "type": "int",
           "key": "key",
           "value": "${temp}"
         }
       ],
       "timeseries": [
         {
           "type": "int",
           "key": "${hum}",
           "value": "${temp}"
         },
         {
           "type": "int",
           "key": "temp",
           "value": "${hum}"
         }
       ]
     }
   ]

    That means that FTP converter will look for the following file structure:

     temp,hum
 1,2
 23,34
  2. For .txt file with SLICED data view 
    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

       "paths": [
     {
       "devicePatternName": "DeviceName",
       "devicePatternType": "[0:1]",
       "delimiter": ",",
       "path": "fol/table.txt",
       "readMode": "FULL",
       "maxFileSize": 5,
       "pollPeriod": 60,
       "txtFileDataView": "SLICED",
       "withSortingFiles": true,
       "attributes": [
         {
           "key": "temp",
           "value": "[1:]"
         },
         {
           "key": "tmp",
           "value": "[0:1]"
         }
       ],
       "timeseries": [
         {
           "type": "int",
           "key": "[0:1]",
           "value": "[0:1]"
         },
         {
           "type": "int",
           "key": "temp",
           "value": "[1:]"
         }
       ]
     }
   ]

    That mean that FTP converter will look for the next file structure:

     1,2
 23,34
  3. For .json file 
    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

     "paths": [
     {
       "devicePatternName": "${temp}",
       "devicePatternType": "Device",
       "delimiter": ",",
       "path": "fol/*.json",
       "readMode": "FULL",
       "maxFileSize": 5,
       "pollPeriod": 60,
       "withSortingFiles": true,
       "attributes": [
         {
           "type": "int",
           "key": "key",
           "value": "${temp}"
         }
       ],
       "timeseries": [
         {
           "type": "int",
           "key": "key",
           "value": "${temp}"
         },
         {
           "type": "int",
           "key": "temp",
           "value": "${tmp}"
         }
       ]
     }
   ]
  4. Combine attributes, telemetry, and serverSideRpc section, for example, in a .json file: 
    1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

      "paths": [
       {
         "devicePatternName": "${temp}",
         "devicePatternType": "Device",
         "delimiter": ",",
         "path": "fol/*.json",
         "readMode": "FULL",
         "maxFileSize": 5,
         "pollPeriod": 60,
         "withSortingFiles": true,
         "attributes": [],
         "timeseries": [
           {
             "type": "int",
             "key": "hum",
             "value": "${hum}"
           },
           {
             "type": "int",
             "key": "temp",
             "value": "${tmp}"
           },
           {
             "type": "string",
             "key": "combine",
             "value": "${tmp}::${hum}"
           }
         ]
       }
     ]

Section “attributeUpdates”

This configuration section is optional. ThingsBoard allows the provisioning of device attributes and fetches some of them from the device application. You can treat this as a remote configuration for devices, enabling them to request shared attributes from ThingsBoard. See user guide for more details.

The “attributeRequests” configuration allows you to configure the format of the corresponding attribute data that will be written to the specific files.

Parameter Default value Description
path fol/${attributeKey}/${attributeValue}.txt JSON-path expression that is used for finding specific files
deviceNameFilter .* Regular expression device name filter, used to determine, which function to execute.
writingMode OVERRIDE/WRITE If writingMode is equal to OVERRIDE, the found files will be overwritten. If writingMode is equal to WRITE, new data will be appended at the end of the found files.
valueExpression ,,,,${attributeKey},,,${attributeValue} Expression is used for creating the message data that will be sent to FTP server. In this case, ‘,’ serves as the delimiter and you can insert your data before it.

This section in configuration file looks like this:

1
2
3
4
5
6
7
8

  "attributeUpdates": [
    {
      "path": "fol/${attributeKey}/${attributeValue}.txt",
      "deviceNameFilter": ".*",
      "writingMode": "OVERRIDE",
      "valueExpression": ",,,,${attributeKey},,,${attributeValue}"
    }
  ]

Server side RPC commands

ThingsBoard allows for sending RPC commands to devices connected directly to ThingsBoard or via Gateway.

Configuration, provided in this section uses for sending RPC requests from ThingsBoard to device.

Parameter Default value Description
deviceNameFilter .* Regular expression device name filter, used to determine, which function to execute.
methodFilter read/write Mode for opening file
valueExpression ,,,,${attributeKey},,,${attributeValue} JSON-path expression, is used to create data for sending to FTP server, if methodFilter is equal to write. If methodFilter is equal to read, this field will be passing.

This section in configuration file looks like this:

1
2
3
4
5
6
7
8
9
10
11
12

"serverSideRpc": [
  {
    "deviceNameFilter": ".*",
    "methodFilter": "read",
    "valueExpression": "${params}"
  },
  {
    "deviceNameFilter": ".*",
    "methodFilter": "write",
    "valueExpression": "${params}"
  }
]

Next steps

Explore guides related to main ThingsBoard features: