Optimizing dashboard performance with enhanced image management

At ThingsBoard, we’ve always been committed to enhancing user experience and system performance. Our latest update, version 3.6.2, introduces significant improvements in how images are managed within dashboards, widgets, and more. This update is a key enhancement designed to tackle inefficiencies identified as our user base expanded.

The challenge with large images

Since the earliest releases, our platform has permitted the embedding of images in dashboards and widgets via external links or Base64 encoded data URLs. While flexible, this approach presented a significant challenge. Users occasionally embedded images up to 50MB in size directly into dashboards. Editing or exporting these dashboards meant transferring massive JSON files to and from the server, a highly inefficient process. This not only affected loading times but also the responsiveness of the dashboard editing experience.

The solution

To address this issue, we’ve introduced a major update in how images are handled within the platform. A central aspect of this update is the introduction of the Image Gallery. By moving dashboards to a separate entity and utilizing our internal, relative links for storing image references, we’ve significantly improved efficiency. This change minimizes the size of dashboard JSON files, streamlining both loading and editing processes. The Image Gallery simplifies the process of uploading, organizing, and selecting images.

Backward compatibility

The platform’s upgrade script automatically converts all existing dashboards to the new format. Users can still upload dashboards in the old format to a new platform instance. These dashboards will automatically be converted to the new format during the upload process.

For example, consider a “Sensors” dashboard with its background image stored in the old format:

{
...
"backgroundImageUrl": "data:image/webp;base64,iVBORw...JRU5ErkJggg=="
...
}

After the upgrade, a new image will be created in the image gallery:

The image name is auto-generated based on the dashboard name and the dashboard element where the image is used. This same behavior applies to images used in widget configuration: markers, backgrounds, etc. The dashboard now references the image using the following link:

{
...
"backgroundImageUrl": "/api/images/tenant/sensors_dashboard_default_background.webp"
...
}

Private and public image links

Images can be provisioned at the system and tenant levels. System-level widgets are accessible to all platform users, while tenant images are available to authorized users of specific tenants.

To embed an image into an Angular HTML template, such as the Markdown widget, HTML section in the widget editor, custom actions, etc., use the following code snippet:

<img [src]="'/api/images/tenant/sensors_dashboard_default_background.webp' | image | async" />

The image pipe automatically adds the necessary authentication headers to the image request, allowing the same relative image URL to return different content for users from different tenants.

However, custom widgets or static HTML blocks that do not support Angular HTML syntax cannot add authentication headers to requests. Therefore, every platform image also has a unique URL for downloading the image without authentication. Using the following code snippet, you may embed an image into the components based on the plain HTML. Such components include HTML card widgets, cell content functions, etc.

<img src="/api/images/public/ZA4biARtpnl970l0j8MMolRKM56fbpK1" alt="&quot;Sensors&quot; dashboard default background" />

Each code snippet is unique to its image, and you can find the corresponding snippets in the Embed image window.

Caching

Our image API now utilizes ETags to optimize caching, ensuring that images are only re-downloaded when they have changed. This approach significantly saves bandwidth and improves load times. While we don’t apply Cache-Control headers by default, users have the flexibility to customize caching behavior through environment variables, including settings for image ETags cache TTL, maximum cache size, and browser cache TTL for both system and tenant images. These settings are crucial for balancing between reducing server load and ensuring content freshness.

The platform offers environment variables to control caching behavior:

• CACHE_SPECS_IMAGE_ETAGS_TTL: Controls the cache TTL for image ETags. The default is set to 44640 minutes (31 days), after which the cache is cleared.
  
• CACHE_SPECS_IMAGE_ETAGS_MAX_SIZE: Determines the maximum ETags cache size on the server. The default is 10000. Setting this to 0 disables the cache.
  
• CACHE_SPECS_IMAGE_SYSTEM_BROWSER_TTL and CACHE_SPECS_IMAGE_TENANT_BROWSER_TTL: These settings manage the browser cache TTL for system and tenant images, respectively, with a default of 0 minutes, meaning caching is disabled by default.

These settings allow administrators to adjust the balance between performance and content freshness as needed.

Improvement Summary

By tackling the challenge of image management within dashboards, we aim to offer a more efficient, robust, and user-friendly platform. Following the upgrade of thingsboard.cloud, we’ve observed an 86% reduction in the size of the dashboards, and some dashboards now load more than twice as fast.