We’re delighted to introduce TBMQ version 2.0.0! This release brings a major update with data migration of persistent sessions for devices from PostgreSQL to Redis. It also expands TBMQ’s MQTT 5.0 feature set, achieving full compatibility with the MQTT 5.0 standard. Here’s an overview of the features and updates included in this release.
Migration from PostgreSQL to Redis
With this release, we’ve made a strategic shift from PostgreSQL to Redis specifically for handling data related to persistent sessions for devices. This migration is a core part of our commitment to optimizing performance and scalability in high-demand MQTT environments where managing persistent client sessions efficiently is crucial.
Why the Change?
While PostgreSQL is a powerful relational database, it wasn’t designed to handle the high-throughput, low-latency requirements of MQTT brokers. Our experience showed that as data volume and request rates increased, PostgreSQL struggled to maintain the speed and responsiveness essential for real-time MQTT workloads due to its lack of horizontal scalability options.
What Redis Brings to the Table
Redis, a fast, in-memory data store, is built for speed, supporting high-performance operations with minimal latency. By migrating to Redis, we now achieve:
- Improved data access speed: Redis allows us to store key MQTT data in memory, drastically reducing retrieval times compared to disk-based systems.
- Enhanced scalability: Redis’s efficient handling of high-frequency read/write operations aligns perfectly with the needs of MQTT brokers, enabling TBMQ to scale horizontally and easily manage millions of concurrent connections.
- Low-latency operations: Redis’s non-blocking I/O and simple data structures mean that even under heavy load, response times remain low, ensuring smoother message processing and session management.
What’s Next?
This migration is only the beginning. Our engineering team is dedicated to leveraging Redis to its fullest potential, and we’re already working on additional performance optimizations. Stay tuned for more detailed performance test results as we push the boundaries of what TBMQ can achieve in real-time MQTT processing.
Advanced session metrics
With the new client session metrics in TBMQ, administrators gain a detailed view of each client’s messaging patterns, broken down by Quality of Service (QoS) levels. These metrics offer several key advantages for managing and optimizing the MQTT broker environment:
- Enhanced session visibility: By seeing how many messages are processed at each QoS level, administrators can identify patterns in client usage and detect anomalies, such as unusually high or low message counts, which may indicate connection issues or performance bottlenecks.
- Proactive troubleshooting: With detailed metrics, issues such as message loss, delayed delivery, or irregular publish rates can be pinpointed and investigated early, minimizing potential disruptions.
- Client behavior analysis: Advanced metrics offer granular insights into individual client behavior, helping administrators identify top publishers and subscribers.
Overall, this feature transforms the monitoring experience, making it easier to ensure the broker remains performant and reliable, even as client numbers and message volumes scale.
Unauthorized clients
To bolster security, TBMQ now includes an ‘Unauthorized Clients’ feature, which provides real-time monitoring of connection attempts made by clients with invalid credentials. This feature logs details such as the client ID, IP address, username, authentication method, and the specific reason for failure, offering administrators a clear view of unauthorized access attempts.
By tracking these failed connections, administrators gain valuable insights into potential security threats, including repeated access attempts from certain IPs or specific usernames, enabling them to take proactive security measures. The Unauthorized Clients feature is essential for maintaining a secure and robust broker environment, especially in deployments where client authentication integrity is critical.
Subscriptions
TBMQ’s new Subscriptions page offers a centralized view that brings clarity and control to broker monitoring by displaying all active subscriptions in one accessible location. With a single click, administrators can open a client’s session details directly from this page, making it easy to add, edit, or delete subscriptions on the spot.
This comprehensive overview streamlines subscription management, helping administrators track which clients are subscribed to specific topics, assess subscription distribution across QoS levels, and quickly make adjustments as needed. By simplifying these processes, the Subscriptions page is instrumental in managing broker activity, diagnosing subscription issues, and ensuring clients connect to the right topics efficiently.
MQTT 5.0: Subscription Identifier
This update introduces the Subscription Identifier feature, an important addition to optimizing message tracking and processing in MQTT communications.
How It Works: The Subscription Identifier in MQTT 5.0 allows the client to assign a unique identifier to each subscription. When the broker sends messages that match a subscription, it includes this identifier in the message properties, allowing the client to recognize which subscription triggered the message. The Subscription Identifier is set when the client subscribes to a topic, and it remains associated with that subscription until it is modified or removed.
Why It’s Needed: The Subscription Identifier is essential for several reasons:
- Efficient message routing: By tagging each message with a Subscription Identifier, clients can quickly determine the origin subscription, making it easier to handle and route messages appropriately.
- Improved performance in message processing: Subscription Identifiers allow clients to reduce the need for complex message parsing, which is especially important in high-throughput environments where rapid response times are critical.
- Enhanced analytics: The feature allows clients to analyze message flow on a per-subscription basis, providing deeper insights into traffic patterns and helping optimize the configuration for various use cases.
Use Case: The Subscription Identifier is particularly useful in scenarios with multiple subscriptions and diverse message flows, such as IoT applications with numerous sensors and devices. For example, in a smart city infrastructure, each subscription (e.g., traffic data, weather data, air quality monitoring) can be uniquely identified, allowing the central system to efficiently process and route data to the appropriate services for further action.
MQTT 5.0: Enhanced Authentication
This update includes the Enhanced Authentication feature, a significant improvement to the MQTT protocol that allows for more flexible and secure client authentication methods.
How It Works: Enhanced Authentication in MQTT 5.0 introduces a new AUTH packet, which enables multi-step authentication processes. This allows the broker and client to exchange multiple authentication-related messages during a session, supporting advanced authentication schemes such as challenge-response mechanisms and token-based authentication. The authentication process can occur at any time during the session, allowing for re-authentication as needed.
Why It’s Needed: Enhanced Authentication is essential for several reasons:
- Greater security flexibility: Traditional MQTT authentication typically relies on a single-step username and password exchange, which can be limiting for applications with stringent security needs. Enhanced Authentication enables the use of more sophisticated methods, such as SCRAM (Salted Challenge Response Authentication Mechanism), which provides secure, challenge-based authentication. SCRAM is particularly valuable for securing data exchanges by preventing password exposure during transmission and mitigating certain attack vectors, such as replay attacks.
- Support for dynamic authentication mechanisms: Enhanced Authentication allows for re-authentication during an ongoing session. This feature is particularly valuable for applications that need to validate client credentials periodically or handle token expiration gracefully, ensuring continuous and secure access.
- Adaptability to industry-specific security standards: Many industries, such as finance and healthcare, have stringent authentication standards. Enhanced Authentication provides the flexibility to meet these standards, allowing businesses to adopt MQTT while complying with industry regulations.
Use Case: Enhanced Authentication is particularly beneficial for IoT deployments in sectors requiring strong security, such as industrial automation and healthcare. For example, in a healthcare setting, devices transmitting sensitive patient data can use token-based authentication to ensure secure, periodic revalidation, protecting against unauthorized access and meeting strict compliance requirements.
Other notable features and enhancements
In addition to the features highlighted earlier, we’ve made significant improvements to enhance user insights, overall system robustness, and performance.
The client session details now include MQTT client credentials and MQTT version information for authenticated clients. This enhancement provides better visibility into session specifics, allowing users to easily track client authentication and version.
A critical improvement in this release enhances the performance of processing received MQTT publish messages. Previously, two queues (one for messages, and another for acknowledgments) were used to handle this process, but with this update, we’ve streamlined it to a single, non-blocking queue. This optimization significantly boosts handling efficiency and reduces latency, making it ideal for high-throughput environments where fast message processing is crucial.
This release brings a range of meaningful enhancements, each designed to improve the system’s transparency, security, and performance, paving the way for a more reliable and efficient MQTT experience. We invite you to try out these powerful new features by installing the latest version of TBMQ.
