Message Queue Research Articles

Boki: Towards Data Consistency and Fault Tolerance with Shared Logs in Stateful Serverless Computing

Bokiis a new serverless runtime that exports a shared log API to serverless functions. Boki shared logs enable stateful serverless applications to manage their state with durability, consistency, and fault tolerance. Boki shared logs achieve high throughput and low latency. The key enabler is the metalog , a novel mechanism that allows Boki to address ordering, consistency and fault tolerance independently. The metalog orders shared log records with high throughput and it provides read consistency while allowing service providers to optimize the write and read path of the shared log in different ways. To demonstrate the value of shared logs for stateful serverless applications, we build Boki support libraries that implement fault-tolerant workflows, durable object storage, and message queues. Our evaluation shows that shared logs can speed up important serverless workloads by up to 4.2×.

Implementation and Analysis of EMQX broker for MQTT Protocol in the Internet of Things

A low-cost monitoring and data transfer system is needed that uses various protocols in the Internet of Things (IoT) technologies. Among the various available protocols for the transmission of messages such as hypertext transfer protocol (HTTP), constrained application protocol (CoAP), etc., message queuing telemetry transport (MQTT) is the most widely used protocol due to its lower power consumption. Further, broker selection is the foremost task in operating the MQTT-based algorithms used in the automation. Researchers have explored several brokers. After a rigorous analysis, the EMQX broker-based MQTT protocol is investigated to transfer the data between the clients efficiently. The implementation and performance analysis of the EMQX broker-based MQTT protocol is assessed in depth with respect to the published and subscribed rates of data packets. Furthermore, new metric sets i.e. average number of messages published and subscribed and messages received with actual QoS are represented and analyzed in combinations of different quality of services (QoSs) to publish and subscribe the data. The analysis shows the efficacy of the investigated protocol.

Worker Presence Monitoring in Complex Workplaces Using BLE Beacon-Assisted Multi-Hop IoT Networks Powered by ESP-NOW

The increasing adoption of Internet of Things (IoT) technologies has facilitated the creation of advanced applications in various industries, notably in complex workplaces where safety and efficiency are paramount. This paper addresses the challenge of monitoring worker presence in vast workplaces such as shipyards, large factories, warehouses, and other construction sites due to a lack of traditional network infrastructure. In this context, we developed a novel system integrating Bluetooth Low Energy (BLE) beacons with multi-hop IoT networks by using the ESP-NOW communications protocol, first introduced by Espressif Systems in 2017 as part of its ESP8266 and ESP32 platforms. ESP-NOW is designed for peer-to-peer communication between devices without the need for a WiFi router, making it ideal for environments where traditional network infrastructure is limited or nonexistent. By leveraging the BLE beacons, the system provides real-time presence data of workers to enhance safety protocols. ESP-NOW, a low-power communications protocol, enables efficient, low-latency communication across extended ranges, making it suitable for complex environments. Utilizing ESP-NOW, the multi-hop IoT network architecture ensures extensive coverage by deploying multiple relay nodes to transmit data across large areas without Internet connectivity, effectively overcoming the spatial challenges of complex workplaces. In addition, the Message Queuing Telemetry Transport (MQTT) protocol is used for robust and efficient data transmission, connecting edge devices to a central Node-RED server for real-time remote monitoring. Moreover, experimental results demonstrate the system’s ability to maintain robust communication with minimal latency and zero packet loss, enhancing worker safety and operational efficiency in large, complex environments. Furthermore, the developed system enhances worker safety by enabling immediate identification during emergencies and by proactively identifying hazardous situations to prevent accidents.

Analysis of Soil Viability Monitoring System for In-House Plantation Growth Using an Internet of Things Approach

Houseplant cultivation has become increasingly popular, allowing individuals to bring nature into their homes. However, successful indoor gardening requires careful monitoring of soil parameters to ensure optimal plant growth. To address this need, sensor technology and Internet of Things (IoT) devices are utilized to monitor soil temperature and moisture levels, which play crucial roles in plant growth. Various soil factors are sensed and collected using an IoT-based microcontroller, with data transmission facilitated by a Message Queue Telemetry Transport (MQTT) broker. Visualization of the data is achieved through the Node-RED programming tool, simplifying dashboard creation for easy monitoring. Furthermore, the collected data is stored in a MySQL server, enabling further analysis through SQL queries. The day is divided into four quarters with six-hour intervals, allowing for soil data collection using temperature and moisture sensors. The resulting information on the dashboard facilitates informed decision-making to enhance soil conditions for optimal indoor plant growth. Experimentation has revealed a reduction in soil temperature of 3°C during daytime due to air conditioning operation, while soil moisture content remains consistently between 60 to 65% during early mornings and late evenings. Additionally, emphasis is placed on remote management using IoT systems, enabling monitoring of plant growth even when access is limited. Overall, monitoring soil factors using IoT technology offers a promising approach to optimizing indoor gardening practices and minimizing environmental resource consumption.

Research on AIS Data Analysis and Its Integration into Discipline-specific Teaching

The processing of ship AIS (Automatic Identification System) data plays a crucial role in the digital transformation of the shipping industry. From data collection to data storage, data analysis, and data visualization, AIS data processing technology provides robust support for the shipping sector. In terms of data collection, the platform utilizes distributed message queues to aggregate AIS data. For data storage, the platform efficiently manages and stores massive amounts of AIS data using data warehouses. When it comes to data analysis, the platform mines and analyzes AIS data through Lambda, extracting valuable information such as ship trajectories and sailing speeds, thereby supporting areas such as ship management, navigation safety, and marine research. In data visualization, the platform intuitively displays ship navigation status through charts and maps, providing strong support for decision-making. As an essential component of big data applications, AIS data analysis technology is playing an increasingly important role in the shipping industry. The big data technology major of Jiangsu Maritime Institute integrates big data applications like AIS data analysis into its curriculum, taking the upgrading needs of the shipping industry as an entry point. It actively plans the cultivation of composite big data skilled talents under the backdrop of smart shipping, comprehensively serving the upgrading of the shipping industry and promoting the transformation from a major shipping country to a shipping powerhouse.

Implementation of Smart Farm Systems Based on Fog Computing in Artificial Intelligence of Things Environments.

Cloud computing has recently gained widespread attention owing to its use in applications involving the Internet of Things (IoT). However, the transmission of massive volumes of data to a cloud server often results in overhead. Fog computing has emerged as a viable solution to address this issue. This study implements an Artificial Intelligence of Things (AIoT) system based on fog computing on a smart farm. Three experiments are conducted to evaluate the performance of the AIoT system. First, network traffic volumes between systems employing and not employing fog computing are compared. Second, the performance of the communication protocols-hypertext transport protocol (HTTP), message queuing telemetry transport protocol (MQTT), and constrained application protocol (CoAP)-commonly used in IoT applications is assessed. Finally, a convolutional neural network-based algorithm is introduced to determine the maturity level of coffee tree images. Experimental data are collected over ten days from a coffee tree farm in the Republic of Korea. Notably, the fog computing system demonstrates a 26% reduction in the cumulative data volume compared with a non-fog system. MQTT exhibits stable results in terms of the data volume and loss rate. Additionally, the maturity level determination algorithm performed on coffee fruits provides reliable results.

Stealthy Messaging: Leveraging Message Queuing Telemetry Transport for Covert Communication Channels

Covert channel methods are techniques for improving privacy and security in network communications. These methods consist of embedding secret data within normal network channels, making it more difficult for unauthorized parties to detect such data. This paper presents a new approach for creating covert channels using the Message Queuing Telemetry Transport (MQTT) protocol, widely used in the context of the Internet of Things (IoT). The proposed method exploits storage channels by altering the field length of MQTT messages. Our solution leverages well-known one-way mathematical functions to ensure that data remain hidden from third parties observing the MQTT stream. In this way, we ensure that not only the content of the communication is preserved but also that the communication itself takes place. We conducted a security analysis to show that our solution offers the above-mentioned property even against severe threats, such as an adversary being able to observe all the messages exchanged in the network (even in the clear). Finally, we conducted an overhead analysis of our solution both in terms of the time required to perform the required operations and of the bytes to send. Our study shows that our solution adds no significant time overhead, and the additional overhead in terms of transmitted bytes remains within acceptable limits.

Implications of architecture and implementation choices on timing analysis of automotive CAN networks

The Controller Area Network (CAN) protocol is widely adopted in industry such as automotive. It has been analyzed in a number of studies in the real-time systems community to compute the worst case response time of messages, based on an ideal model on the behavior of the message queuing and CAN controller. Recently, more results are being added to study systems that are not constructed according to the ideal model.In this paper, we further investigate the architecture choices that may affect the timing analysis. Specifically, we provide an assessment on the practical relevance of several analysis results. We also present theory and empirical studies on the relative importance of architecture implementation issues that are quite common in real systems but further deviate from the ideal behavior. We also experimentally evaluate the response time while using TxObjects without preemption. We propose a heuristic for the design of multiple software queues when TxObjects are not preemptable. Finally, we derive an upper bound on the worst case response time when message output at the CAN driver is polling based.

Performance evaluation framework of MQTT client libraries for IoT applications in manufacturing

The Internet of Things (IoT) is a revolutionary technology in the field of manufacturing that enables real-time data communication and control of various industrial sensors and devices. MQTT (Message Queuing Telemetry Transport) is one of the most popular communication protocols for IoT applications which offers lightweight, efficient, and reliable communication. There are several MQTT client libraries available for implementing MQTT communication for IoT applications. The selection of an appropriate client library can significantly impact the performance of an IoT application. This paper presents a performance evaluation framework for MQTT client libraries for IoT applications in the manufacturing industry. This paper aims to create a standardized virtual testing environment to compare the performance of different MQTT client libraries using various test conditions. It offers a structured and standardized methodology for evaluating performance aspects such as latency, scalability, and resource utilization. By systematically evaluating the performance of MQTT client libraries against these metrics, users can make informed decisions while selecting the most appropriate library for their IoT applications. The framework takes into consideration the unique requirements and constraints of manufacturing applications, such as the need for low-latency communication, secure data exchange, and compatibility with resource-constrained devices. Ultimately, this research contributes to the optimization of IoT applications in the manufacturing industry, facilitating the implementation of efficient, reliable, and scalable data communication.

Design of ECG Circuit and Patient Remote Monitoring System Using MQTT Protocol for ECG Signals

In the healthcare sector, leveraging the Internet of Things (IoT) brings significant advantages, especially in remote monitoring of patients' health using electrocardiogram (ECG) signals. This study employs IoT technology to enable remote ECG monitoring, utilizing the MQTT (Message Queuing Telemetry Transport) protocol to establish a web server. The dedicated measuring device integrates the ADAS1000-3 IC with ESP32 for real-time ECG signal acquisition. In this study, we utilize the Pan-Tompkins algorithm to extract QRS complexes from electrocardiographic signals, count heartbeats, and early detect cardiovascular abnormalities. This solution was tested on a sample dataset from the MIT-BIH (Massachusetts Institute of Technology - Beth Israel Deaconess Medical Center) database. Finally, the article proposes to design and develop a device using advanced IC technologies such as IC ADAS1000-3 combined with ESP32. This device will be designed to meet the requirements of real-time processing and data transmission via webserver.

A novel lightweight multi-factor authentication scheme for MQTT-based IoT applications

The present authentication solutions employed in the Internet of Things (IoT) are either inadequate or computationally intensive, given the resource-constrained nature of IoT devices. This challenges the researchers to devise efficient solutions to embed an important security tenet like authentication. In IoT, the most popular machine-to-machine communication protocol used at the application layer is Message Queuing Telemetry Transport (MQTT). However, the MQTT protocol inherently lacks security-related functions, like authentication, authorization, confidentiality, access control, and data integrity, which is unacceptable for IoT-driven mission-critical applications when connected over public networks. In such a situation, the security is hardened by employing a transport layer security protocol like TLS, which entails significant computational overheads. This paper presents a novel scheme to enhance MQTT security by providing a lightweight multi-factor authentication scheme based on Elliptical curve cryptography. The proposed scheme uses a low-cost signature and a fuzzy extractor to correct errors in imprinted biometrics in noisy environments. This scheme attains mutual authentication, generates a securely agreed-upon session key for secret communication, and guarantees perfect forward secrecy. Furthermore, the rigorous informal security analysis shows the proposed scheme resists cryptographic attacks, including known session critical attacks. Furthermore, an empirical study has been carried out to assess the effectiveness of the proposed scheme in the Cooja simulated environment.

IoT Frameworks: Comparing Efficiency and Scalability in Smart City Applications

As Smart City apps change quickly, IoT platforms are very important for making it possible to install linked devices and services quickly and on a large scale. The main goal of this study is to compare how well and how easily different IoT systems can be expanded to work in Smart City settings. Being efficient in IoT systems means that they can make the best use of resources, reduce delay, use less energy, and keep data transfer stable. Concerning scalability, on the other hand, we look at how well these frameworks can grow to support more devices and people without affecting security or speed. This study looks at a number of well-known IoT systems, each with its own layout and set of rules made to solve specific Smart City problems. Frameworks like MQTT (Message Queuing Telemetry Transport) focus on lightweight communication and publish-subscribe messaging models. This makes them perfect for situations where low bandwidth and fast data sharing between many devices in cities are needed. CoAP (Constrained Application Protocol) is unique because of its RESTful architecture, which allows for scalability thanks to its stateless design and support for resource discovery. This makes it ideal for use in situations where device interactions are dynamic and spread out, like in traffic management systems or environmental monitoring. The study also looks into systems such as LwM2M (Lightweight Machine-to-Machine), which is known for its effective device control and support for devices that don't have a lot of memory or processing power. Its protocol design makes sure that data sharing and device control work reliably, which is important for keeping service going in large-scale setups like those found in Smart City infrastructures. The study also looks at how these frameworks compare in terms of performance measures like speed, delay, and flexibility when networks are changed and workloads are distributed in ways that are common in Smart City settings.

EduSolar: A Remote-Controlled Photovoltaic/Thermal Educational Lab with Integrated Daylight Simulation

This study presents a compact educational photovoltaic/thermal (PV/T) system designed for thorough performance assessment under simulated weather conditions. As an affordable educational tool, the system offers significant pedagogical value. The PV/T system features two photovoltaic modules: a thermally enhanced module and a standard one. The thermally enhanced module uses water as a coolant, which transfers heat from the PV cells to a fan-operated heat exchanger, with the coolant being recirculated to maintain optimal conditions. A halogen lamp, placed between the modules, simulates solar radiation to ensure effective electrical current generation. The system’s remote-control capabilities, managed via the Message Queuing Telemetry Transport (MQTT) protocol, enable real-time adjustments to the coolant flow rate, heat exchanger efficiency, and lamp brightness, as well as monitoring of electrical parameters. Experimental findings indicate that the PV/T module achieves a 7.71% increase in power output compared to the standard PV module and offers a 17.41% improvement in cooling efficiency over scenarios without cooling. Additionally, the numerical methods used in the study show a maximum deviation of 4.29% from the experimental results, which is considered acceptable. This study showcases a best practice model for solar training, applicable from elementary to university levels, and suggests innovative approaches to enhancing solar energy education.

A hybrid intrusion detection approach based on message queuing telemetry transport (MQTT) protocol in industrial internet of things

AbstractThe number of attacks against Industrial Internet of Things (IIoT) devices has increased over the past years, particularly on widely used communication protocols like Message Queuing Telemetry Transfer (MQTT). The fast increase in IIoT applications brings both critical challenges and technical gaps in cybersecurity. On the other hand, traditional cyber‐attack detection approaches scrap to address and support the run‐time responsibilities of IIoT environments. This study presents a hybrid Genetic Algorithm and Random Forest (GA_RF) method for detecting cyber‐attacks in Industrial Control Machines (ICS) that use MQTT protocol in the IIoT environment. This architecture integrates ICS with edge devices and cloud servers, using a GA_RF algorithm to detect anomalies in data collected by sensors. Normal data is processed locally and then sent to the cloud for storage and return, ensuring continuous monitoring and security. Also, the MQTT‐IOT‐IDS2020 dataset as a real test case was applied for prediction of the proposed GA_RF method with compare to some other powerful machine and deep learning models. The experimental results show that the proposed GA_RF method has an optimum accuracy of 99.87%–100% for detecting cyber‐attacks. This hybrid algorithm also achieved 0–0.0015 in Mean Absolute Error (MAE) and 100% in Precision, Recall, and F‐score factors. This result led to the proposed architecture, which connects the ICS to a server while running GA_RF on the IIoT environment. In conclusion, this study indicates the effectiveness of GA_RF and aims to improve security by using the MQTT protocol in IIoT.

Enhancing a fog-oriented IoT authentication and encryption platform through deep learning-based attack detection

The term Internet of Things (IoT) refers to a network that connects smart things with sensors. Healthcare, transportation, and smart cities are some IoT applications. IoT technologies integrate objects in the cloud-based Internet. The massive scale of IoT exposes some systems to attacks. There is an urgent need for solutions that efficiently handle IoT authentication, encryption, and attack detection. This work proposes a Fog-based IoT security platform named IoTSafe. It contains mechanisms for authentication and encryption and a deep learning-based attack detection module. The IoTSafe attack detection module uses the Message Queue Telemetry Transport (MQTT). Tests were performed to evaluate the IoTSafe platform in three different environments. A case study demonstrated that the platform is efficient with all proposed mechanisms and modules. The results for the attack detection module show the proposal’s effectiveness with an accuracy of 99.57% and a precision of 99.66%. The IoTSafe time response was less than one second, guaranteeing the quality of service.

Internet of things and radio frequency identification based embedded system to reduce shopping time in supermarkets

Doing daily shopping in a Peruvian supermarket means a large investment of time for many people, usually due to inaccurate and faulty scanning of products by barcodes at supermarket checkout counters. For this reason, an embedded system based on internet of things (IoT) and radio frequency identification (RFID) is designed to reduce shopping time in a supermarket. The system uses an ESP32 development board with embedded hardware specialized in IoT projects and firmware development based on C language and real-time operating systems (FreeRTOS) through espressif’s IoT development framework (ESP-IDF). RFID tags were used to scan the products and IoT with message queuing telemetry transport (MQTT) communication protocol are implemented to a local database in real time. The system achieves a significant reduction in terms of scanning time compared to self-service checkouts using barcodes, which allows to statistically analyze the reduced time per quantity of products and the linear trend of the 2 samples.

Enhance the Detection of DoS and Brute Force Attacks within the MQTT Environment Through Feature Engineering and Employing an Ensemble Technique

The rapid development of the Internet of Things (IoT) environment has introduced unprecedented levels of connectivity and automation. The Message Queuing Telemetry Transport (MQTT) protocol has become recognized in IoT applications due to its lightweight and efficient features; however, this simplicity also renders MQTT vulnerable to multiple attacks that can be launched against the protocol, including denial of service (DoS) and brute-force attacks. This study aims to improve the detection of intrusion DoS and bruteforce attacks in an MQTT traffic intrusion detection system (IDS). Our approach utilizes the MQTT dataset for model training by employing effective feature engineering and ensemble learning techniques. Following our analysis and comparison, we identified the top 10 features demonstrating the highest effectiveness, leading to improved model accuracy. We used supervised machine learning models, including Random Forest, Decision Trees, k-Nearest Neighbors, and XGBoost, in combination with ensemble classifiers. Stacking, voting, and bagging ensembles utilize these four supervised machinelearning methods to combine models. This study's results illustrate the proposed technique's efficacy in enhancing the accuracy of detecting DoS and brute-force attacks in MQTT traffic. Stacking and voting classifiers achieved the highest accuracy of 0.9538. Our approach outperforms the most recent study that utilized the same dataset.

Enabling Seamless Connectivity: Networking Innovations in Wireless Sensor Networks for Industrial Application.

The wide-ranging applications of the Internet of Things (IoT) show that it has the potential to revolutionise industry, improve daily life, and overcome global challenges. This study aims to evaluate the performance scalability of mature industrial wireless sensor networks (IWSNs). A new classification approach for IoT in the industrial sector is proposed based on multiple factors and we introduce the integration of 6LoWPAN (IPv6 over low-power wireless personal area networks), message queuing telemetry transport for sensor networks (MQTT-SN), and ContikiMAC protocols for sensor nodes in an industrial IoT system to improve energy-efficient connectivity. The Contiki COOJA WSN simulator was applied to model and simulate the performance of the protocols in two static and moving scenarios and evaluate the proposed novelty detection system (NDS) for network intrusions in order to identify certain events in real time for realistic dataset analysis. The simulation results show that our method is an essential measure in determining the number of transmissions required to achieve a certain reliability target in an IWSNs. Despite the growing demand for low-power operation, deterministic communication, and end-to-end reliability, our methodology of an innovative sensor design using selective surface activation induced by laser (SSAIL) technology was developed and deployed in the FTMC premises to demonstrate its long-term functionality and reliability. The proposed framework was experimentally validated and tested through simulations to demonstrate the applicability and suitability of the proposed approach. The energy efficiency in the optimised WSN was increased by 50%, battery life was extended by 350%, duplicated packets were reduced by 80%, data collisions were reduced by 80%, and it was shown that the proposed methodology and tools could be used effectively in the development of telemetry node networks in new industrial projects in order to detect events and breaches in IoT networks accurately. The energy consumption of the developed sensor nodes was measured. Overall, this study performed a comprehensive assessment of the challenges of industrial processes, such as the reliability and stability of telemetry channels, the energy efficiency of autonomous nodes, and the minimisation of duplicate information transmission in IWSNs.

Improved Message Mechanism-Based Cross-Domain Security Control Model in Mobile Terminals

Dual-domain terminal with two built-in independent operating systems - Life Domain and Work Domain, provides convenience for daily use and mobile office. However, the security isolation between the two domains also causes that message reminders cannot be delivered and viewed across domains, which restricts the improvement of work efficiency and the expansion of mobile services. This paper conducts an in-depth study on this pain point and proposes the concept and implementation method of a cross-domain instant messaging reminder service system for mobile office, focusing on solving the problems of: cross-domain isolated boundary exchange of message reminders, timeliness and delivery rate guarantee of message reminders, and security check filtering of message contents. Technically, on the side of mobile office platform, based on AMQP technical framework and protocol, the cross-domain isolated border message queue push and synchronization services are built, which are real-time, reliable and high-throughput.

A Reliable Publish–Subscribe Mechanism for Internet of Things-Enabled Smart Greenhouses

Messaging protocols for the Internet of Things (IoT) play a crucial role in facilitating efficient product creation and waste reduction, and in enhancing agricultural process efficiency within the realm of smart greenhouses. Publish–subscribe (pub-sub) systems improve communication between IoT devices and cloud platforms. Nevertheless, IoT technology is required to effectively handle a considerable volume of subscriptions or topic adjustments from several clients concurrently. In addition, subscription throughput is an essential factor of the pub-sub mechanism, as it directly influences the speed at which messages may be sent to subscribers. The primary focus of this paper pertains to a performance assessment of the proposed message categorization architecture for the Message Queue Telemetry Transport (MQTT) broker. This architecture aims to establish a standardized approach to pub-sub topics and generate new topics with various performance characteristics. We also standardize the form of MQTT protocol broker topic categorization and payload based on greenhouse specifications. The establishment of topic classification enhances the operational effectiveness of the broker, reduces data volume, and concurrently augments the number of messages and events transmitted from the greenhouse environment to the central server on a per-second basis. Our proposed architecture is validated across multiple MQTT brokers, including Mosquitto, ActiveMQ, Bevywise, and EMQ X, showing enhanced flexibility, extensibility, and simplicity while maintaining full compatibility with greenhouse environments. Key findings demonstrate significant improvements in performance metrics. The message processing time for the proposed Active MQ broker was increased approximately five-fold across all QoS levels compared to the original. Subscription throughput for the Bevywise MQTT Route 2.0 broker at QoS0 reached 1453.053, compared to 290.610 for the original broker. The number of messages in the Active MQ broker at QoS0 surged from 394.79 to 1973.95. These improvements demonstrate the architecture’s potential for broader IoT applications in pub-sub systems.