Real-time Data Processing Research Articles

GPU-accelerated OCT imaging: Real-time data processing and artifact suppression for enhanced monitoring of 3D bioprinted tissues and vascular-like networks

Optical coherence tomography (OCT) imaging technology has significant advantages in in situ and noninvasive monitoring of biological tissues. However, it still faces the following challenges: including data processing speed, image quality, and improvements in three-dimensional (3D) visualization effects. OCT technology, especially functional imaging techniques like optical coherence tomography angiography (OCTA), requires a long acquisition time and a large data size. Despite the substantial increase in the acquisition speed of swept source optical coherence tomography (SS-OCT), it still poses significant challenges for data processing. Additionally, during in situ acquisition, image artifacts resulting from interface reflections or strong reflections from biological tissues and culturing containers present obstacles to data visualization and further analysis. Firstly, a customized frequency domain filter with anti-banding suppression parameters was designed to suppress artifact noises. Then, this study proposed a graphics processing unit (GPU)-based real-time data processing pipeline for SS-OCT, achieving a measured line-process rate of 800[Formula: see text]kHz for 3D fast and high-quality data visualization. Furthermore, a GPU-based real-time data processing for CC-OCTA was integrated to acquire dynamic information. Moreover, a vascular-like network chip was prepared using extrusion-based 3D printing and sacrificial materials, with sacrificial material being printed at the desired vascular network locations and then removed to form the vascular-like network. OCTA imaging technology was used to monitor the progression of sacrificial material removal and vascular-like network formation. Therefore, GPU-based OCT enables real-time processing and visualization with artifact suppression, making it particularly suitable for in situ noninvasive longitudinal monitoring of 3D bioprinting tissue and vascular-like networks in microfluidic chips.

Maintenance optimization methodology of edge cloud collaborative systems based on a gateway cost index in IIoT

In the Industrial Internet of Things (IIoT), cloud computing faces enormous data processing pressure and security challenges due to the surge in industrial devices and sensors, especially in real-time data processing and analysis applications. However, fewer researchers study the reliability of data transmission and the maintenance of data interruption processing of edge cloud collaborative systems (ECCS) in IIoT. This paper proposes a maintenance optimization methodology of ECCS based on a gateway cost index. Firstly, a reliability model of different network topologies of ECCS is proposed considering key communication indicators of task runtime, packet loss rate, and bandwidth. Then, considering the impact of external environments on ECCS, an importance-based Gateway Cost Index is proposed. A maintenance optimization is studied by conducting a multi-objective programming. The specific maintenance steps are analyzed when the intelligent gateways of ECCS are subjected to large-scale failures caused by external interference. At last, two numerical examples with star topology and bus topology are provided to demonstrate the proposed methods.

Application of Event Cameras and Neuromorphic Computing to VSLAM: A Survey.

Simultaneous Localization and Mapping (SLAM) is a crucial function for most autonomous systems, allowing them to both navigate through and create maps of unfamiliar surroundings. Traditional Visual SLAM, also commonly known as VSLAM, relies on frame-based cameras and structured processing pipelines, which face challenges in dynamic or low-light environments. However, recent advancements in event camera technology and neuromorphic processing offer promising opportunities to overcome these limitations. Event cameras inspired by biological vision systems capture the scenes asynchronously, consuming minimal power but with higher temporal resolution. Neuromorphic processors, which are designed to mimic the parallel processing capabilities of the human brain, offer efficient computation for real-time data processing of event-based data streams. This paper provides a comprehensive overview of recent research efforts in integrating event cameras and neuromorphic processors into VSLAM systems. It discusses the principles behind event cameras and neuromorphic processors, highlighting their advantages over traditional sensing and processing methods. Furthermore, an in-depth survey was conducted on state-of-the-art approaches in event-based SLAM, including feature extraction, motion estimation, and map reconstruction techniques. Additionally, the integration of event cameras with neuromorphic processors, focusing on their synergistic benefits in terms of energy efficiency, robustness, and real-time performance, was explored. The paper also discusses the challenges and open research questions in this emerging field, such as sensor calibration, data fusion, and algorithmic development. Finally, the potential applications and future directions for event-based SLAM systems are outlined, ranging from robotics and autonomous vehicles to augmented reality.

Real-Time Data Processing: Frameworks, Machine Learning Integration, and Traffic Analysis Using Computer Vision

Real-Time Data Processing: Frameworks, Machine Learning Integration, and Traffic Analysis Using Computer Vision

RADAR-IoT: An Open-Source, Interoperable, and Extensible IoT Gateway Framework for Health Research.

IoT sensors offer a wide range of sensing capabilities, many of which have potential health applications. Existing solutions for IoT in healthcare have notable limitations, such as closed-source, limited I/O protocols, limited cloud platform support, and missing specific functionality for health use cases. Developing an open-source internet of things (IoT) gateway solution that addresses these limitations and provides reliability, broad applicability, and utility is highly desirable. Combining a wide range of sensor data streams from IoT devices with ambulatory mHealth data would open up the potential to provide a detailed 360-degree view of the relationship between patient physiology, behavior, and environment. We have developed RADAR-IoT as an open-source IoT gateway framework, to harness this potential. It aims to connect multiple IoT devices at the edge, perform limited on-device data processing and analysis, and integrate with cloud-based mobile health platforms, such as RADAR-base, enabling real-time data processing. We also present a proof-of-concept data collection from this framework, using prototype hardware in two locations. The RADAR-IoT framework, combined with the RADAR-base mHealth platform, provides a comprehensive view of a user's health and environment by integrating static IoT sensors and wearable devices. Despite its current limitations, it offers a promising open-source solution for health research, with potential applications in managing infection control, monitoring chronic pulmonary disorders, and assisting patients with impaired motor control or cognitive ability.

Analysis of Real-time Processing Approaches for Large Data Volumes in Metering Infrastructure

Smart grid systems and communication technologies, such as Advanced Metering Infrastructure (AMI), have revolutionized utility service management and monitoring. AMI leverages smart meters equipped with advanced communication capabilities, facilitating bidirectional communication between utilities and consumers. The increasing deployment of smart meters and the adoption of sub-hourly data collection requirements by utility companies highlight significant data volume growth. Thus, there is a need for efficient real-time data processing solutions as existing approaches may not meet previously established Service-Level Agreements (SLAs) concerning performance, accuracy, and scalability metrics. This research aims to comprehensively review the latest publications relevant to distributed real-time data processing methods for smart grid applications and outline problems for further research. Specifically, the study delves into the effectiveness and application of reviewed approaches in managing the constant stream of data from smart meters and IoT devices within the smart grid context. By analysing existing methodologies and advancements, this study seeks to identify challenges and opportunities in real-time data processing for smart grid infrastructures, focusing on addressing the complexities of processing, managing, and storing large volumes of real-time data. The literature review revealed two primary applications of real-time data processing: optimization of data streaming performance and data analysis. The review encompasses various studies, each presenting distinct methodologies and technologies applied to address the challenges of processing large volumes of real- time data from smart meters and IoT devices. Future research should address the challenges and limitations discovered in this study.

Deep Learning for Anomaly Detection in IoT Systems: Techniques, Applications, and Future Directions

The proliferation of Internet of Things (IoT) devices has revolutionized various sectors by enabling real-time data collection and processing, leading to unprecedented levels of efficiency and convenience. However, this increased connectivity and data flow also introduce significant security risks, particularly in the realm of anomaly detection. This study delves into the use of deep learning techniques for identifying abnormalities in IoT systems, providing a thorough analysis of state-of-the-art methods, assessing their effectiveness across different IoT scenarios, and exploring future research directions. We review current deep learning-based anomaly detection techniques, examine their applications in real-world settings, and discuss potential improvements and innovations. By synthesizing the latest research and developments, this paper aims to offer a comprehensive understanding of how deep learning can bolster the security and performance of IoT systems. Our findings highlight the importance of robust anomaly detection mechanisms in safeguarding IoT networks and underscore the need for continued advancements in this area. Through this study, we seek to contribute valuable insights into the application of deep learning for enhancing IoT system security and reliability, ultimately supporting the sustainable growth and integration of IoT technologies across various domains.

Task offloading framework to meet resiliency demand in mobile edge computing system

With the development of 5 G mobile users are increasing massively. Some mobile applications like healthcare are latency-critical and requires real-time data processing. A preference-based task offloading framework in mobile edge computing with a device-to-device offloading (MECD2D) system has been proposed to fulfill the latency demands of such applications for minimum energy consumption ensuring resiliency. The problem is formulated as a constraint-based non-linear optimization problem which is complex. The resources are allocated in two steps. In the first step, resources are allocated based on latency demand to ensure resiliency. In the second step, allocated resources are optimized using a non-cooperative mean field game for dynamic system. To ensure the performance of the system for dynamic network, the results are executed on a real-time Shanghai dataset. The computational results indicate that the proposed algorithm performs better in terms of energy consumption. Other parameters such as throughput, network utilization and task computation are also analysed. The results are verified by performing the proposed algorithm with existing Q learning and mean-field game algorithms. The results performed on the dataset indicate an improvement in energy consumption by 5–10 %, and 10–50 % as compared to Q learning and mean-field game respectively.

The Impact of Edge Computing on Real-Time Data Processing

Purpose: The study sought to explore the impact of edge computing on real-time data processing. Methodology: The study adopted a desktop research methodology. Desk research refers to secondary data or that which can be collected without fieldwork. Desk research is basically involved in collecting data from existing resources hence it is often considered a low cost technique as compared to field research, as the main cost is involved in executive’s time, telephone charges and directories. Thus, the study relied on already published studies, reports and statistics. This secondary data was easily accessed through the online journals and library. Findings: The findings reveal that there exists a contextual and methodological gap relating to the impact of edge computing on real-time data processing. Preliminary empirical review reveled that edge computing significantly reduced latency and enhanced efficiency in real-time data processing across various industries by bringing computational resources closer to data sources. It highlighted the technology's ability to handle large volumes of IoT-generated data, improve security by localizing data processing, and drive innovation and economic growth through new applications and services. Edge computing's decentralized approach proved essential for reliable and robust data handling, particularly in critical sectors like healthcare and finance, ultimately solidifying its importance in the digital transformation landscape. Unique Contribution to Theory, Practice and Policy: The Diffusion of Innovations Theory, Resource-Based View (RBV) and Sociotechnical Systems Theory may be used to anchor future studies on edge computing on real-time data processing. The study recommended expanding theoretical frameworks to include the unique aspects of edge computing, investing in robust edge infrastructure, and developing standardized protocols and best practices. It emphasized the need for government incentives and supportive regulatory frameworks to promote adoption, and suggested that academic institutions incorporate edge computing into curricula. Additionally, the study called for ongoing research to address emerging challenges and opportunities, ensuring continuous advancement and effective implementation of edge computing technologies.

Applications of Fog Computing in Healthcare.

Fog computing is a decentralized computing infrastructure that processes data at or near its source, reducing latency and bandwidth usage. This technology is gaining traction in healthcare due to its potential to enhance real-time data processing and decision-making capabilities in critical medical scenarios. A systematic review of existing literature on fog computing in healthcare was conducted. The review included searches in major databases such as PubMed, IEEE Xplore, Scopus, and Google Scholar. The search terms used were "fog computing in healthcare," "real-time diagnostics and fog computing," "continuous patient monitoring fog computing," "predictive analytics fog computing," "interoperability in fog computing healthcare," "scalability issues fog computing healthcare," and "security challenges fog computing healthcare." Articles published between 2010 and 2023 were considered. Inclusion criteria encompassed peer-reviewed articles, conference papers, and review articles focusing on the applications of fog computing in healthcare. Exclusion criteria were articles not available in English, those not related to healthcare applications, and those lacking empirical data. Data extraction focused on the applications of fog computing in real-time diagnostics, continuous monitoring, predictive analytics, and the identified challenges of interoperability, scalability, and security. Fog computing significantly enhances diagnostic capabilities by facilitating real-time data analysis, crucial for urgent diagnostics such as stroke detection, by processing data closer to its source. It also improves monitoring during surgeries by enabling real-time processing of vital signs and physiological parameters, thereby enhancing patient safety. In chronic disease management, continuous data collection and analysis through wearable devices allow for proactive disease management and timely adjustments to treatment plans. Additionally, fog computing supports telemedicine by enabling real-time communication between remote specialists and patients, thereby improving access to specialist care in underserved regions. Fog computing offers transformative potential in healthcare, improving diagnostic precision, patient monitoring, and personalized treatment. Addressing the challenges of interoperability, scalability, and security will be crucial for fully realizing the benefits of fog computing in healthcare, leading to a more connected and efficient healthcare environment.

Research on the Optimization of Cross-Border Logistics Routes under the Belt and Road Initiative

Within the framework of the Belt and Road Initiative, optimizing cross-border logistics routes is essential for enhancing the efficiency of logistics services and reducing transportation costs. This paper analyzes the current state of logistics networks in countries along the Belt and Road route, identifying major obstacles to the efficiency of cross-border logistics such as uneven infrastructure development, complex transnational policies and regulations, and significant geographical and political risks. The study further introduces the theoretical foundation of logistics route optimization and discusses the application of various optimization models and algorithms, such as linear programming and network flow analysis. The paper also highlights the role of modern technologies, such as artificial intelligence and the Internet of Things, in route optimization. These technologies enhance route selection and freight scheduling through real-time data processing and predictive models. Based on these analyses, specific strategies and implementation suggestions are proposed to reduce transportation time and costs by optimizing the logistics network configuration.

Efficient Heuristic Management of Cloud Data Lakes for Enhanced Big Data Handling in Cloud Computing Environments

Efficient control of cloud data lakes is important for improving big data dealing in cloud computing settings. Cloud data lakes serve as stores for vast amounts of organized, semi-structured, and unstructured data from varied sources, offering scaling and freedom in data storage and processing. However, the sheer amount and range of data offer difficulties in improving storage, access, and analysis processes. This study suggests a heuristic method to handle these issues by improving the order and recovery of data within cloud data lakes. The heuristic method focuses on automatically handling data splitting, storage, and caching strategies based on data access trends and task factors. By leveraging heuristic principles, such as adaptable splitting and predictive caching, the suggested method aims to reduce delay and improve total query speed. Key components of the heuristic management include adaptable splitting methods that actively change data partition sizes based on access frequency and data distribution patterns. This flexible method ensures that frequently accessed data is easily available, while less accessed data is efficiently saved to improve storage usage. Furthermore, predictive storing methods expect future data access patterns using machine learning models or previous data trends. By strategically saving relevant datasets in memory or high-speed storage, the system reduces delay for repeated questions, thereby improving real-time data processing capabilities. The success of the suggested heuristic management method is tested through models and benchmarks using real big data tasks. Comparative research against standard static splitting and caching methods shows significant changes in query response times and resource utilization efficiency.

A State-of-the-Art Review in Big Data Management Engineering: Real-Life Case Studies, Challenges, and Future Research Directions

The explosion of data volume in the digital age has completely changed the corporate and industrial environments. In-depth analysis of large datasets to support strategic decision-making and innovation is the main focus of this paper’s exploration of big data management engineering. A thorough examination of the basic elements and approaches necessary for efficient big data use—data collecting, storage, processing, analysis, and visualization—is given in this paper. With real-life case studies from several sectors to complement our exploration of cutting-edge methods in big data management, we present useful applications and results. This document lists the difficulties in handling big data, such as guaranteeing scalability, governance, and data quality. It also describes possible future study paths to deal with these issues and promote ongoing creativity. The results stress the need to combine cutting-edge technology with industry standards to improve decision-making based on data. Through an analysis of approaches such as machine learning, real-time data processing, and predictive analytics, this paper offers insightful information to companies hoping to use big data as a strategic advantage. Lastly, this paper presents real-life use cases in different sectors and discusses future trends such as the utilization of big data by emerging technologies.

Real-time data processing method for CO2 dispersion interferometer on EAST

A real-time data processing system is designed for the carbon dioxide dispersion interferometer (CO2-DI) on EAST. The system utilizes the parallel and pipelining capabilities of an field-programmable gate array (FPGA) to digitize and process the intensity of signals from the detector. Finally, the real-time electron density signals are exported through a digital-to-analog converter (DAC) module in the form of analog signals. The system has been successfully applied in the CO2-DI system to provide low-latency electron density input to the plasma control system on EAST. Experimental results of the latest campaign with long-pulse discharges on EAST (2022–2023) demonstrate that the system can respond effectively in the case of rapid density changes, proving its reliability and accuracy for future electron density calculation.

MICROSERVICE ARCHITECTURE FOR CYBER-PHYSICAL SYSTEMS

Cyber-Physical Systems connect the physical and digital worlds. They are composed of hardware connected to the physical world, software, and potentially other types of systems. They are utilized across various industries, including robotics, healthcare, smart cities, automotive, industry, and space. These systems are very complex to design and implement. Cloud computing technologies provide an excellent environment for the Cyber-Physical Systems software and tools for maintaining and scaling the infrastructure. One of the main challenges is how to design cloud systems properly for Cyber-Physical Systems. Currently, microservice architecture is broadly used for software in the cloud. At its core, Microservices provide flexibility, availability, scalability, and independence of modules, as well as agile development and deployment processes. These advantages are well-aligned with the needs of Cyber-Physical Systems. However, many challenges still exist in implementing a microservice architecture for Cyber-Physical Systems. The challenges include complex distributed system networking, real-time data processing, microservice software architecture, microservice availability, and reliability of the system components. This paper provides a study whose primary goal is to give the main microservice architectural principles and patterns used, summarize the advantages and challenges, and improve the knowledge of the microservice architecture used for Cyber-Physical Systems. At first, a literature review of modern research papers was conducted. Then, each paper was analyzed. A summary of all selected research papers was produced. The results and conclusion sections deliver the summaries and give future research directions.

Federated Learning Empowered Real-Time Medical Data Processing Method for Smart Healthcare.

Computer-aided diagnosis (CAD) has always been an important research topic for applying artificial intelligence in smart healthcare. Sufficient medical data are one of the most critical factors in CAD research. However, medical data are usually obtained in chronological order and cannot be collected all at once, which poses difficulties for the application of deep learning technology in the medical field. The traditional batch learning method consumes considerable time and space resources for real-time medical data, and the incremental learning method often leads to catastrophic forgetting. To solve these problems, we propose a real-time medical data processing method based on federated learning. We divide the process into the model stage and the exemplar stage. In the model stage, we use the federated learning method to fuse the old and new models to mitigate the catastrophic forgetting problem of the new model. In the exemplar stage, we use the most representative exemplars selected from the old data to help the new model review the old knowledge, which further mitigates the catastrophic forgetting problem of the new model. We use this method to conduct experiments on a simulated medical real-time data stream. The experimental results show that our method can learn a disease diagnosis model from a continuous medical real-time data stream. As the amount of data increases, the performance of the disease diagnosis model continues to improve, and the catastrophic forgetting problem has been effectively mitigated. Compared with the traditional batch learning method, our method can significantly save time and space resources.

Method of field measurement of electromagnetic environment of high speed maglev train

Abstract In order to reduce the complexity of the magnetic field test of maglev trains and improve the accuracy of measurement data, this paper proposes a field measurement method of maglev train radiation emission based on high-speed parallel adaptive filter denoising technology. This measurement method is not limited by special measurement sites, and has the characteristics of high-speed measurement, real-time data processing, and large-capacity data storage. It breaks through the technical bottleneck of the traditional measurement system. The electromagnetic radiation emission of high-speed maglev trains is measured in real-time through multi-antenna synchronous measurement technology, and multi-channel high-speed RF signal measurement and acquisition of external signals are further carried out by blind source separation technology, environmental pre-scanning is carried out, environmental interference characteristics are obtained through feature statistics, and the radiation signal to be measured is estimated by blind source separation algorithm. It helps to achieve the purpose of accurately measuring the electromagnetic radiation emission of high-speed maglev trains. Through field testing, the denoising performance of the method reached 20 dB. Through field testing, the denoising performance of the method reached 20 dB.

UWLPIPE: Ultra-wide Bandwidth Low-frequency Pulsar Data Processing Pipeline

For real-time processing of ultra-wide bandwidth low-frequency pulsar baseband data, we designed and implemented an ultra-wide bandwidth low-frequency pulsar data processing pipeline (UWLPIPE) based on the shared ringbuffer and GPU parallel technology. UWLPIPE runs on the GPU cluster and can simultaneously receive multiple 128 MHz dual-polarization VDIF data packets preprocessed by the front-end FPGA. After aligning the dual-polarization data, multiple 128M subband data are packaged into PSRDADA baseband data or multi-channel coherent dispersion filterbank data, and multiple subband filterbank data can be spliced into wideband data after time alignment. We used the Nanshan 26 m radio telescope with the L-band receiver at 964 ∼ 1732 MHz to observe multiple pulsars. Finally, we processed the data using DSPSR software, and the results showed that each subband could correctly fold out the pulse profile, and the wideband pulse profile accumulated by multiple subbands could be correctly aligned.

Enhancing fire safety through IoT-enabled flame detection systems: A cost-effective and scalable approach

The Internet of Things (IoT), which connects and automates numerous systems and gadgets, has completely changed how we live and work. One such application of IoT technology is in fire detection systems, which can help prevent and mitigate the devastating effects of fires on different types of facilities. The research presents a n IoT architecture for a fire detection system using small, low-cost cameras to collect surveillance feeds from large buildings. The data is uploaded to the cloud, where a Machine Learning algorithm detects fires in digital images. The proposed architecture consists of cameras, cloud, and clients, using an inexpensive camera for surveillance feeds and a convolutional neural network for image classification based on large datasets. However, the architecture's cloud component processes surveillance feeds and runs a Machine Learning (ML) model, utilizing computing resources for real-time data processing and continuous training for improved accuracy. Clients can subscribe to the data from the cloud and receive alerts in real-time when the ML model detects a fire in the surveillance feeds. There are significant benefits in comparing the proposed design to conventional fire detection systems. First and foremost, it is economical since the cameras used are compact, affordable, and simple to install around the building without the need for elaborate wiring or infrastructure. Secondly, it is scalable, as the cloud provides the necessary computing resources and storage capacity to handle large amounts of data, making it possible to monitor large structures with many cameras.

Supply Chain Optimization: Machine Learning Applications in Inventory Management for E-Commerce

This study delves into the potential impact of machine learning (ML) on supply chain optimization and inventory management for e-commerce. Our primary focus is analyzing the accuracy of demand forecasting, optimizing inventory levels, and evaluating the impact of real-time decision-making on supply chain efficiency. Using a secondary data-based review methodology, this study explores the implementation of advanced predictive analytics, real-time data processing, autonomous operations, and personalized customer experiences in prominent e-commerce companies like Amazon, Walmart, and Alibaba. Our findings show impressive advancements in demand forecasting accuracy, dynamic inventory management, and operational responsiveness. These improvements have led to cost savings and increased customer satisfaction. Nevertheless, some drawbacks exist, such as the significant expenses associated with implementation, concerns about data privacy, and the potential for overfitting the model. Policy implications call for solid data protection regulations, financial assistance for smaller businesses, and ethical guidelines for AI to promote fair and responsible machine learning applications. By tackling these obstacles, companies can maximize the potential of ML technologies to enhance efficiency, promote sustainability, and gain a competitive edge in the ever-changing world of e-commerce.