Real-time Framework Research Articles

BronchoTrack: Airway Lumen Tracking for Branch-Level Bronchoscopic Localization.

Localizing the bronchoscope in real time is essential for ensuring intervention quality. However, most existing vision-based methods struggle to balance between speed and generalization. To address these challenges, we present BronchoTrack, an innovative real-time framework for accurate branch-level localization, encompassing lumen detection, tracking, and airway association. To achieve real-time performance, we employ benchmark light weight detector for efficient lumen detection. We firstly introduce multi-object tracking to bronchoscopic localization, mitigating temporal confusion in lumen identification caused by rapid bronchoscope movement and complex airway structures. To ensure generalization across patient cases, we propose a training-free detection-airway association method based on a semantic airway graph that encodes the hierarchy of bronchial tree structures. Experiments on 11 patient datasets demonstrate BronchoTrack's localization accuracy of 81.72%, while accessing up to the 6th generation of airways. Furthermore, we tested BronchoTrack in an in-vivo animal study using a porcine model, where it localized the bronchoscope into the 8th generation airway successfully. Experimental evaluation underscores BronchoTrack's real-time performance in both satisfying accuracy and generalization, demonstrating its potential for clinical applications.

A New Iterative Algorithm for Magnetic Motion Tracking.

Motion analysis is of great interest to a variety of applications, such as virtual and augmented reality and medical diagnostics. Hand movement tracking systems, in particular, are used as a human-machine interface. In most cases, these systems are based on optical or acceleration/angular speed sensors. These technologies are already well researched and used in commercial systems. In special applications, it can be advantageous to use magnetic sensors to supplement an existing system or even replace the existing sensors. The core of a motion tracking system is a localization unit. The relatively complex localization algorithms present a problem in magnetic systems, leading to a relatively large computational complexity. In this paper, a new approach for pose estimation of a kinematic chain is presented. The new algorithm is based on spatially rotating magnetic dipole sources. A spatial feature is extracted from the sensor signal, the dipole direction in which the maximum magnitude value is detected at the sensor. This is introduced as the "maximum vector". A relationship between this feature, the location vector (pointing from the magnetic source to the sensor position) and the sensor orientation is derived and subsequently exploited. By modelling the hand as a kinematic chain, the posture of the chain can be described in two ways: the knowledge about the magnetic correlations and the structure of the kinematic chain. Both are bundled in an iterative algorithm with very low complexity. The algorithm was implemented in a real-time framework and evaluated in a simulation and first laboratory tests. In tests without movement, it could be shown that there was no significant deviation between the simulated and estimated poses. In tests with periodic movements, an error in the range of 1° was found. Of particular interest here is the required computing power. This was evaluated in terms of the required computing operations and the required computing time. Initial analyses have shown that a computing time of 3 μs per joint is required on a personal computer. Lastly, the first laboratory tests basically prove the functionality of the proposed methodology.

Operational electricity cost reduction using real-time simulators

Ensuring the safe and cost-effective electricity transmission to consumers, along with the efficient and sustainable operation of power systems, has long been a primary objective for power system managers and operators. However, achieving optimal performance in a modern power system requires timely planning and operational optimal routines to be run within the realm of real-time simulation programs, and sometimes, the divergence becomes a problem with these developed routines. Despite extensive research aimed at advancing these objectives, the growing complexity of networks and their constraints has often led to the oversight of simultaneously considering both technical and economic aspects during operation. In this paper, an innovative real-time framework is introduced to reduce the operational costs of a power system swiftly. This framework addresses various nonlinear constraints, such as transmission losses, generation-consumption balance, power device limitations, and transient stability constraints, across various operational scenarios within a real-time simulator. This novel approach leverages scattered search and intentional contingencies within the network to pinpoint the optimal operating point, taking into account various network dynamics, including Automatic Voltage Regulators (AVR), governors, and tap of the transformers. The proposed method offers a fast and robust approach to identifying the most effective operating conditions for a power system. It incorporates considerations for sudden contingencies and extends capabilities through parallel simulations. It not only facilitates pre-determined preventive actions but also enables the adjustment of control parameters during post-contingency periods, such as fine-tuning of the active and reactive power generation of generators and adjusting the tap settings of transformers within power networks. Since the network simulation can be executed on distributed computers, referred to as ’Global,’ it is possible to achieve global network optimization. This approach allows for the consideration of grid networks, including renewable energy sources with models distributed through PCs. Also, it accounts for the induction motor losses within all factories involved in the global simulation. The results obtained from simulating the proposed method on a commercial real-time simulator demonstrate the superior effectiveness of the proposed framework compared to the existing methodologies, highlighting its potential to enhance the operational efficiency and economic viability of power systems.

Due date-oriented picker routing, an efficient exact solution algorithm, and its application to pick-from-store omnichannel retailing

The advent of e-commerce and omnichannel retailing has sparked renewed interest in picker routing in warehouses. This paper presents two significant methodological advances in this well-established field. First, it is a well-known fact that the parallel-aisle layout of warehouses, as opposed to general graphs, allows for polynomial-time solutions of the Traveling Salesman Problem. We show that the parallel-aisle structure can also be exploited when pickers are tasked with visiting storage positions associated with specific due dates. We establish that picker routing in warehouses, subject to soft due date constraints, is a binary NP-hard problem. We also present an exact branch-and-bound algorithm with pseudo-polynomial time complexity. This algorithm effectively solves instances with up to 60 picking positions and five cross aisles within a few seconds while guaranteeing optimality. Second, for even larger pick lists, we demonstrate the successful integration of our algorithm into a real-time framework. This approach allows us to avoid extended solution times that would otherwise delay the picker’s departure, without compromising the quality of the solution. To illustrate the practical relevance of these two methodological innovations, we apply our routing algorithm to the context of pick-from-store omnichannel retailing. By assigning due dates to critical products, we significantly reduce stockout occurrences for online customers. These stockouts occur when the stock level, initially deemed sufficient to confirm an online order, is depleted by walk-in customers before the picker reaches the relevant shelf.

Real-time assessment of ship collision risk using image processing techniques

The poor quality or the miss of Automatic Identification System (AIS) data may cause erroneous judgement of the potential navigational risk. Therefore, this study proposes a real-time framework for assessing ship collision risk using onboard video data in order to improve the risk perception ability of navigators. Firstly, the Squeeze-and-Excitation (SE) attention mechanism and the K-means algorithm are simultaneously utilized for the framework to enhance the multi-scale ship detection capability. The Deep-SORT is employed to complete multi-ship feature matching. Secondly, the distances between two ships and their speeds are measured using the pinhole imaging principle based on the ship visual feature extraction results. Moreover, the ship distance-speed correction method is designed to improve the reliability of estimated results. Finally, the effectiveness of the framework is validated using naturalistic driving data from the “He Hua Hai” ship. The results show that the proposed framework could demonstrate an excellent performance in assessing ship collision risk using the onboard video data. The proposed framework could help precisely detect and promptly provide warnings about potential ship collision risks. This could help prevent catastrophic accidents that pose a threat to oceans and coasts, particularly in situations when AIS data proves to be unreliable or ineffective.

An Integrated Software-Defined Networking–Network Function Virtualization Architecture for 5G RAN–Multi-Access Edge Computing Slice Management in the Internet of Industrial Things

The Internet of Things (IoT), namely, the set of intelligent devices equipped with sensors and actuators and capable of connecting to the Internet, has now become an integral part of the most competitive industries, as it enables optimization of production processes and reduction in operating costs and maintenance time, together with improving the quality of products and services. More specifically, the term Industrial Internet of Things (IIoT) identifies the system which consists of advanced Internet-connected equipment and analytics platforms specialized for industrial activities, where IIoT devices range from small environmental sensors to complex industrial robots. This paper presents an integrated high-level SDN-NFV architecture enabling clusters of smart devices to interconnect and manage the exchange of data with distributed control processes and databases. In particular, it is focused on 5G RAN-MEC slice management in the IIoT context. The proposed system is emulated by means of two distinct real-time frameworks, demonstrating improvements in connectivity, energy efficiency, end-to-end latency and throughput. In addition, its scalability, modularity and flexibility are assessed, making this framework suitable to test advanced and more applications.

Real-time object detection, tracking, and monitoring framework for security surveillance systems

The concept of security is becoming a global challenge, and governments, stakeholders, corporate societies, and individuals must urgently create a reasonable protection mechanism for good. Therefore, a real-time surveillance system is essential for detection, tracking, and monitoring. Many studies have attempted to provide better solutions but more research and better approaches are essential. This study presents a real-time framework for object detection and tracking for security surveillance systems. The system has been designed based on approximate median filtering, component labeling, background subtraction, and deep learning approaches. The new algorithms for object detection, tracking, and recognition have been implemented using Python and integrated with C# programming languages for ease of use. A software application framework is designed, implemented, and evaluated. The experimental results based on MOT-Challenge performance metrics show that the proposed algorithms have much better performance in terms of accuracy and precision on the MOT15, MOT16, and MOT17 datasets compared to state-of-the-art approaches. This framework also provides an accurate and effective means of monitoring and recognizing moving objects. The software development, including the design of the framework user interfaces, is coded in the C# programming language and integrated with Python using Microsoft Visual Studio (2019 edition). The integration is performed to provide a convenient user interface and to enable the execution of the framework as a standard and standalone software application. Future studies will consider the dynamic scalability of the framework to accommodate different surveillance application areas in overcrowded scenarios. Multiple data sources are integrated to enhance the performance for different scene times, locations, and weather conditions. Furthermore, other object-detection techniques such as You Only Look Once (YOLO) and its variants shall be considered in future studies. These techniques allow the framework to adapt to complex situations in which security surveillance is challenging.

Real-Time Deep Learning Framework for Accurate Speed Estimation of Surrounding Vehicles in Autonomous Driving

Accurate speed estimation of surrounding vehicles is of paramount importance for autonomous driving to prevent potential hazards. This paper emphasizes the critical role of precise speed estimation and presents a novel real-time framework based on deep learning to achieve this from images captured by an onboard camera. The system detects and tracks vehicles using convolutional neural networks and analyzes their trajectories with a tracking algorithm. Vehicle speeds are then accurately estimated using a regression model based on random sample consensus. A synthetic dataset using the CARLA simulator has been generated to validate the presented methodology. The system can simultaneously estimate the speed of multiple vehicles and can be easily integrated into onboard computer systems, providing a cost-effective solution for real-time speed estimation. This technology holds significant potential for enhancing vehicle safety systems, driver assistance, and autonomous driving.

Advanced Millimeter-Wave Radar System for Real-Time Multiple-Human Tracking and Fall Detection.

This study explored an indoor system for tracking multiple humans and detecting falls, employing three Millimeter-Wave radars from Texas Instruments. Compared to wearables and camera methods, Millimeter-Wave radar is not plagued by mobility inconveniences, lighting conditions, or privacy issues. We conducted an initial evaluation of radar characteristics, covering aspects such as interference between radars and coverage area. Then, we established a real-time framework to integrate signals received from these radars, allowing us to track the position and body status of human targets non-intrusively. Additionally, we introduced innovative strategies, including dynamic Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering based on signal SNR levels, a probability matrix for enhanced target tracking, target status prediction for fall detection, and a feedback loop for noise reduction. We conducted an extensive evaluation using over 300 min of data, which equated to approximately 360,000 frames. Our prototype system exhibited a remarkable performance, achieving a precision of 98.9% for tracking a single target and 96.5% and 94.0% for tracking two and three targets in human-tracking scenarios, respectively. Moreover, in the field of human fall detection, the system demonstrates a high accuracy rate of 96.3%, underscoring its effectiveness in distinguishing falls from other statuses.

Real-time correlators in 3+1D thermal lattice gauge theory

We present the first direct computation of unequal-time correlation functions in non-Abelian lattice gauge theory. We demonstrate nontrivial consistency relations among correlators, time-translation invariance, and agreement with Monte Carlo results for thermal equilibrium in 3+1 dimensions by employing our stabilized complex Langevin method. Our work sets the stage to extract real-time observables, relevant to quark-gluon plasma physics within a first-principles real-time framework. Published by the American Physical Society 2024

Robust compression and detection of epileptiform patterns in ECoG using a real-time spiking neural network hardware framework

Interictal Epileptiform Discharges (IED) and High Frequency Oscillations (HFO) in intraoperative electrocorticography (ECoG) may guide the surgeon by delineating the epileptogenic zone. We designed a modular spiking neural network (SNN) in a mixed-signal neuromorphic device to process the ECoG in real-time. We exploit the variability of the inhomogeneous silicon neurons to achieve efficient sparse and decorrelated temporal signal encoding. We interface the full-custom SNN device to the BCI2000 real-time framework and configure the setup to detect HFO and IED co-occurring with HFO (IED-HFO). We validate the setup on pre-recorded data and obtain HFO rates that are concordant with a previously validated offline algorithm (Spearman’s ρ = 0.75, p = 1e-4), achieving the same postsurgical seizure freedom predictions for all patients. In a remote on-line analysis, intraoperative ECoG recorded in Utrecht was compressed and transferred to Zurich for SNN processing and successful IED-HFO detection in real-time. These results further demonstrate how automated remote real-time detection may enable the use of HFO in clinical practice.

BRAND: a platform for closed-loop experiments with deep network models.

Objective.Artificial neural networks (ANNs) are state-of-the-art tools for modeling and decoding neural activity, but deploying them in closed-loop experiments with tight timing constraints is challenging due to their limited support in existing real-time frameworks. Researchers need a platform that fully supports high-level languages for running ANNs (e.g. Python and Julia) while maintaining support for languages that are critical for low-latency data acquisition and processing (e.g. C and C++).Approach.To address these needs, we introduce the Backend for Realtime Asynchronous Neural Decoding (BRAND). BRAND comprises Linux processes, termednodes, which communicate with each other in agraphvia streams of data. Its asynchronous design allows for acquisition, control, and analysis to be executed in parallel on streams of data that may operate at different timescales. BRAND uses Redis, an in-memory database, to send data between nodes, which enables fast inter-process communication and supports 54 different programming languages. Thus, developers can easily deploy existing ANN models in BRAND with minimal implementation changes.Main results.In our tests, BRAND achieved <600 microsecond latency between processes when sending large quantities of data (1024 channels of 30 kHz neural data in 1 ms chunks). BRAND runs a brain-computer interface with a recurrent neural network (RNN) decoder with less than 8 ms of latency from neural data input to decoder prediction. In a real-world demonstration of the system, participant T11 in the BrainGate2 clinical trial (ClinicalTrials.gov Identifier: NCT00912041) performed a standard cursor control task, in which 30 kHz signal processing, RNN decoding, task control, and graphics were all executed in BRAND. This system also supports real-time inference with complex latent variable models like Latent Factor Analysis via Dynamical Systems.Significance.By providing a framework that is fast, modular, and language-agnostic, BRAND lowers the barriers to integrating the latest tools in neuroscience and machine learning into closed-loop experiments.

A real-time framework for HD video defogging using modified dark channel prior

A real-time framework for HD video defogging using modified dark channel prior

PhagoStat a scalable and interpretable end to end framework for efficient quantification of cell phagocytosis in neurodegenerative disease studies

Quantifying the phagocytosis of dynamic, unstained cells is essential for evaluating neurodegenerative diseases. However, measuring rapid cell interactions and distinguishing cells from background make this task very challenging when processing time-lapse phase-contrast video microscopy. In this study, we introduce an end-to-end, scalable, and versatile real-time framework for quantifying and analyzing phagocytic activity. Our proposed pipeline is able to process large data-sets and includes a data quality verification module to counteract potential perturbations such as microscope movements and frame blurring. We also propose an explainable cell segmentation module to improve the interpretability of deep learning methods compared to black-box algorithms. This includes two interpretable deep learning capabilities: visual explanation and model simplification. We demonstrate that interpretability in deep learning is not the opposite of high performance, by additionally providing essential deep learning algorithm optimization insights and solutions. Besides, incorporating interpretable modules results in an efficient architecture design and optimized execution time. We apply this pipeline to quantify and analyze microglial cell phagocytosis in frontotemporal dementia (FTD) and obtain statistically reliable results showing that FTD mutant cells are larger and more aggressive than control cells. The method has been tested and validated on several public benchmarks by generating state-of-the art performances. To stimulate translational approaches and future studies, we release an open-source end-to-end pipeline and a unique microglial cells phagocytosis dataset for immune system characterization in neurodegenerative diseases research. This pipeline and the associated dataset will consistently crystallize future advances in this field, promoting the development of efficient and effective interpretable algorithms dedicated to the critical domain of neurodegenerative diseases’ characterization. https://github.com/ounissimehdi/PhagoStat.

Assessment of prosumer-based energy system for rural areas by using TRNSYS software

The prosumers are the new performers progressing towards a low-carbon future. Renewable-based power generation is essential, even in rural areas, to pave a path toward sustainable development. This study establishes a simulation model for two identical residential buildings to assess the prosumer's impact on future local (standalone) energy systems. According to the climate condition of Pakistan, the technical, economic, and environmental performance of the standalone system is evaluated using TRNSYS software. Assumptions and modeling of different components, photovoltaic (PV) panels, and batteries are also discussed. This study provides the starting point of a real-time framework of energy trade between two rural houses connected with a common microgrid. Both houses are medium-sized family houses with almost identical electricity demands. One house has PV mounted on the rooftop, with the battery as an optimal energy storage option. Attention is given to peak demand management and surplus energy during enough production hours. A dynamic energy management approach between two buildings is proposed based on TRNSYS and blockchain. Simulation results show that the real-time implementation of local energy production and energy trading at the household level facilitates achieving the dual benefits of reducing consumer costs and maximizing self-consumption.

An Artificial Intelligence and Industrial Internet of Things-Based Framework for Sustainable Hydropower Plant Operations

Hydropower plays a crucial role in supplying electricity to developed nations and is projected to expand its capacity in various developing countries such as Sub-Saharan Africa, Argentina, Colombia, and Turkey. With the increasing demand for sustainable energy and the emphasis on reducing carbon emissions, the significance of hydropower plants is growing. Nevertheless, numerous challenges arise for these plants due to their aging infrastructure, impacting both their efficiency and structural stability. In order to tackle these issues, the present study has formulated a specialized real-time framework for identifying damage, with a particular focus on detecting corrosion in the conductors of generators within hydropower plants. It should be noted that corrosion processes can be highly complex and nonlinear, making it challenging to develop accurate physics-based models that capture all the nuances. Therefore, the proposed framework leverages autoencoder, an unsupervised, data-driven AI technology with the Mahalanobis distance, to capture the intricacies of corrosion and automate its detection. Rigorous testing shows that it can identify slight variations indicating conductor corrosion with over 80% sensitivity and a 5% false alarm rate for ‘medium’ to ‘high’ severity damage. By detecting and resolving corrosion early, the system reduces disruptions, streamlines maintenance, and mitigates unscheduled repairs’ negative effects on the environment. This enhances energy generation effectiveness, promotes hydroelectric facilities’ long-term viability, and fosters community prosperity.

Transfer learning promotes acquisition of individual BCI skills.

Subject training is crucial for acquiring brain-computer interface (BCI) control. Typically, this requires collecting user-specific calibration data due to high inter-subject neural variability that limits the usability of generic decoders. However, calibration is cumbersome and may produce inadequate data for building decoders, especially with naïve subjects. Here, we show that a decoder trained on the data of a single expert is readily transferrable to inexperienced users via domain adaptation techniques allowing calibration-free BCI training. We introduce two real-time frameworks, (i) Generic Recentering (GR) through unsupervised adaptation and (ii) Personally Assisted Recentering (PAR) that extends GR by employing supervised recalibration of the decoder parameters. We evaluated our frameworks on 18 healthy naïve subjects over five online sessions, who operated a customary synchronous bar task with continuous feedback and a more challenging car racing game with asynchronous control and discrete feedback. We show that along with improved task-oriented BCI performance in both tasks, our frameworks promoted subjects' ability to acquire individual BCI skills, as the initial neurophysiological control features of an expert subject evolved and became subject specific. Furthermore, those features were task-specific and were learned in parallel as participants practiced the two tasks in every session. Contrary to previous findings implying that supervised methods lead to improved online BCI control, we observed that longitudinal training coupled with unsupervised domain matching (GR) achieved similar performance to supervised recalibration (PAR). Therefore, our presented frameworks facilitate calibration-free BCIs and have immediate implications for broader populations-such as patients with neurological pathologies-who might struggle to provide suitable initial calibration data.

User Interest Identification with Social Media Information using Natural Language and Meta-Heuristic Technique

As the number of Internet users and social networking apps has grown in recent years, interest-based recommendation systems have been more commonly used in practice. Given the vast quantity of data available from LinkedIn and Twitter, as well as the expanding number of users, it was critical to create a real-time framework for recommending and monitoring relevant tweets or posts based on the user's interests. Using association rules, the interests of social network users can be uncovered. A considerable number of association rules extracted from social networks were found to be mostly dependent on coverage requirements. After finding patterns of frequent and non-frequent patterns, a large number of non-frequent terms were eliminated in association rule mining. In order to reduce the complexity of the association rule mining process, the more relevant terms are selected by the Hybridized Competitive Swarm Optimizer and Gravitational Search Algorithm (CSO-GSA), which is utilized for association rule generation and classification using deep learning techniques. In this research, numerous relevant rules for human interest are identified. The numerical outcome of the proposed strategy is compared with existing state-of-the-art techniques. The proposed CSO with GSA outperforms the existing techniques.

A novel artificial intelligent-based approach for real time prediction of telecom customer’s coming interaction

&lt;p&gt;&lt;span&gt;Predicting customer’s behavior is one of the great challenges and obstacles for business nowadays. Companies take advantage of identifying these future behaviors to optimize business outcomes and create more powerful marketing strategies. This work presents a novel real-time framework that can predict the customer’s next interaction and the time of that interaction (when that interaction takes place). Furthermore, an extensive data exploratory analysis is performed to gain more insights from the data to identify the important features. Transactional data and static profile data are integrated to feed a deep learning model which is implemented using two methodologies: time-series approach and statistical approach. It is found that the time-series approach gives the best performance and fulfills all the requirements. The experiments show that the proposed framework introduces a good overall performance in comparison to existing approaches based on standard metrics like accuracy and mean absolute error (MAE) values. What makes the proposed work novel and special is that it is the first approach that addresses the telecom customer’s next future interaction not just churn prediction like the other approaches in literature.&lt;/span&gt;&lt;/p&gt;

Prediction and quantification of occupational health risk in road construction workers exposed to asphalt emissions

Asphalt emissions, which are released during asphalt pavement construction, pose adverse health effects on construction workers. While previous studies have primarily focused on post hoc health risk assessments, limited research has aimed to predict occupational health risk in advance, a crucial step in providing mitigation recommendations for workers before they begin their tasks. This study endeavors to develop a predictive model for evaluating occupational health risk arising from exposure to asphalt emissions in road construction. The model is grounded in a validated Computational Fluid Dynamics (CFD) simulation approach and accounts for the influence of on-site mitigation measures. Our findings demonstrate that paver operators face the highest health risks, followed by screedmen and rakers. Additionally, engineering controls hold the most significant weight in preventing on-site health risk (weighted at 0.5396), followed by administrative actions (weighted at 0.2970), and personal protective equipment (weighted at 0.1634). Moreover, a decrease in the emission source temperature and an increase in ventilation conditions on-site can reduce the value of workers' health risk. Overall, the model developed in this study can be adapted for diverse construction projects, providing a real-time framework to assess and implement mitigation measures in advance.