Real-time Algorithm Research Articles

Instant processing of large-scale image data with FACT, a real-time cell segmentation and tracking algorithm

Quantifying cellular characteristics from a large heterogeneous population is essential to identify rare, disease-driving cells. A recent development in the combination of high-throughput screening microscopy with single-cell profiling provides an unprecedented opportunity to decipher disease-driving phenotypes. Accurately and instantly processing large amounts of image data, however, remains a technical challenge when an analysis output is required minutes after data acquisition. Here, we present fast and accurate real-time cell tracking (FACT). FACT can segment ∼20,000 cells in an average of 2.5s (1.9-93.5 times faster than the state of the art). It can export quantifiable features minutes after data acquisition (independent of the number of acquired image frames) with an average of 90%-96% precision. We apply FACT to identify directionally migrating glioblastoma cells with 96% precision and irregular cell lineages from a 24h movie with an average F1 score of 0.91.

A general visual-servoing control strategy for active non-cooperative target tracking of space manipulators

Non-cooperative target tracking plays an important role in many space missions. Recent studies on non-cooperative target tracking are mostly concentrated in the field of passive target tracking. Passive non-cooperative target tracking can effectively be used in the initial phases of a close-range rendezvous manoeuvre. Instead, during the terminal phases of such manoeuvres, active object tracking(AOT) techniques are more convenient. This study propose a general visual servo strategy for the active non-cooperative target tracking of space manipulator. Our proposed method do not need any a priori information about the targets and can be used for space manipulators of any configuration and binocular cameras of any types. Our strategy includes a real-time and accurate position estimation algorithm for non-cooperative targets and a motion planning algorithm combined Position-based visual servo(PBVS) and Image-based visual servo(IBVS). The experiment verified that the proposed strategy well solved the active non-cooperative target tracking task and can be used as a baseline for solving such tracking problems which provides a reference for subsequent studies.

Real-Time Road Intersection Detection in Sparse Point Cloud Based on Augmented Viewpoints Beam Model.

Road intersection is a kind of important navigation landmark, while existing detection methods exhibit clear limitations in terms of their robustness and efficiency. A real-time algorithm for road intersection detection and location in large-scale sparse point clouds is proposed in this paper. Different from traditional approaches, our method establishes the augmented viewpoints beam model to perceive the road bifurcation structure. Explicitly, the spatial features from point clouds are jointly extracted in various viewpoints in front of the robot. In addition, the evaluation metrics are designed to self-assess the quality of detection results, enabling our method to optimize the detection process in real time. Considering the scarcity of datasets for intersection detection, we also collect and annotate a VLP-16 point cloud dataset specifically for intersections, called NCP-Intersection. Quantitative and qualitative experiments demonstrate that the proposed method performs favorably against the other parallel methods. Specifically, our method performs an average precision exceeding 90% and an average processing time of approximately 88 ms/frame.

Understanding and optimizing fish counting techniques based on electrical impedance measurements.

Sustainable aquatic resources management requires reliable methods for fish detection in various environmental conditions. Herein, we study fundamental mechanisms underlying the application of electrical impedance measurements in this regard. We present results of experimental studies conducted in laboratory conditions using a low-cost impedance measurement circuit, as well as the corresponding numerical models. We also present evaluation results of a newly developed, real-time detection algorithm based on adaptive thresholding. The numerical model was validated by extracting fish tracks in 3D space from the experimental datasets, and then comparing the calculated versus measured impedance values as functions of fish coordinates in time. Numerical predictions closely resemble the experimental data. The detection sensitivity and specificity values determined for various settings exceeded 90%. Electrode width to spacing ratio is demonstrated to be a crucial parameter influencing the system sensitivity distribution. The introduced approach can constitute a framework for designing electrical impedance-based fish counting systems.

Research on the Real-Time Ambiguity Resolution Algorithm of GPS/Galileo/BDS Based on CNES Real-Time Products

Real-Time (RT) Precise Point Positioning (PPP) uses precise satellite orbits and clock corrections, and employs a separate receiver for positioning. With the growing demand, RT PPP is becoming an increasingly popular research topic. The ambiguity resolution (AR) can significantly improve the positioning accuracy and convergence time of PPP, so it is essential to study PPPAR in RT mode. In this paper, 37 MGEX stations from around the world are chosen, and the RT orbit, clock, and phase biases products broadcast by the Centre National d’Etudes Spatiales (CNES) are applied to PPPAR. Additionally, the residuals of the RT phase biases products, convergence time, and positioning accuracy are investigated. The results indicate that GPS products have the best quality of AR, with wide-lane (WL) and narrow-lane (NL) residuals of 98.9% and 95.3%, respectively, within ±0.25 cycles. Within ±0.25 cycles, the WL and NL residuals of the Galileo are 98.2% and 94.3%, respectively. Within ±0.25 cycles, the (Beidou Navigation Satellite System) BDS has a poor quality of AR, with WL and NL residuals of 97.3% and 73.1%, respectively. Due to the poor quality of the BDS AR, the convergence time of the BDS is not calculated in this paper. The convergence time of other systems is significantly reduced after AR processing, and the convergence time of the GPS/Galileo combination is the fastest, being 17.14 min in kinematic mode and only 11.85 min in static mode. The positioning accuracy of the GPS, Galileo, GPS/Galileo, and GPS/Galileo/BDS in the E and U directions is significantly improved after PPPAR.

Energy Consumption Auditing Anomaly Detection System of Robotic Manipulators based on a Generative Adversarial Network

Unexpected anomalies pose significant risks to the health and security of intelligent manufacturing systems. This paper proposes a generative adversarial network (GAN)-based anomaly detection framework specifically for monitoring robotic manipulator operation using a side-channel energy auditing mechanism. To tackle the limitation arising from the lack of labeled data, the GAN model is trained by a semi-supervised learning approach that identifies anomalies during online operations as outliers. The overfitting is purposely utilized during the model training to enlarge the difference between normal energy consumption patterns used for training and anomalous profiles in real-time testing. In addition, the GAN model is modified to use multiple discriminators to analyze the individual energy profile associated with each joint or motor. The anomaly is detected by evaluating the mean and standard deviation values of anomaly scores' distribution, and both values are continuously updated by Welford's algorithm in real time to take into account the effect of environmental variations during operations. The detection performance on our custom dataset demonstrates the feasibility of the proposed pipeline. Specifically, for physical attacks, the framework can achieve an accuracy of approximately 0.93 for instant-wise detection and 0.84 for event-wise detection.

Mobility Rehab visual feedback system for gait rehabilitation in older adults

BackgroundGait and balance impairments are among the main causes of falls in older adults. The feasibility and effectiveness of adding sensor-based feedback to physical therapy (PT) in an outpatient PT setting is unknown. We evaluated the feasibility and effectiveness of PT intervention combined with a therapist-assisted visual feedback system, called Mobility Rehab, (PT + MR) in older adults.MethodsTwenty-eight older adults with and without neurological diseases were assigned either PT + MR (n = 22) or PT alone (n = 6). Both groups performed 8 sessions (individualized) of 45 min long (30 min for gait training and 15 min for endurance, strength, and balance exercises) in an outpatient clinic. Mobility Rehab uses unobtrusive, inertial sensors on both wrists and feet, and at the sternum level with real-time algorithms to provide real-time feedback on five gait metrics (step duration, stride length, elevation at mid-swing, arm swing range-of-motion [ROM], and trunk coronal ROM), which are displayed on a tablet. The primary outcome was the Activities-specific Balance Confidence scale (ABC). The secondary outcome was gait speed measured with wearable inertial sensors during 2 min of walking.ResultsThere were no between-group differences at baseline for any variable (P > 0.05). Neither PT + MR nor PT alone showed significant changes on the ABC scores. PT + MR, but not PT alone, showed significant improvements in gait speed and arm swing ROM. The system was evaluated as ‘easy to use’ by the PT.ConclusionsOur preliminary results show that PT + MR improves gait speed in older adults with and without neurological diseases in an outpatient clinic.Clinical Trial Registrationwww.ClinicalTrials.gov, identifier: NCT03869879.

Comparison and analysis of deep learning models for vehicle re-identification

The rapid urbanization and increasing number of vehicles on the roads demand efficient and accurate vehicle re-identification (Re-ID) techniques for intelligent transportation systems, traffic monitoring, and surveillance applications. This paper offers a detailed analysis of deep learning approaches to vehicle Re-ID, covering feature learning, attention mechanism, unsupervised learning, self-supervised learning, and specialized loss function. The efficiency of these methods is assessed using VeRi-776 and VehicleID datasets. Key metrics, such as mean Average Precision (mAP) and Rank-n accuracy, are employed to gauge their success. Results show that feature learning, attention mechanism, and specialized loss function play pivotal roles in achieving high performance in vehicle Re-ID tasks, with unsupervised and self-supervised learning methods displaying potential for practical applications due to their scalability. The paper also highlights several challenges, including enhancing the interpretability of attention mechanisms, exploring the relationship between popular loss functions, and addressing the infeasibility of existing methods for real-time applications. The paper concludes with several recommendations for the prospects of vehicle Re-ID, including developing advanced real-time algorithms, enhancing deep learning techniques, investigating innovative approaches, and addressing existing challenges. These proposed advances could significantly improve the efficacy of feature learning and spark fresh innovations in this field.

Rapid estimation of electroporation-dependent tissue properties in canine lung tumors using a deep neural network

The efficiency of electroporation treatments depends on the application of a critical electric field over the targeted tissue volume. Both the electric field and temperature distribution strongly depend on the tissue-specific electrical properties, which both differ between patients in healthy and malignant tissues and change in an electric field-dependent manner from the electroporation process itself. Therefore, tissue property estimations are paramount for treatment planning with electroporation therapies. Ex vivo methods to find electrical tissue properties often misrepresent the targeted tissue, especially when translating results to tumors. A voltage ramp is an in situ method that applies a series of increasing electric potentials across treatment electrodes and measures the resulting current. Here, we develop a robust deep neural network, trained on finite element model simulations, to directly predict tissue properties from a measured voltage ramp. There was minimal test error (R2>0.94;p<0.0001) in three important electric tissue properties. Further, our model was validated to correctly predict the complete dynamic conductivity curve in a previously characterized ex vivo liver model (R2>0.93;p<0.0001) within 100 s from probe insertion, showing great utility for a clinical application. Lastly, we characterize the first reported electrical tissue properties of lung tumors from five canine patients (R2>0.99;p<0.0001). We believe this platform can be incorporated prior to treatment to quickly ascertain patient-specific tissue properties required for electroporation treatment planning models or real-time treatment prediction algorithms. Further, this method can be used over traditional ex vivo methods for in situ tissue characterization with clinically relevant geometries.

Inertial Measurement Unit-Based Real-Time Adaptive Algorithm for Human Walking Pattern and Gait Event Detection

In this work, a lightweight adaptive hybrid gait detection method with two inertial measurement units (IMUs) on the foot and thigh was developed and preliminarily evaluated. An adaptive detection algorithm is used to eliminate the pre-training phase and to modify parameters according to the changes within a walking trial using an adaptive two-level architecture. The present algorithm has a two-layer structure: a real-time detection algorithm for detecting the current gait pattern and events at 100 Hz., and a short-time online training layer for updating the parameters of gait models for each gait pattern. Three typical walking patterns, including level-ground walking (LGW), stair ascent (SA), and stair descent (SD), and four events/sub-phases of each pattern, can be detected on a portable Raspberry-Pi platform with two IMUs on the thigh and foot in real-time. A preliminary algorithm test was implemented with healthy subjects in common indoor corridors and stairs. The results showed that the on-board model training and event decoding processes took 20 ms and 1 ms, respectively. Motion detection accuracy was 97.8% for LGW, 95.6% for SA, and 97.1% for SD. F1-scores for event detection were over 0.86, and the maximum time delay was steadily below 51 ± 32.4 ms. Some of the events in gait models of SA and SD seemed to be correlated with knee extension and flexion. Given the simple and convenient hardware requirements, this method is suitable for knee assistive device applications.

INTELLIGENT AGRICULTURAL PEST MANAGER DRONE IN PAKISTAN

This paper’s primary goal is to develop an agriculture drone for spraying pesticides. We discuss an architecture based on unmanned aerial vehicles (UAVs) in this study. Pesticides must be used in agriculture if the quality of large-scale output is to be maintained. It is crucial to increase agriculture's production and efficiency by employing cutting-edge technology to replace employees with intelligent equipment like robots. The research suggests a novel method to replace people in a number of agricultural tasks, including the identification of insect presence, the application of pesticides and fertilizers, etc. The created method entails building a prototype that makes use of basic, affordable equipment including a microprocessor, different motors, and terminal equipment to assist farmers in a variety of operations related to crop fields. Design and build an autonomous drone-based surveillance system capable of identifying injured crops and spraying pesticides in specified regions as needed. To combine an image processing and Artificial Intelligence based real time algorithm to determine crop health and evaluate the need for pesticides, as well as a weather monitoring system that can assist anticipate weather conditions

Walking-Speed-Adaptive Gait Phase Estimation for Wearable Robots.

This paper introduces a Gait Phase Estimation Module (GPEM) and its real-time algorithm designed to estimate gait phases continuously and monotonically across a range of walking speeds and accelerations/decelerations. To address the challenges of real-world applications, we propose a speed-adaptive online gait phase estimation algorithm, which enables precise estimation of gait phases during both constant speed locomotion and dynamic speed changes. Experimental verification demonstrates that the proposed method offers smooth, continuous, and repetitive gait phase estimation when compared to conventional approaches such as the phase portrait method and time-based estimation. The proposed method achieved a 48% reduction in gait phase deviation compared to time-based estimation and a 48.29% reduction compared to the phase portrait method. The proposed algorithm is integrated within the GPEM, allowing for its versatile application in controlling gait assistive robots without incurring additional computational burden. The results of this study contribute to the development of robust and efficient gait phase estimation techniques for various robotic applications.

Transforming unmanned pineapple picking with spatio-temporal convolutional neural networks

Automated pineapple harvesting has emerged as a prominent prospective development within the agricultural domain. Nevertheless, the intricate growth conditions that pineapples encounter in the field, such as inadequate light, overexposure, obstructions caused by fruit leaves, or the overlapping of fruits, pose substantial challenges to the accuracy and robustness of traditional real-time detection algorithms. In recent times, Transformer models, when applied to computer vision, have exhibited commendable performance, underscoring their potential for target detection in smart agricultural applications. In this report, we propose a spatio-temporal convolutional neural network model that leverages the shifted window Transformer fusion region convolutional neural network model for the purpose of detecting pineapple fruits. Our study includes a comparative analysis of these results and those obtained through the utilization of conventional models. Additionally, we investigate the influence of various aspects of data preparation, including image resolution, object size, and object complexity, on the ultimate pineapple detection outcomes. Experimental findings elucidate that, in the case of detecting a single-category target like a pineapple, the employment of 2000 annotated supervised data points yields the optimal detection accuracy. Further augmenting the size of the training dataset does not yield any significant improvement in detection accuracy. Furthermore, images of pineapples captured from greater distances encompass smaller targets and an increased number of pineapple instances, rendering them more intricate and challenging to accurately detect. In summary, our study employs the spatio-temporal convolutional neural network model to attain pineapple detection with an impressive accuracy rate of 92.54% and an average inference time of 0.163 s, thus affirming the efficacy of our developed model in achieving superior detection results.

Real-time navigation solutions of low-cost off-the-shelf GNSS receivers on board the Astrocast constellation satellites

With the ever-increasing number of small satellites launched into space, on-board orbit determination is becoming more and more important. Precise satellite positions are needed for constellation maintenance, formation flying, communication optimization, collision avoidance and Earth observation. In addition, accurate time information is required for the functioning of the instruments on board. Low-cost off-the-shelf Global Navigation Satellite System (GNSS) receivers and antennas, adapted to the small size and weight of the satellite, allow precise on-board orbit determination and time synchronization for even the smallest satellites.We analysed more than one year of on-board navigation solutions (NAVSOL) from 8 Astrocast CubeSats, which are equipped with our multi-GNSS single-frequency payload. Based on the residuals from a dynamic orbit fitting, we found that the quality of the NAVSOL from the Astrocast satellites varies strongly both over time and between satellites, with an RMS ranging from a few meters to several tens of meters.In this study, we focus on three main effects that influence the quality of the NAVSOL from the Astrocast satellites: (1) The orbit-fit residuals show a positive radial offset in the range of about 6 m to 15 m, which is caused by ionospheric refraction. Our data show that GNSS signals at negative elevations are very strongly affected by the ionospheric refraction and thus account for the major part of the radial offset. (2) Discrepancies between the NAVSOL positions and velocities, and simulations with a GNSS signal generator show a once-per-revolution periodicity in the out-of-plane component, with amplitudes of 2 m and 5 cm/s in position and velocity, respectively. This effect is probably due to an inappropriate dynamic model used in the real-time navigation algorithm of the GNSS receiver. (3) Depending on the location of the GNSS antenna on board the satellite, the noise level varies by a multiple. This is assumed to be due to the antenna orientations, interference of the GNSS signals with electromagnetic waves coming from the satellite platform, signal attenuation during the transmission from the antenna to the receiver and differences in the soldering or tuning of the individual GNSS antennas.These findings help us refine hardware configurations and firmware settings in preparation for future Astrocast satellites or other nanosatellite missions. By eliminating or minimizing the mentioned effects, we expect to consistently achieve an accuracy on the level of a few meters for the real-time navigation solution of commercial off-the-shelf single-frequency GNSS receivers in space.

Real-Time Online Goal Recognition in Continuous Domains via Deep Reinforcement Learning.

The problem of goal recognition involves inferring the high-level task goals of an agent based on observations of its behavior in an environment. Current methods for achieving this task rely on offline comparison inference of observed behavior in discrete environments, which presents several challenges. First, accurately modeling the behavior of the observed agent requires significant computational resources. Second, continuous simulation environments cannot be accurately recognized using existing methods. Finally, real-time computing power is required to infer the likelihood of each potential goal. In this paper, we propose an advanced and efficient real-time online goal recognition algorithm based on deep reinforcement learning in continuous domains. By leveraging the offline modeling of the observed agent's behavior with deep reinforcement learning, our algorithm achieves real-time goal recognition. We evaluate the algorithm's online goal recognition accuracy and stability in continuous simulation environments under communication constraints.

Real-time eye blink detection using general cameras: a facial landmarks approach

Eyes are essential in Human-Computer Interaction (HCI) as they provide valuable insights into a person's thoughts and intentions. However, current eye movement analysis methods require specialized equipment or high-quality videos, making them less accessible and usable. This paper proposes a real-time eye blink detection algorithm that uses standard cameras, making it widely applicable and convenient. The approach achieves accurate and efficient eye blink detection by leveraging the Eye Aspect Ratio (EAR) algorithm and facial landmarks technique. This paper developed a Python application and conducted tests using a laptop webcam to validate the performance and practicality of the algorithm across various settings. The algorithm's effectiveness depends on carefully tuning parameters such as threshold and frame rate. The results demonstrate the algorithm's potential in real-time eye blink detection, with potential applications in drowsiness detection during driving, prevention of Computer Vision Syndrome, and assistance for individuals with disabilities. By enabling eye blink detection using commonly available cameras, the algorithm paves the way for integrating eye movement analysis into everyday devices and systems, enhancing user experience and enabling more natural interactions. Further refinement and optimization of this approach hold promise for a wide range of applications in HCI, healthcare, and beyond, opening up new possibilities for research and innovation.

Corrigendum: Water Supply 1 May 2023; 23 (5): 2197–2209. Evaluation on real-time self-organization algorithm of forest cultivation, tea planting, and tea processing for sustainable ecosystem management of agricultural water resources. Zhenhong Wang, Liyong Zhang, Minfeng Song, Sangjie Luobu, Jiumei, Danzhen, Hong Peng

Corrigendum: <i>Water Supply</i> 1 May 2023; 23 (5): 2197–2209. Evaluation on real-time self-organization algorithm of forest cultivation, tea planting, and tea processing for sustainable ecosystem management of agricultural water resources. Zhenhong Wang, Liyong Zhang, Minfeng Song, Sangjie Luobu, Jiumei, Danzhen, Hong Peng

3D reconstruction of the scapula from biplanar X-rays for pose estimation and morphological analysis.

Patient-specific scapular shape in functional posture can be highly relevant to clinical research. Biplanar radiography is a relevant modality for that purpose with already two existing assessment methods. However, they are either time-consuming or lack accuracy. The aim of this study was to propose a new, more user-friendly and accurate method to determine scapular shape. The proposed method relied on simplified manual inputs and an upgraded version of the first 3D estimate based on statistical inferences and Moving-Least Square (MLS) deformation of a template. Then, manual adjustments, with real-time MLS algorithm and contour matching adjustments with an adapted minimal path method, were added to improve the match between the projected 3D model and the radiographic contours. The accuracy and reproducibility of the method were assessed (with 6 and 12 subjects, respectively). The shape accuracy was in average under 2mm (1.3mm in the glenoid region). The reproducibility study on the clinical parameters found intra-observer 95% confidence intervals under 3mm or 3° for all parameters, except for glenoid inclination and Critical Shoulder Angle, ranging between 3° and 6°. This method is a first step towards an accurate reconstruction of the scapula to assess clinical parameters in a functional posture. This can already be used in clinical research on non-pathologic bones to investigate the scapulothoracic joint in functional position.

A Real-Time Image Encryption Algorithm for a Distributed Energy System Based on the 13-D Complex Chaotic Sequence

A Real-Time Image Encryption Algorithm for a Distributed Energy System Based on the 13-D Complex Chaotic Sequence

Subjective speech enhancement performance of a hearable with transparency and spatial frequency-weighted dereverberation

Preserving natural binaural cues combined with directionality and robust dereverberation is crucial for successful sound enhancement via hearables. In this study, we subjectively evaluate the performance of a real-time hearable prototype that aims to enhance frontal speech and spatial cues in complex environments using a new dereverberation algorithm and acoustic transparency. Our real-time algorithm exploits the spatial frequency-weighted coherence-to-diffuse ratio estimated between the conchal microphones and produces directional binaural gains using a Wiener filter for sound enhancement. The new hearable system can be customized or adapted to a given environment for optimum performance in different frequency bands and preserves natural binaural cues for all individuals by symmetrical spatial filtering. We use the real-time prototype to evaluate the perceived speech intelligibility in noise by a Matrix framework for normal-hearing subjects in a reverberant classroom. We also use a multi-stimulus with hidden reference and anchor-like framework to evaluate the perceived speech quality and listening comfort of several stimuli simulated for HATS in the same environment. Results show that speech intelligibility and listening comfort in noise can be significantly improved for the real-time hearable using the new algorithm with optimized parameters compared with several state-of-the-art coherence-based algorithms implemented in a similar device.