Real-time Computation Research Articles

Investigating the efficiency and performance gains of FPGA-accelerated Convolutional Neural Networks

With the rise of big data and artificial intelligence (AI), Convolutional Neural Networks (CNNs) have become instrumental in numerous applications, from image and speech recognition to natural language processing. However, these networks computational demands often exceed the capabilities of traditional processing units, leading to a search for more effective computing platforms. This research aims to evaluate the potential of Field-Programmable Gate Array (FPGA) technology in accelerating CNN computations, considering FPGAs unique attributes such as reprogrammability, energy efficiency, and custom logic potential. The primary aim of this research is to compare the efficiency and performance of FPGA acceleration of CNNs with conventional processing units like CPUs and GPUs and to explore its potential for future AI applications. This research employs a mixed-methods approach, including an integrated literature review and comparative analysis. This paper reviews state-of-the-art research on FPGA-accelerated CNNs, benchmark performance metrics of FPGA, CPU, and GPU platforms across various CNN models, and compare FPGA-based AI applications with other real-world AI applications. The findings suggest a significant potential for FPGA-accelerated CNNs, particularly in scenarios requiring real-time computation or power-limited environments. However, challenges persist in the areas of development complexity and limited on-chip memory. Future work must focus on surmounting these barriers to unlock the full potential of FPGA-accelerated CNNs.

Evaluation on model-driven knowledge graph and platform for grid operation and maintenance

With the continuous expansion of the power grid, the forms of faults are becoming increasingly complex, with a wide range of impacts and long maintenance cycles, posing increasingly severe challenges for power grid operators. In the current power system (PS), due to system limitations, there is a large amount of data, and the current computing system is limited by hardware and computing power, making it difficult to satisfy the requirements of real-time computing, comprehensive analysis, and expansion. This paper analyzed the technical basis of the knowledge graph (KG) of power grid operation and maintenance (O&M), and clarified the key role of model driving in the current large-scale smart grid (SG), which can effectively improve the efficiency and reliability of data processing in the SG. This paper also analyzed the model-driven PG O&M KG platform, and analyzed the role of particle swarm optimization (PSO) algorithm in PG operation and fault maintenance. After applying the PSO algorithm in this article, in the experimental results section, the fault rate of Transformer 5 was 2.1% lower than that of Transformer 4. The knowledge mapping model of grid O&M and the particle swarm algorithm in this paper can significantly reduce the failure rate of the grid, which has wide extension value.

Zhang neural networks: an introduction to predictive computations for discretized time-varying matrix problems

This paper wants to increase our understanding and computational know-how for time-varying matrix problems and Zhang Neural Networks. These neural networks were invented for time or single parameter-varying matrix problems around 2001 in China and almost all of their advances have been made in and most still come from its birthplace. Zhang Neural Network methods have become a backbone for solving discretized sensor driven time-varying matrix problems in real-time, in theory and in on-chip applications for robots, in control theory and other engineering applications in China. They have become the method of choice for many time-varying matrix problems that benefit from or require efficient, accurate and predictive real-time computations. A typical discretized Zhang Neural Network algorithm needs seven distinct steps in its initial set-up. The construction of discretized Zhang Neural Network algorithms starts from a model equation with its associated error equation and the stipulation that the error function decrease exponentially fast. The error function differential equation is then mated with a convergent look-ahead finite difference formula to create a distinctly new multi-step style solver that predicts the future state of the system reliably from current and earlier state and solution data. Matlab codes of discretized Zhang Neural Network algorithms for time varying matrix problems typically consist of one linear equations solve and one recursion of already available data per time step. This makes discretized Zhang Neural network based algorithms highly competitive with ordinary differential equation initial value analytic continuation methods for function given data that are designed to work adaptively. Discretized Zhang Neural Network methods have different characteristics and applicabilities than multi-step ordinary differential equations (ODEs) initial value solvers. These new time-varying matrix methods can solve matrix-given problems from sensor data with constant sampling gaps or from functional equations. To illustrate the adaptability of discretized Zhang Neural Networks and further the understanding of this method, this paper details the seven step set-up process for Zhang Neural Networks and twelve separate time-varying matrix models. It supplies new codes for seven of these. Open problems are mentioned as well as detailed references to recent work on discretized Zhang Neural Networks and time-varying matrix computations. Comparisons are given to standard non-predictive multi-step methods that use initial value problems ODE solvers and analytic continuation methods.

Real-Time Evaluation of the Uncertainty in Weather Forecasts Through Machine Learning-Based Models

Meteorological events have always been of great interest because they have influenced everyday activities in critical areas, such as water resource management systems. Weather forecasts are solved with numerical weather prediction models. However, it sometimes leads to unsatisfactory performance due to the inappropriate setting of the initial state. Precipitation forecasting is essential for water resource management in semi-arid climate and seasonal rainfall areas such as the Ebro basin. This research aims to improve the estimation of the uncertainty associated with real-time precipitation predictions presenting a machine learning-based method to evaluate the uncertainty of a weather forecast obtained by the Weather Research and Forecasting model. We use a model trained with ground-truth data from the Confederación Hidrográfica del Ebro, and WRF forecast results to compute uncertainty. Experimental results show that Decision Tree-based ensemble methods get the lowest generalization error. Prediction models studied have above 90% accuracy, and root mean square error has similar results compared to those obtained with the ground truth data. Random Forest presents a difference of -0.001 concerning the 0.535 obtained with the ground truth data. Generally, using the ML-based model offers good results with robust performance over more traditional forms for uncertainty calculation and an effective alternative for real-time computation.

A Rigid-Flexible Coupling Dynamic Model for Robotic Manta with Flexible Pectoral Fins

The manta ray, exemplifying an agile swimming mode identified as the median and paired fin (MPF) mode, inspired the development of underwater robots. Robotic manta typically comprises a central rigid body and flexible pectoral fins. Flexible fins provide excellent maneuverability. However, due to the complexity of material mechanics and hydrodynamics, its dynamics are rarely studied, which is crucial for the advanced control of robotic manta (such as trajectory tracking, obstacle avoidance, etc.). In this paper, we develop a multibody dynamic model for our novel manta robot by introducing a pseudo-rigid body (PRB) model to consider passive deformation in the spanwise direction of the pectoral fins while avoiding intricate modeling. In addressing the rigid-flexible coupling dynamics between flexible fins and the actuation mechanism, we employ a sequential coupling technique commonly used in fluid-structure interaction (FSI) problems. Numerical examples are provided to validate the MPF mode and demonstrate the effectiveness of the dynamic model. We show that our model performs well in the rigid-flexible coupling analysis of the manta robot. In addition to the straight-swimming scenario, we elucidate the viability of tailoring turning gaits through systematic variations in input parameters. Moreover, compared with finite element and CFD methods, the PRB method has high computational efficiency in rigid-flexible coupling problems. Its potential for real-time computation opens up possibilities for future model-based control.

Detecting IoT Anomalies Using Fuzzy Subspace Clustering Algorithms

There are many applications of anomaly detection in the Internet of Things domain. IoT technology consists of a large number of interconnecting digital devices not only generating huge data continuously but also making real-time computations. Since IoT devices are highly exposed due to the Internet, they frequently meet with the challenges of illegitimate access in the form of intrusions, anomalies, fraud, etc. Identifying these illegitimate accesses can be an exciting research problem. In numerous applications, either fuzzy clustering or rough set theory or both have been successfully employed. As the data generated in IoT domains are high-dimensional, the clustering methods used for lower-dimensional data cannot be efficiently applied. Also, very few methods were proposed for such applications until today with limited efficacies. So, there is a need to address the problem. In this article, mixed approaches consisting of nano topology and fuzzy clustering techniques have been proposed for anomaly detection in the IoT domain. The methods first use nano topology of rough set theory to generate CORE as a subspace and then employ a couple of well-known fuzzy clustering techniques on it for the detection of anomalies. As the anomalies are detected in the lower dimensional space, and fuzzy clustering algorithms are involved in the methods, the performances of the proposed approaches improve comparatively. The effectiveness of the methods is evaluated using time-complexity analysis and experimental studies with a synthetic dataset and a real-life dataset. Experimentally, it has been found that the proposed approaches outperform the traditional fuzzy clustering algorithms in terms of detection rates, accuracy rates, false alarm rates and computation times. Furthermore, nano topological and common Mahalanobis distance-based fuzzy c-means algorithm (NT-CM-FCM) is the best among all traditional or nano topology-based algorithms, as it has accuracy rates of 84.02% and 83.21%, detection rates of 80.54% and 75.37%, and false alarm rates of 7.89% and 9.09% with the KDDCup’99 dataset and Kitsune Network Attack Dataset, respectively.

Development and application of an inexpensive open-source dendrometer for detecting xylem water potential and radial stem growth at high spatial and temporal resolution.

There is currently a need for inexpensive, continuous, non-destructive water potential measurements at high temporal resolution (<1min). We describe here the development and testing of an entirely open-source dendrometer that, when combined with periodic Scholander pressure chamber measurements, provides sub-minute resolution estimates of water potential when placed on tissues exhibiting little or no secondary growth (petioles, monocotyledon stems). The dendrometer can also be used to measure radial growth of stems and branches when placed on dicotyledon and gymnosperm species. The dendrometer can be interfaced directly with a computer in real time in the lab or greenhouse, or connected to a datalogger for long periods of use in the field on batteries. We tested this device on a herbaceous dicotyledon (Helianthus annuus) (petioles and stems) and a monocotyledon (Zea mays) species (stems) for 1 week during dehydration and re-watering treatments under laboratory conditions. We also demonstrated the ability of the device to record branch and trunk diameter variation of a woody dicotyledon (Rhus typhina) in the field. Under laboratory conditions, we compared our device (hereafter 'contact' dendrometer) with modified versions of another open-source dendrometer (the 'optical' dendrometer). Overall, contact and optical dendrometers were well aligned with one another, with Pearson correlation coefficients ranging from 0.77 to 0.97. Both dendrometer devices were well aligned with direct measurements of xylem water potential, with calibration curves exhibiting significant non-linearity, especially at water potentials near the point of incipient plasmolysis, with pseudo R2 values (Efron) ranging from 0.89 to 0.99. Overall, both dendrometers were comparable and provided sufficient resolution to detect subtle differences in stem water potential (ca. 50 kPa) resulting from light-induced changes in transpiration, vapour pressure deficit and drying/wetting soils. All hardware designs, alternative configurations, software and build instructions for the contact dendrometers are provided.

Real-Time Heterogeneous Road-Agents Trajectory Prediction Using Hierarchical Convolutional Networks and Multi-Task Learning

Trajectory prediction of heterogeneous road agents such as vehicles, cyclists, and pedestrians in dense traffic plays an essential role in self-driving. Despite breakthroughs in trajectory prediction technology in recent years, challenges remain in world state representation, social interaction modeling, real-time computing, and road agent heterogeneity. To address these challenges, we propose a new model that employs hierarchical convolutional networks and multi-task learning to predict agents' trajectories. The model first achieves effective and unified representation of agent and scene context by rendering heterogeneous world states in a top-down multi-channel raster map. Based on this representation, we propose hierarchical convolutional networks to extract global interaction and local features of all agents simultaneously, enabling the model to predict multiple agents' trajectories in real-time in a single forward inference. In addition, we specifically design multi-task learning branches with dynamic adaptive anchors to capture differences in behavioral patterns of heterogeneous agents, allowing a single model to accurately predict multimodal trajectories of multi-class agents. Extensive experiments on public nuScenes and Lyft datasets demonstrate top model performance. Importantly, our model is faster (2.2x) and more computationally stable than state-of-the-art models, making it well-suited for mass-produced self-driving systems that require both performance and computational efficiency.

Control of the Probe-Sample Interaction Force at the Piconewton Scale by a Magnetic Microprobe in Aqueous Solutions

This article describes the design and implementation of a control system that controls the probe-sample interaction force at the piconewton scale by manipulating a magnetic microprobe in aqueous solutions under a microscope. The control system has two functions, namely accurate force generation by magnetic flux control and precise control of the probe-sample interaction force. Magnetic flux control uses an improved six-input-six-output digital control law along with a disturbance estimator to achieve two control objectives. First, the magnetic flux at each pole tip of a previously developed hexapole electromagnetic actuator can be individually and precisely controlled at frequencies over 4 kHz, and the experimental results precisely followed the theoretical predictions based on the design. Second, together with the optimized flux allocation, the flux control system generates precise magnetic forces, making the six-input hexapole actuator behave like a decoupled three-axis force generator with a bandwidth of more than 4 kHz. Interaction force control compares the measured deformation of the sample with the expected deformation, updated in real time by a deformation predictor, to generate the magnetic force to control the interaction between the probe and the sample. Digital control laws, the estimator, and the predictor are all implemented using a high-speed Field Programmable Gate Array (FPGA) system. Experiments confirm that through mathematical modeling, digital control technology, high-speed electronics, and real-time computation, accurate three-dimensional force generation at the piconewton scale with high bandwidth and precise interaction force control with zero-mean random error, attributed to random thermal force and measurement noise, are achieved.

Study of active food processing technology using computer vision and AI in coffee roasting.

In the modern food processing industry, which is more complex than in the past, it is important to utilize real-time computer vision for active food processing technology using artificial intelligence. An integrated solution of computer vision and Deep Learning (DL) technology provides quality control and optimization of food processing in complex environments with obstacles. In this study, Coffee Bean Classification Model (CBCM) made by Machine Learning (ML) showed excellent performance, accurately distinguishing coffee beans through the avoidance of obstacles and empty spaces inside a rotating roasting machine. CBCM achieved a maximum validation accuracy of 98.44% and a minimum validation loss of 5.40% after the fifth epoch. Using a test dataset of 137 samples, CBCM achieved an accuracy of 99.27% and a loss of 2.82%. The developed solution using the CBCM was able to quantify the color change of the coffee beans during roasting.

Influence phase of a dS observer. Part I. Scalar exchange

Inspired by real-time computations in AdS black holes, we propose a method to obtain the influence phase of a cosmological observer by calculating the on-shell action on a doubled spacetime geometry. The influence phase is the effective action for an open system: for a dS static patch observer coupled to a scalar field it incorporates the radiation reaction due to the bulk fields and their dS Hawking radiation. For a general extended source in dS, we describe how to account for finite size effects. In the long-time limit, we get a Markovian open quantum system susceptible to cosmological fluctuations, whereas the short-time limit reproduces the worldline theory of flat-space radiation reaction. We also present a fully covariantised form for the cubic corrections to the radiation reaction in even spacetime dimensions, including Hubble contributions, and find an intriguing recursive structure across dimensions.

SAM-SFM: High-Efficiency and High-Resolution Tandem Mass Spectrometry Enabled by Sinusoidal Amplitude Modulation of Multiple Sinusoidal Frequency-Modulated Waveforms.

In miniature ion trap mass spectrometry, achieving a balance between isolation resolution and efficiency is a formidable challenge. The presence of absorption curves causes target ions to inadvertently absorb energy from AC signal components near their resonant frequencies. To mitigate this issue, SAM-SFM waveforms introduce a parameter known as the decreasing factor. Unlike SWIFT waveforms, SAM-SFM's spectral profile intentionally departs from a rectangular window, adopting an arch-shaped excitation window to minimize the impact on target ions and improve ion isolation efficiency. SAM-SFM waveforms have the advantage of low computational complexity, enabling real-time computation using an embedded FPGA technology. Regardless of any parameter changes, the FPGA can consistently guarantee waveform output within 1 μs. This not only enhances throughput but also eliminates the need for a PC in miniature mass spectrometry devices. The performance of SAM-SFM has been validated on an improved "Brick" miniature ion trap mass spectrometer. Comparative experiments with SWIFT waveforms confirm the lossless unit-mass isolation capabilities of SAM-SFM. This waveform has the capability to simultaneously isolate multiple target ions, even allowing for the lossless isolation of ions with lower abundance within isotopic clusters, albeit at the cost of requiring extended isolation durations.

Introduction to the Special Issue on Real-Time Computing in the IoT-to-Edge-to-Cloud Continuum

Introduction to the Special Issue on Real-Time Computing in the IoT-to-Edge-to-Cloud Continuum

An offloading method in new energy recharging based on GT-DQN

The utilization of green edge has emerged as a promising paradigm for the development of new energy vehicle (NEV). Nevertheless, the recharging of these vehicles poses a significant challenge in due to limited power resources and enormous transmission demands. A novel architecture based on Wifi-6 communication is proposed, which makes the most of heterogeneous edge nodes to achieve real-time processing and computation of tasks. To address the collaborative power resource optimization problem, the interference between different vehicles is considered, and the task offloading is optimized. In particular, the power contention among recharging clusters is modeled as an exact game and a task offloading strategy model is proposed jointly with the Deep Q-Network (DQN) algorithm, which is employed by a secondary application. Thereby, the recharging efficiency and task offloading computation are optimized and improved. Results indicate that the total resource consumption is favorably improved with this architecture and algorithm and the Nash equilibrium is also demonstrated.

Real-time nonstandard-shaped gold fiducial marker tracking on x-ray fluoroscopic images for prostate radiotherapy

Objective. The prostate moves in accordance with the movement of surrounding organs. Tumor position can change by ≥3 mm during radiotherapy. Given the difficulties of visualizing the prostate fluoroscopically, fiducial markers are generally implanted into the prostate to monitor its motion during treatment. Recently, internally motion guidance methods of the prostate using a 99.5% gold/0.5% iron flexible notched wire fiducial marker (Gold Anchor® , Naslund Medical AB, Huddinge, Sweden), which requires a 22 gauge needle, has been used. However, because the notched wire can retain its linear shape, acquire a spiral shape, or roll into an irregular ball, detecting it on fluoroscopic images in real-time incurs higher computation costs. Approach. We developed a fiducial tracking algorithm to achieve real-time computation. The marker is detected on the first image frame using a shape filter that employs inter-class variance for the marker likelihood calculated by the filter, focusing on the large difference in densities between the marker and its surroundings. After the second frame, the marker is tracked by adding to the shape filter the similarity to the template cropped from the area around the marker position detected in the first frame. We retrospectively evaluated the algorithm’s marker tracking accuracy for ten prostate cases, analyzing two fractions in each case. Main results. Tracking positional accuracy averaged over all patients was 0.13 ± 0.04 mm (mean ± standard deviation, Euclidean distance) and 0.25 ± 0.09 mm (95th percentile). Computation time was 2.82 ± 0.20 ms/frame averaged over all frames. Significance. Our algorithm successfully and stably tracked irregularly-shaped markers in real time.

Physically accurate real-time synthesis of ocean waves for maritime simulators

The most efficient approach to the synthesis of multi-directional ocean waves is the so-called Fourier approach, based on a spectral description of waves in wavenumber space. According to this approach, 2D wavenumber spectra are transformed into spatial wave quantities by means of 2D Inverse Discrete Fourier Transforms. The Fourier approach is well described in literature, but the majority of works dealing with it are focused on graphical realism, and usually neglect the physical accuracy of the synthesized ocean scene. This paper presents a new implementation of the Fourier approach which exploits the equivalence between wavenumber and frequency/direction spectra to result in a flexible, performant and physically accurate model that is especially suitable for maritime simulators. In the proposed method, input wave spectra expressed in frequency/direction space are mapped at runtime on the wavenumber spectra used for the ocean synthesis. A GPU implementation based on the multi-band approach allows highly efficient real-time computations. The performance of the method is described in detail, investigating different configurations of the wavenumber spectrum and comparing the results obtained on different hardware. The physical accuracy is assessed by recording time series of synthesized wave elevations and using them to reconstruct a directional wave spectrum. Four benchmark spectra are examined, highlighting a very satisfactory match between the input and the reconstructed spectra, even for complex sea states with multiple wave systems originated from different sources.

Chip-In-Loop SNN Proxy Learning: a new method for efficient training of spiking neural networks.

The primary approaches used to train spiking neural networks (SNNs) involve either training artificial neural networks (ANNs) first and then transforming them into SNNs, or directly training SNNs using surrogate gradient techniques. Nevertheless, both of these methods encounter a shared challenge: they rely on frame-based methodologies, where asynchronous events are gathered into synchronous frames for computation. This strays from the authentic asynchronous, event-driven nature of SNNs, resulting in notable performance degradation when deploying the trained models on SNN simulators or hardware chips for real-time asynchronous computation. To eliminate this performance degradation, we propose a hardware-based SNN proxy learning method that is called Chip-In-Loop SNN Proxy Learning (CIL-SPL). This approach effectively eliminates the performance degradation caused by the mismatch between synchronous and asynchronous computations. To demonstrate the effectiveness of our method, we trained models using public datasets such as N-MNIST and tested them on the SNN simulator or hardware chip, comparing our results to those classical training methods.

Metaversal 6G: Deciphering Complex Requirements and Multivariate KPIs in High-Performance Computing

As the next frontier in wireless technology, 6G represents the sixth generation of mobile communication. Expected to surpass the capabilities of 5G, 6G aims to provide unprecedented data speeds, ultra-low latency, and enhanced connectivity, unlocking new possibilities for advanced technologies like augmented reality, artificial intelligence, and the Internet of Things. It intertwines seamlessly with the Metaverse, a digital universe where users can interact with computer-generated environments, objects, and other users in real-time high-performance computing. This research attempts to explore the transformative domain of Metaverse integration in 6G Networks, delving into the potential landscape with a keen focus on emerging prerequisites, key performance indicators (KPIs), and future opportunities in the HPC environment. The integration of 6G and the metaverse signifies a fundamental shift in connectivity and user experiences, offering assurances of heightened data speeds, decreased latency, and inventive applications. Within this framework, the article introduces the all-encompassing structure of 6G Cloudification and Cloud Native Technologies, highlighting the critical role played by sophisticated cloud technologies in shaping the evolution of the metaverse ensuring the scalability, flexibility, and efficiency required to support the dynamic nature of immersive digital experiences.

A Real-time Light-weight Computer Vision Application for Driver’s Drowsiness Detection

A Real-time Light-weight Computer Vision Application for Driver’s Drowsiness Detection

Spark Distributed Real-Time Data and GPU Parallel Computing Based on 5G Virtual Reality

5G virtual reality has attracted many manufacturers and users with its unique immersion, interactivity and imagination characteristics, which has become the focus of new markets. The purpose of this article is to use Spark distributed real-time data system and GPU parallel computing to quickly process and analyze data. This article mainly designs a general-purpose real-time data analysis and processing system based on Spark, which mainly includes new ETL and real-time processing engine modules, and is committed to achieving higher real-time performance than traditional Hadoop. And realize fast calculation. At the same time there is universality and stability. Includes real-time flow calculations. Fast batch processing and machine learning The various types of data computers are included in this article by preparing the cutting device and adjusting the cutting output. The device is ready to effectively terminate the CUDA environment. The cudamalloc function is used to allocate a linear space of bytes to the device, and then transfer the data from the host to the device to determine the number of GPU blocks and threads. GPU parallel computing can increase the data processing speed by 27%, while the secondary programming algorithm can reduce the optimization time of the cup by 12%.