High Real-time Performance Research Articles

Pedestrian Crossing Sensing Based on Hough Space Analysis to Support Visually Impaired Pedestrians.

There are many visually impaired people globally, and it is important to support their ability to walk independently. Acoustic signals and escort zones have been installed on pedestrian crossings for the visually impaired people to walk safely; however, pedestrian accidents, including those involving the visually impaired, continue to occur. Therefore, to realize safe walking for the visually impaired on pedestrian crossings, we present an automatic sensing method for pedestrian crossings using images from cameras attached to them. Because the white rectangular stripes that mark pedestrian crossings are aligned, we focused on the edges of these rectangular stripes and proposed a novel pedestrian crossing sensing method based on the dispersion of the slope of a straight line in Hough space. Our proposed method possesses unique characteristics that allow it to effectively handle challenging scenarios that traditional methods struggle with. It excels at detecting crosswalks even in low-light conditions during nighttime when illumination levels may vary. Moreover, it can detect crosswalks even when certain areas are partially obscured by objects or obstructions. By minimizing computational costs, our method achieves high real-time performance, ensuring efficient and timely crosswalk detection in real-world environments. Specifically, our proposed method demonstrates an impressive accuracy rate of 98.47%. Additionally, the algorithm can be executed at almost real-time speeds (approximately 10.5 fps) using a Jetson Nano small-type computer, showcasing its suitability as a wearable device.

Multi-Head multimodal deep interest recommendation network

From machine learning recommendation to deep learning recommendation, reinforcement learning recommendation, and recommendation model compression, the network structure of the recommendation algorithm becomes more complex, more expressive, and more lightweight with higher real-time performance. However, most of these models are dedicated to recommendations by mining ratings and text information, and researchers have optimized the network structures of many recommendation models. However, mining image information and enriching recommendation model features are less studied, and there is still some room for optimization of deep recommendation model structures and real-time performance. To this end, this paper proposes a model called MMDIN by adding multi-head and multi-modal modules to the DIN model. The multi-modal module extracts image features, which enriches the feature set that the model can use and strengthens the cross-combination and fitting expression capabilities of the model. Meanwhile, the multi-head mechanism extracts features from different dimensions, which further enhances the model prediction performance. Experimental results show that the proposed MMDIN model improves the recommendation prediction performance, and it outperforms state-of-the-art methods in comprehensive indicators.

A geostationary orbit microwave multi-channel radiometer

The geostationary orbit microwave multi-channel radiometer has the advantages of high real-time performance and large coverage, which plays an important role in typhoon, strong precipitation detection, and medium-to-short-term meteorological/oceanic forecasting. However, due to the difficulty in engineering development of the payload, its application on-orbit has not yet been achieved at present. To satisfy the requirements of fine and quantitative application of satellite observation data, a geostationary orbit microwave multi-channel radiometer with a 10-m-caliber is developed, in which the spatial resolution at horizontal polarization is better than 24 km at 54 GHz. In geostationary orbit microwave multi-channel radiometer, a quasi-optical feed network covering nearly 28 frequency octave bands and ranging from 23.8 to 664 GHz is proposed to solve the technical problem of multi-frequency sharing in the system. Meanwhile, a high-precision reflector preparation method and a high-precision unfolding scheme are proposed, which are considered as a solution for the large-diameter reflector with a high maintaining surface accuracy. A high-precision antenna prototype with 0.54-m is developed, and the tests are performed to verify the key technologies, such as the preparation of high-precision grating reflectors at the micron level, high surface accuracy detection, and sub-millimeter wave antenna electrical performance testing. The results indicate that measured main beam efficiency of the 664 GHz antenna is better than 95.5%. In addition, the system sensitivity is greater than 1.5 K, and the calibration accuracy is better than 1.8 K, according to the results of an analysis of the multi-channel radiometer’s essential parameters and calibration errors.

Offline economic dispatch for multi-area power system via hierarchical reinforcement learning

In this paper, a novel hierarchical reinforcement learning (HRL) architecture is put forward to resolve the multi-area economic dispatch (MAED) problem in a fully off-line optimization manner, while the focus of MAED is to minimize the total generation cost of the entire multi-area power system by determining output power within area and the power transfer among areas. The original MAED problem is transformed into an augmented sub-MAED problem by exploring the relationship between power balance constraint and tie line capacity constraint in each area. This architecture first calculates the scheduled area load in the top layer through the primal–dual consensus-based distributed Q-learning, and then calculates load-power table assembled by the optimal output power within single area according to the multi-stage decision algorithm in the bottom layer. Acting as an underlying framework, the assigned load-power table is implemented with the combination of off-line calculation and on-line inquiry. It is capable of parallel and asynchronous optimization in each area while satisfying various constraints. Furthermore, a power-triggered algorithm is proposed to satisfy the power balance among areas, and its convergence is theoretically analyzed, which illustrates that the area power error will converge to zero in the limited iteration. Finally, the proposed approach is validated in three different cases: 20-unit, 40-unit, and 120-unit. The comparative results confirmed the superiority of the proposed HRL architecture with high real-time performance and less computation burden. Specifically, the HRL architecture reduces the fuel cost in the range of 0.01656%–0.19173% and accelerates the speed more than 10 times than others.

Real-time fire detection algorithms running on small embedded devices based on MobileNetV3 and YOLOv4

AimFires are a serious threat to people’s lives and property. Detecting fires quickly and effectively and extinguishing them in the nascent stage is an effective way to reduce fire hazards. Currently, deep learning-based fire detection algorithms are usually deployed on the PC side.MethodsAfter migrating to small embedded devices, the accuracy and speed of recognition are degraded due to the lack of computing power. In this paper, we propose a real-time fire detection algorithm based on MobileNetV3-large and yolov4, replacing CSP Darknet53 in yolov4 with MobileNetV3-large to achieve the initial extraction of flame and smoke features while greatly reducing the computational effort of the network structure. A path connecting PANet was explored on Gbneck(104, 104, 24), while SPP was embedded in the path from MobileNetV3 to PANet to improve the feature extraction capability for small targets; the PANet in yolo4 was improved by combining the BiFPN path fusion method, and the improved PANet further improved the feature extraction capability; the Vision Transformer model is added to the backbone feature extraction network and PANet of the YOLOv4 model to give full play to the model’s multi-headed attention mechanism for pre-processing image features; adding ECA Net to the head network of yolo4 improves the overall recognition performance of the network.ResultThese algorithms run well on PC and reach 95.14% recognition accuracy on the public dataset BoWFire. Finally, these algorithms were migrated to the Jeston Xavier NX platform, and the entire network was quantized and accelerated with the TensorRT algorithm. With the image propagation function of the fire robot, the overall recognition frame rate can reach about 26.13 with high real-time performance while maintaining a high recognition accuracy.ConclusionSeveral comparative experiments have also validated the effectiveness of this paper’s improvements to the YOLOv4 algorithm and the superiority of these structures. With the effective integration of these components, the algorithm shows high accuracy and real-time performance.

A Wavelet Transform-Based Real-Time Filtering Algorithm for Fusion Magnet Power Signals and Its Implementation

With high real-time performance and a good filtering effect, the wavelet transform-based real-time filter is more commonly used in the fusion magnet power supply. Considering the signal characteristics of the fusion magnet power supply, the sliding time window method is widely used to satisfy the requirements of filtering efficiency. In this paper, an updated half-edge extension method is proposed to improve the boundary effect of sliding window data. Moreover, the hardware-based wavelet loopback structure is proposed to avoid the double computation of long sliding window data, thereby improving real-time performance and satisfying the demands of fusion magnet power supply filtering. The proposed half-edge extension method and the wavelet loopback structure are implemented by designed FPGA and experimented on the Experimental and Advanced Superconducting Tokamak (EAST) fusion magnet power supply system to verify the filtering efficiency. According to the experiment’s results, the proposed half-edge extension method has a better filtering effect, and the proposed wavelet loopback structure decreases the time by 67.4%, which can satisfy the requirements of the power supply system.

A novel method for intelligent analysis of rice yield traits based on LED transmission imaging and cloud computing

Rice yield traits extraction is necessary for rice breeding and genetic analysis, in which the discrimination of filled and unfilled grain was the bottleneck. The traditional methods including wind separating and water sinking are inefficient, unreliable, and poorly reproducible, meanwhile the machine vision methods based on X-ray and 3D structured imaging have the disadvantages of high cost, ionizing radiation, and poor scalability. To address the above problems, this study has proposed a novel method for intelligent analysis of rice yield traits based on LED transmission imaging and cloud application. The study has found that the red-light LED transmission imaging was able to obtain high-contrast images of filled and unfilled grains and had the advantages of radiation-free, low-cost, and high-efficiency imaging. Using 3,200 rice grain images as the training set, the results showed that the YOLOv5 model outperformed the Faster RCNN, RetinaNet, SSD, and Cascade RCNN models, and its filled and unfilled grain recognition precision was 98.94% and 90.96% on the 800 test sets, while the recall rate was 97.91% and 94.94%. To further improve the recognition accuracy of the YOLOv5 model, five model optimization methods were adopted in the study, including TPH, CBAM, adding a small target detection layer, using Bi-FPN and Multi-NMS. The results proved that the accuracy of unfilled grain recognition had significantly improved, the recognition precision increased by 4.54% to 95.44%, and the recall rate increased by 0.77% to 95.71%. Finally, this study has designed the specified algorithms based on the recognized rectangular box, which could accurately measure filled and unfilled grain numbers, seed setting rate and grain shape parameters. Moreover, the study has designed a local Qt software and a cloud-based website, which can support the simultaneous input of multiple images with high real-time performance and good scalability, providing a practical novel tool for rice breeding and genetic analysis.

Voltage sag source location based on sag event amplitude information

Voltage sag source location provides a solution to improve the operation and maintenance efficiency and resolve disputes between power supply and consumption. The existing sag source locating methods are based on the fault recording system’s sampling value waveform data, which requires more storage space and transmission channels. Based on sag event list with easy access and high real-time performance and takes the line as the location result, this paper proposes a voltage sag source locating method based on sag event amplitude information. Firstly, Based on the voltage amplitude of the monitoring nodes recorded in the sag event list, a matrix of voltage amplitude of disturbed monitoring nodes is formed. Secondly, the voltage amplitude and fault type of the disturbed nodes are used as inputs, and the line number is used as a classification number. The multi-classifier model based on multi-layer perceptron is trained to locate the voltage sag source. Finally, the method in this paper is verified by IEEE39 node simulation model and the actual monitoring data of sag events in an area in East China. The results show that the method is effective and feasible. Compared with the existing commonly used multi classification algorithms, it is found that the accuracy of sag source location in this method is 10% ∼ 20% higher than other multi classification algorithms.

A Scalable Distributed Control Algorithm for Bearing-Only Passive UAV Formation Maintenance.

Unmanned Aerial Vehicles (UAVs) can cooperate through formations to perform tasks. Wireless communication allows UAVs to exchange information, but for the situations requiring high security, electromagnetic silence is needed to avoid potential threats. The passive UAV formation maintenance strategies can fulfill the requirement of electromagnetic silence at the cost of heavy real-time computing and precise locations of UAVs. To pursue high real-time performance without the localization of UAVs, this paper proposes a scalable distributed control algorithm for bearing-only passive UAV formation maintenance. By minimizing necessary communication, pure angle information is applied to maintain UAV formations through distributed control, without the knowledge of the UAVs' precise locations. The convergency of the proposed algorithm is proven strictly and the converging radius is derived. Through simulation, the proposed algorithm is proven to be suitable for a general case and demonstrates fast convergence speed, strong anti-interference capability, and high scalability.

Biologically Inspired Machine Learning-Based Trajectory Analysis in Intelligent Dispatching Energy Storage System

The present work expects to explore the application effect of biologically inspired Plasticity Neural Network in the industrial intelligent dispatching energy storage system, and highlight the intelligence and fault detection performance of the control system. To address the faults in intelligent dispatching energy storage system, the present work implements a fault diagnosis model of intelligent dispatching energy storage system based on Deep Belief Network (DBN), and simulates and analyzes the model. The results show that the transmission probability of the fault diagnosis model of the constructed intelligent energy storage scheduling system is 100% and when the parameters <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\lambda $ </tex-math></inline-formula> is between 0.01 and 0.05, the real-time performance of data transmission is the highest. Compared with other classical algorithm models, the success rate and detection accuracy of the proposed algorithm are about 85%, the energy consumption is lower, and the detection effect is more obvious. Therefore, the constructed system obviously has higher real-time performance and more accurate fault detection performance, and significantly better system detection and protection performance. The results provide an experimental basis for the operation and fault detection of intelligent dispatching energy storage system.

An INN-Based Solution for the Underdetermined Image Reconstruction Problem in EIT

Electrical impedance tomography (EIT) has been widely concerned in online nondestructive testing of carbon fiber-reinforced polymers (CFRPs) due to its significant advantages of high real-time performance, zero radiation, simple structure, and low cost. However, its inverse problem is highly underdetermined and nonlinear, thus limiting its imaging quality and application scope. Based on the architecture of an invertible neural network (INN), an INN image reconstruction algorithm with latent output variables is proposed in this article. With latent output variables, the information lost during training is reused, and a unique input can be determined by sampling latent output variables and predicted outputs. In addition, the forward and reverse processes are combined together to weaken the dependency of effective output and latent output variables, reduce the errors between input and output domains, and solve the underdetermination and nonlinearity problems of EIT. The simulation and experimental results showed that compared with the traditional algorithms and the convolution neural network (CNN) network algorithm, the INN can accurately reconstruct the damage location, yield a clearer damaged edge, and reduce artifacts. Noise-added simulations and prototype experiments showed that the INN achieves both good imaging quality and enhanced noise immunity, to improve the performance of the EIT system in CFRP laminate inspection.

APDP: Attribute-Based Personalized Differential Privacy Data Publishing Scheme for Social Networks

In the Big Data era, the wide usage of mobile devices has led to large amounts of information release and sharing through social networks, where sensitive information of the data owners may be leaked. Traditional approaches that provide the identical privacy protection levels for all users result in poor quality of service, thus the concept of personalized privacy has been proposed in recent years. However, existing methods that add different noises to each user will require both high real-time performance and resource consumption. This paper presents a fine-grained personalized differential privacy data publishing scheme (APDP) for social networks. Specifically, we design a new mechanism that defines the privacy protection levels of different users based on their attribute values. In particular, we exploit the TOPSIS method to map the attribute values to the amount of noise required to add. Furthermore, to prevent illegal download of data, the access control is integrated with differential privacy. Compared with traditional attribute-based encryption data publishing schemes, our scheme can get rid of the expensive computation overhead. Theoretical analyses and simulations show that APDP can realize efficient personalized differential privacy data publishing with reasonable data utility.

A comprehensive and up-to-date web-based interactive 3D emergency response and visualization system using Cesium Digital Earth: taking landslide disaster as an example

ABSTRACT As with the fast advances in the technologies of big Earth data and information communication, Web-based 3D GIS system has come a long way from a few years ago. These advances reflect in many aspects of 3D GIS such as higher real-time performance, enhanced interactivity, more realistic 3D visualization effect and improved user interface. This paper aims to present a comprehensive and up-to-date 3D Web GIS for Emergency Response using the current vue.js web application framework and the well-known Cesium API, taking landslide disaster as an example. Building upon recent advances in WebGL technology, we developed a suite of enhanced 3D spatial analysis functions, including interactive route planning, instant text/image/video messaging being incorporated into both 3D WebGL page and mobile GIS applications, and progressive 3D construction and AR visualization using LiDAR and camera over local emergency network or internet. Moreover, professional functions such as landslide susceptibility mapping, landslide monitoring, spatial temporal contingency plan management, landslide information management, personnel and equipment management, and communication are all implemented and integrated in the 3D GIS system. Most of the functions of the system are implemented using open-source projects, which is beneficial to the development of the 3D GIS research community.

Research on key technologies of multi-core DSP in smart substation redundancy network testing

Aiming at the network characteristics of a smart substation redundant network with complex communication messages, a large amount of interactive data, and high real-time performance, the test model of multiple submachines in a High-Availability Seamless Redundancy (HSR) loop network is analyzed. The requirements of packet types, size, and transmission speed are evaluated, and a hardware solution based on a multi-core Digital Signal Processor (DSP) and Parallel architecture FPGA is designed. The multi-core interaction strategy of DSP software and the multi-port high-speed communication method of FPGA are proposed, and the inter-core communication (IPC) interrupts and shared memory to achieve fast data migration between cores, based on the Serial RapidIO (SRIO) to complete high-speed message transmission. The throughput, delay, packet loss rate, and error frame detection functions are realized, tested, and verified, providing theoretical and practical support for the evaluation of redundant network performance of smart substations.

Multiobjective Reinforcement Learning-Based Neural Architecture Search for Efficient Portrait Parsing.

This article dedicates to automatically explore efficient portrait parsing models that are easily deployed in edge computing or terminal devices. In the interest of the tradeoff between the resource cost and performance, we design the multiobjective reinforcement learning (RL)-based neural architecture search (NAS) scheme, which comprehensively balances the accuracy, parameters, FLOPs, and inference latency. Finally, under varying hyperparameter configurations, the search procedure emits a bunch of excellent objective-oriented architectures. The combination of two-stage training with precomputing and memory-resident feature maps effectively reduces the time consumption of the RL-based NAS method, so that we complete approximately 1000 search iterations in two GPU days. To accelerate the convergence of the lightweight candidate architecture, we incorporate knowledge distillation into the training of the search process. This also provides a reasonable evaluation signal to the RL controller that enables it to converge well. In the end, we conduct full training with outstanding Pareto-optimal architectures, so that a series of excellent portrait parsing models (with only approximately 0.3M parameters) is received. Furthermore, we directly transfer the architectures searched on CelebAMask-HQ (Portrait Parsing) to other portrait and face segmentation tasks. Finally, we achieve the state-of-the-art performance of 96.5% MIOU on EG1800 (portrait segmentation) and 91.6% overall F1 -score on HELEN (face labeling). That is, our models significantly surpass the artificial network on the accuracy, but with lower resource consumption and higher real-time performance.

Time-optimal trajectory planning based on event-trigger and conditional proportional control.

Trajectory planning is an important issue for manipulators and robots. To get a optimal trajectory, many constraints including actuators specifications, motion range of joints, workspace limitations, etc, and many objectives including the shortest time, the shortest distance, the lowest energy consumption, the minimum oscillations, obstacles-avoiding, etc, should be considered both. In this paper, firstly, the forward kinematics and inverse kinematics of a five axis manipulator are deduced. And, a simple method to choose one appropriate solution from multi solutions of inverse kinematics is proposed. Secondly, an easy-implemented optimization method of trajectory planning is proposed based on seventh order polynomial interpolation, event-trigger mechanism and conditional proportional control (P control). The proposed optimization method can capture the time optimal trajectory, and the actuators specifications including velocity, acceleration of motor can be guaranteed as well. Thirdly, comparative simulations and experiments validate the effectiveness and efficiency of proposed optimization method. The research provides an insight for the application of trajectory optimization on the micro controller with low computing capability and high real-time performance requirement.

Side-Channel Attack of Lightweight Cryptography Based on MixColumn: Case Study of PRINCE

Lightweight cryptography is implemented in unrolled architecture generally, which has the characteristics of low latency and high real-time performance but also faces the threat of Side-Channel Attack (SCA). Different from traditional loop architecture, the unrolled architecture requires separate protection against SCA in each round. This leads to the unrolled architecture that is very sensitive to the number of rounds that need to be protected against SCA. In this paper, we propose an optimized method for the chosen-input attack that can effectively increase the number of rounds of differential propagation and recover the key from the fourth round of unrolled PRINCE for the first time. This research also evaluates the hardware overhead and performance of two types of Threshold implementation (TI) for PRINCE. The experimental results indicate that TI imposes substantial hardware overhead on the circuit, therefore a specified number of protection rounds is required.

Flame smoke detection algorithm based on YOLOv5 in petrochemical plant

PurposeFire smoke detection in petrochemical plant can prevent fire and ensure production safety and life safety. The purpose of this paper is to solve the problem of missed detection and false detection in flame smoke detection under complex factory background.Design/methodology/approachThis paper presents a flame smoke detection algorithm based on YOLOv5. The target regression loss function (CIoU) is used to improve the missed detection and false detection in target detection and improve the model detection performance. The improved activation function avoids gradient disappearance to maintain high real-time performance of the algorithm. Data enhancement technology is used to enhance the ability of the network to extract features and improve the accuracy of the model for small target detection.FindingsBased on the actual situation of flame smoke, the loss function and activation function of YOLOv5 model are improved. Based on the improved YOLOv5 model, a flame smoke detection algorithm with generalization performance is established. The improved model is compared with SSD and YOLOv4-tiny. The accuracy of the improved YOLOv5 model can reach 99.5%, which achieves a more accurate detection effect on flame smoke. The improved network model is superior to the existing methods in running time and accuracy.Originality/valueAiming at the actual particularity of flame smoke detection, an improved flame smoke detection network model based on YOLOv5 is established. The purpose of optimizing the model is achieved by improving the loss function, and the activation function with stronger nonlinear ability is combined to avoid over-fitting of the network. This method is helpful to improve the problems of missed detection and false detection in flame smoke detection and can be further extended to pedestrian target detection and vehicle running recognition.

Detection algorithm for bearing roller end surface defects based on improved YOLOv5n and image fusion

For the current problems of low accuracy and poor reliability of defect detection for bearing roller end surfaces in industrial production, this paper proposes a bearing roller end surface defect detection algorithm based on improved YOLOv5n and the fusion of gamma-corrected maps and curvature maps. First, this paper uses photometric stereo vision to reconstruct the three-dimensional shape of the surface and proposes an improved Frankot–Chellappa integration algorithm to solve the problem of reconstructing surface deformation. Secondly, the DenseFuse network is used to fuse gamma-corrected maps and curvature maps to generate an image dataset that combines the strengths of both images to enhance defect features and improve the precision of target detection. Finally, the improved target detection model YOLOv5n is proposed to detect defects in the end surfaces of bearing rollers. The experimental results show that by using fused images for training, detection models with higher mean average precision(mAP) than traditional images can be obtained, and the improved YOLOv5n algorithm maintains the high real-time performance of the original algorithm while the mAP0.5 and mAP0.5:0.95 of improved YOLOv5n are 98.6% and 87.4%, respectively, which are respectively 0.9% and 2.8% higher than YOLOv5n.

AD-YOLOv5s based UAV detection for low altitude security

UAV (Unmanned Aerial Vehicle) black flight at low altitude could cause serious safety risks. Consequently, it is crucial to detect and manage low altitude small UAVs. The existing methods of low altitude small UAV detection suffer from problems such as high false alarm rate, and poor real-time performance. In order to solve the above problems, we present a novel approach, named AD-YOLOv5s, to achieve low altitude small UAV detection with high precision and high real-time performance. Firstly, the feature enhancement method is used to expand the dataset. We optimize the model feature fusion, the prediction head structure, and the loss function. Based on the CBAM (Convolutional Block Attention Module) attention mechanism, feature enhancement is performed to improve the detection accuracy. Secondly, the ghost module and depthwise separable convolution are used to reduce the number of parameters of the model, and we propose the method of lightweight design of model to improve the detection speed. Compared with the YOLOv5s model, the experiment result shows that our proposed AD-YOLOv5s model improves the value of mAP by 2.2% and the value of Recall by 1.8%, reduces the value of GFLOPs by 29.9% and parameters by 38.8%, and achieves 27.6 FPS when the proposed model deploy on a low-cost edge computing device (jetson nano).