Real-time Processing Requirements Research Articles

Welding Groove Edge Detection Method Using Lightweight Fusion Model Based on Transfer Learning

Groove edge detection is the prerequisite for weld seam deviation identification. A welding groove edge detection method based on transfer learning is presented as a solution to the inaccuracy of the conventional image processing method for extracting the edge of the welding groove. DenseNet and MobileNetV2 are used as feature extractors for transfer learning. Dense-Mobile Net is constructed using the skip connections structure and depthwise separable convolution. The Dense-Mobile Net training procedure consists of two stages: pre-training and model fusion fine-tuning. Experiments demonstrate that the proposed model accurately detects groove edges in MAG welding images. Using MIG welding images and the Pascal VOC2012 dataset to evaluate the generalization ability of the model, the relevant indicators are greater than those of Support Vector Machine (SVM), Fully Convolutional Networks (FCN), and UNet. The average single-frame detection time of the proposed model is 0.14 s, which meets the requirements of industrial real-time performance.

Self-Interference Suppression of Unmanned Underwater Vehicle with Vector Hydrophone Array Based on an Improved Autoencoder

The self-interference of an unmanned underwater vehicle (UUV) weakens its ability to detect targets of interest. Due to limitations in the size of the sonar array and the complexity of the interference, the performance of existing self-interference suppression methods in practical applications is unsatisfactory. Our research focuses on analyzing the influence of near-field interferences on the sample covariance matrix (SCM) and proposes an interference suppression algorithm based on an improved autoencoder. The proposed algorithm effectively learns the feature distribution of near-field interferences within the covariance domain and reconstructs the pure signal covariance matrix through the cancellation of the near-field interference features. Moreover, the proposed algorithm can meet the requirements of real-time processing and does not require prior knowledge about the positions or propagation of interference. Simulations demonstrate that the proposed algorithm outperforms comparison methods, particularly in scenarios with low signal-to-interference ratios and a limited number of sensors. Furthermore, lake experiments provide additional evidence of the proposed algorithm’s good performance in practical applications.

Edge Computing on IoT for Machine Signal Processing and Fault Diagnosis: A Review

Edge computing is an emerging paradigm that offloads the computations and analytics workloads onto the internet of things (IoT) edge devices to accelerate the computation efficiency, reduce the channel occupation of signal transmission, and reduce the storage and computation workloads on the cloud servers. These distinct merits make it a promising tool for IoT-based machine signal processing and fault diagnosis. This article reviews the edge computing methods in signal processing-based machine fault diagnosis from the aspects of concepts, state-of-the-art methods, case studies, and research prospects. In particular, the light-weight designed algorithms and application-specific hardware platforms of edge computing in the typical fault diagnosis procedures including signal acquisition, signal preprocessing, feature extraction, and pattern recognition are reviewed and discussed in detail. The review provides an insight into the edge computing framework, methods, and applications, so as to meet the requirements of IoT-based machine real-time signal processing, low-latency fault diagnosis, and high-efficient predictive maintenances.

A Novel Underwater Image Enhancement Using Optimal Composite Backbone Network.

Continuous exploration of the ocean has made underwater image processing an important research field, and plenty of CNN (convolutional neural network)-based underwater image enhancement methods have emerged over time. However, the feature-learning ability of existing CNN-based underwater image enhancement is limited. The networks were designed to be complicated or embed other algorithms for better results, which cannot simultaneously meet the requirements of suitable underwater image enhancement effects and real-time performance. Although the composite backbone network (CBNet) was introduced in underwater image enhancement, we proposed OECBNet (optimal underwater image-enhancing composite backbone network) to obtain a better enhancement effect and shorten the running time. Herein, a comprehensive study of different composite architectures in an underwater image enhancement network was carried out by comparing the number of backbones, connection strategies, pruning strategies for composite backbones, and auxiliary losses. Then, a CBNet with optimal performance was obtained. Finally, cross-sectional research of the obtained network with the state-of-the-art underwater enhancement network was performed. The experiments showed that our optimized composite backbone network achieved better-enhanced images than those of existing CNN-based methods.

Mining belt foreign body detection method based on YOLOv4_GECA model

In the process of mining belt transportation, various foreign objects may appear, which will have a great impact on the crusher and belt, thus affecting production progress and causing serious safety accidents. Therefore, it is important to detect foreign objects in the early stages of intrusion in mining belt conveyor systems. To solve this problem, the YOLOv4_GECA method is proposed in this paper. Firstly, the GECA attention module is added to establish the YOLOv4_GECA foreign object detection model in the mineral belt to enhance the foreign object feature extraction capability. Secondly, based on this model, the learning rate decay of restart cosine annealing is used to improve the foreign object image detection performance of the model. Finally, we collected belt transport image information from the Pai Shan Lou gold mine site in Shenyang and established a belt foreign body detection dataset. The experimental results show that the average detection accuracy of the YOLOv4_GECA method proposed in this paper is 90.1%, the recall rate is 90.7%, and the average detection time is 30 ms, which meets the requirements for detection accuracy and real-time performance at the mine belt transportation site.

A Rolling Real-Time Correction Method for Minute Precipitation Forecast Based on Weather Radars

The quantitative precipitation estimation by weather radar plays an important role in observations and forecasts of meteorological processes. The National Minute Quantitative Precipitation Forecast system of China (MQPF), providing location-based refined short-term and imminent precipitation forecasting services, filled the gap in the official minute precipitation service products in China’s meteorological field. However, due to the technical limitations of radar itself and the complexity of the atmosphere, the corresponding relationship between radar echoes and surface precipitation is unstable. Based on radar and precipitation data from meteorological stations, a rolling real-time correction method is proposed to improve precipitation prediction accuracy through rolling correction of spatial and temporal structural errors in MQPF products. The results show the following: (1) Although this method may lead to a certain increase in the missing ratio, the significant improvement in the false alarm ratio after rolling correction has a positive guiding effect on short-term public meteorological services. (2) Regarding the time to complete rolling correction, the longest and shortest times appear in April and December, respectively. The mean running time to achieve correction of spatial and temporal error corrections ranges from 3.8 s to 6.4 s and 7.7 s to 11.5 s, respectively, which fully meets the real-time operational requirements of radar business.

The Ultrasound Signal Processing Based on High-Performance CORDIC Algorithm and Radial Artery Imaging Implementation

The radial artery reflects the largest amount of physiological and pathological information about the human body. However, ultrasound signal processing involves a large number of complex functions, and traditional digital signal processing can hardly meet the requirements of real-time processing of ultrasound data. The research aims to improve computational accuracy and reduce the hardware complexity of ultrasound signal processing systems. Firstly, this paper proposes to apply the coordinate rotation digital computer (CORDIC) algorithm to the whole radial artery ultrasound signal processing, combines the signal processing characteristics of each sub-module, and designs the dynamic filtering module based on the radix-4 CORDIC algorithm, the quadrature demodulation module based on the partitioned-hybrid CORDIC algorithm, and the dynamic range transformation module based on the improved scale-free CORDIC algorithm. A digital radial artery ultrasound imaging system was then built to verify the accuracy of the three sub-modules. The simulation results show that the use of the high-performance CORDIC algorithm can improve the accuracy of data processing. This provides a new idea for the real-time processing of ultrasound signals. Finally, radial artery ultrasound data were collected from 20 volunteers using different probe scanning modes at three reference positions. The vessel diameter measurements were averaged to verify the reliability of the CORDIC algorithm for radial artery ultrasound imaging, which has practical application value for computer-aided clinical diagnosis.

Unmanned aerial vehicle visual scene understanding based on multitask learning network

With the continuous development of intelligent unmanned aerial vehicles reconnaissance, single-object detection or semantic segmentation can no longer meet the diversified requirements, while simple model stacking will cause the model to be too complex, which will seriously affect the real-time running effect. We propose a visual scene understanding algorithm based on a multitask learning network with an encoder–decoder structure. First, the efficient classification network VoVNet is selected as the feature-sharing network to obtain multiscale coding features. Second, based on the one-stage anchor-free object detector, the feature screening supplementary module implements feature interaction with segmented characters to enhance the detection capability of potential targets. Then for semantic segmentation and depth estimation pixel-level classification tasks, a cascaded chained residual pooling module is used as a parameter-sharing decoder, through the parameter-sharing mechanism to reduce the repeated decoding process to ensure the running speed. Finally, to improve the generalization ability of the model, a general-purpose dataset was constructed and network training was carried out based on the idea of knowledge distillation. Experiments on the dataset show that the performance of the multitask learning network can reach the mainstream algorithm level in semantic segmentation and depth estimation branches, whereas the object detection branch is better in multitarget recall rate, and the average time of each subtask is 39 ms, which meets the requirements of real-time performance and proves the effectiveness of the multitask network.

Multiagent Multimodal Trajectory Prediction in Urban Traffic Scenarios Using a Neural Network-Based Solution

Trajectory prediction in urban scenarios is critical for high-level automated driving systems. However, this task is associated with many challenges. On the one hand, a scene typically includes different traffic participants, such as vehicles, buses, pedestrians, and cyclists, which may behave differently. On the other hand, an agent may have multiple plausible future trajectories based on complex interactions with the other agents. To address these challenges, we propose a multiagent, multimodal trajectory prediction method based on neural networks, which encodes past motion information, group context, and road context to estimate future trajectories by learning from the interactions of the agents. At inference time, multiple realistic future trajectories are predicted. Our solution is based on an encoder–decoder architecture that can handle a variable number of traffic participants. It uses vectors of agent features as inputs rather than images, and it is designed to run on a physical autonomous car, addressing the real-time operation requirements. We evaluate the method using the inD dataset for each type of traffic participant and provide information about its integration into an actual self-driving car.

An Optical Remote Sensing Image Matching Method Based on the Simple and Stable Feature Database

Satellite remote sensing has entered the era of big data due to the increase in the number of remote sensing satellites and imaging modes. This presents significant challenges for the processing of remote sensing systems and will result in extremely high real-time data processing requirements. The effective and reliable geometric positioning of remote sensing images is the foundation of remote sensing applications. In this paper, we propose an optical remote sensing image matching method based on a simple stable feature database. This method entails building the stable feature database, extracting local invariant features that are comparatively stable from remote sensing images using an iterative matching strategy, and storing useful information about the features. Without reference images, the feature database-based matching approach potentially saves storage space for reference data while increasing image processing speed. To evaluate the performance of the feature database matching method, we train the feature database with various local invariant feature algorithms on different time phases of Gaofen-2 (GF-2) images. Furthermore, we carried out matching comparison experiments with various satellite images to confirm the viability and stability of the feature database-based matching method. In comparison with direct matching using the classical feature algorithm, the feature database-based matching method in this paper can essentially improve the correct rate of feature point matching by more than 30% and reduce the matching time by more than 40%. This method improves the accuracy and timeliness of image matching, potentially solves the problem of large storage space occupied by the reference data, and has great potential for fast matching of optical remote sensing images.

EnsembleVehicleDet: Detection of Faraway Vehicles with Real-Time Consideration

While detecting surrounding vehicles in autonomous driving is possible with advances in object detection using deep learning, there are cases where small vehicles are not being detected accurately. Additionally, real-time processing requirements must be met for implementation in autonomous vehicles. However, detection accuracy and execution speed have an inversely proportional relationship. To improve the accuracy–speed tradeoff, this study proposes an ensemble method. An input image is downsampled first, and the vehicle detection result is acquired for the downsampled image through an object detector. Then, warping or upsampling is performed on the Region of Interest (RoI) where the small vehicles are located, and the small vehicle detection result is acquired for the transformed image through another object detector. If the input image is downsampled, the effect on the detection accuracy of large vehicles is minimal, but the effect on the detection accuracy of small vehicles is significant. Therefore, the detection accuracy of small vehicles can be improved by increasing the pixel sizes of small vehicles in the transformed image more than the given input image. To validate the proposed method’s efficiency, the experiment was conducted with Argoverse vehicle data used in an autonomous vehicle contest, and the accuracy–speed tradeoff improved by up to a factor of two using the proposed ensemble method.

A Shallow Neural Network for Recognition of Strip Steel Surface Defects Based on Attention Mechanism

This research proposes an efficient strip steel surface defect classification model (ASNet) based on convolutional neural network (CNN), which can run in real time on commonly used serial computing platforms. We only used a very shallow CNN structure to extract features of the defect images, and an attention layer which makes the model ignore some irrelevant noise and obtain an effective description of the defects is designed. In addition, a nonlinear perceptron is added to the top of the model to recognize defects based on the extracted features. On the strip steel surface defect image dataset NEU-CLS, our model achieves an average classification accuracy of 99.9%, while the number of parameters of the model is only 0.041 M and the computational complexity of the model is 98.1 M FLOPs. It can meet the requirements of real-time operation and large-scale deployment on a common serial computing platform with high recognition accuracy.

Research Challenges for Energy-Efficient Computing in Automated Vehicles

Fully automated retail vehicles will have stringent, real-time operational safety, sensing, communication, inference, planning, and control requirements. Meeting them with existing technologies will impose prohibitive energy-provisioning and thermal management requirements. This article summarizes the research challenges facing the designers of computationally energy-efficient, economical, safe, and reliable automated retail vehicles.

앙상블 기반 자동차 탐지 정확도-속도 트레이드오프 개선

While detecting surrounding vehicles in autonomous driving is possible with advances of object detection using deep learning, there are cases where small-sized vehicles not being detected. Also, real-time processing requirement should be satisfied to be implemented into autonomous vehicles. However, detection accuracy and execution speed have inverse proportion relationship. To improve the accuracy-speed tradeoff, this study proposes an ensemble method. An input image is down sampled first, and vehicle detection result is acquired for the down sampled image through an object detector. Then, perspective transformation or upsampling is performed on RoI(Region of Interest) where the small-sized vehicles are located, and small-sized vehicle detection result is acquired for the transformed image through another object detector. To validate the proposed method efficiency, the experiment was conducted with Argoverse vehicle data used in autonomous vehicle contest and the accuracy-speed tradeoff was shown to improve by up to 44% through the proposed ensemble method.

Fast Resolution Enhancement for Real Beam Mapping Using the Parallel Iterative Deconvolution Method

Super-resolution methods for real beam mapping (RBM) imagery play a significant role in many microwave remote sensing applications. However, the existing super-resolution methods require high-dimensional matrix operations in the case of wide-field imaging, which makes it difficult to satisfy the requirements of real-time signal processing. To solve this problem, this paper introduces an improved Poisson distribution-based maximum likelihood (IPML) method by adding an adaptive iterative acceleration factor to effectively improve the algorithm convergence speed without introducing high-dimensional matrix operations. Furthermore, a GPU-based parallel processing architecture is proposed through the multithreading characteristics of the computing platform, and a cooperative CPU–GPU working model is constructed. This can realize the parallel optimization of the echo reception, preprocessing, and super-resolution processing. We verify that the proposed parallel super-resolution method can significantly improve the computational efficiency without sacrificing performance, using a real dataset.

Lightweight Pedestrian Detection Based on Feature Multiplexed Residual Network

As an important part of autonomous driving intelligence perception, pedestrian detection has high requirements for parameter size, real-time, and model performance. Firstly, a novel multiplexed connection residual block is proposed to construct the lightweight network for improving the ability to extract pedestrian features. Secondly, the lightweight scalable attention module is investigated to expand the local perceptual field of the model based on dilated convolution that can maintain the most important feature channels. Finally, we verify the proposed model on the Caltech pedestrian dataset and BDD 100 K datasets. The results show that the proposed method is superior to existing lightweight pedestrian detection methods in terms of model size and detection performance.

Penetration Overload Prediction Method Based on a Deep Neural Network with Multiple Inputs

In the process of high-speed penetration, penetrating ammunition is prone to problems such as penetration overload signal vibration and mixings and projectile attitude deflection. It is easy to misjudge if a fuze relies only on the overload data from the ground or the utilized program, and the actual penetration overload measured under actual launch conditions cannot be taken as the dynamic judgement basis. Therefore, a real-time penetration overload prediction method based on a deep neural network is proposed, which can predict overload values according to the projectile parameter settings, the real-time collection of overload information, and the calculation speed and assist the fuze in judging the target layer and projectile attitude. In this paper, we adopt a deep learning model with multiple time series inputs and modify the input coding mode so that the model can output a 48 μs overload curve within 20 μs, meeting the real-time signal processing requirements of the high-speed missile penetration process. The mean squared error between the predicted curve and the actual curve is 0.221 for the prediction of multilayer penetrating targets and 0.452 for the prediction of thick penetrating targets. A penetration overload prediction function can be realized.

Anomaly detection based on lightweight deep learning algorithms

The safe and reliable operation of power transmission lines is the key to the sustainable and stable operation of power grids. Using the artificial intelligence techniques for the channel status monitoring greatly improves the inspection efficiency of transmission lines. However, the discrimination is mostly conducted on the cloud side, which does not meet the requirement of real-time processing. In this paper, we build an intelligent monitoring system for transmission lines based on edge computing. According to the specific characteristics of the real-world power filed applications, a lightweight deep learning algorithm model is designed. It is then transplanted on a self-developed AI chip and a field test is carried out. The experimental results show that the model designed in this paper balances well between the accuracy, the model size and the inference speed. It realizes status local discrimination of the transmission lines, and promising detection results are obtained.

Time-optimal trajectory planning based on event-trigger and conditional proportional control.

Trajectory planning is an important issue for manipulators and robots. To get a optimal trajectory, many constraints including actuators specifications, motion range of joints, workspace limitations, etc, and many objectives including the shortest time, the shortest distance, the lowest energy consumption, the minimum oscillations, obstacles-avoiding, etc, should be considered both. In this paper, firstly, the forward kinematics and inverse kinematics of a five axis manipulator are deduced. And, a simple method to choose one appropriate solution from multi solutions of inverse kinematics is proposed. Secondly, an easy-implemented optimization method of trajectory planning is proposed based on seventh order polynomial interpolation, event-trigger mechanism and conditional proportional control (P control). The proposed optimization method can capture the time optimal trajectory, and the actuators specifications including velocity, acceleration of motor can be guaranteed as well. Thirdly, comparative simulations and experiments validate the effectiveness and efficiency of proposed optimization method. The research provides an insight for the application of trajectory optimization on the micro controller with low computing capability and high real-time performance requirement.

A novel method for real-time ATR system of AUV based on Attention-MobileNetV3 network and pixel correction algorithm

This paper proposes an Automatic Target Recognition (ATR) system based on the Autonomous Underwater Vehicle (AUV) equipped with side scan sonar for real-time detection and positioning of underwater targets. The system introduces a pre-processing algorithm for the original Side Scan Sonar (SSS) image, which performs pixel-level correction of the image according to the slant range setting of the SSS sensor and the height from the seabed to eliminate the influence of target geometric distortion on the subsequent recognition algorithm and improve the recognition accuracy. At the same time, the attitude angle information of the AUV is obtained to correct the target position, making each pixel of the pre-processed image has underwater positioning (latitude and longitude) information. Since SSS images are seriously disturbed by noise, in order to accurately extract target features in real time, on the basis of selecting the lightweight network MobileNetV3, we introduce attention mechanism and integrated learning to meet the requirements of underwater target recognition accuracy and real-time performance. The multi-classifier ensemble prediction results further reduce the mis-identification rate of submarine targets. The ATR system fuses the prediction results and the submarine target position and sends it to the AUV main control system to guide the AUV how to navigate and plan path at the subsequent. The experimental results show that, we validate that the proposed method matches the state of the art in segmentation accuracy, the ATR system proposed in this paper can be embedded in the underwater unmanned platform, and achieve accurate underwater target detection and positioning, The real-time recognition speed of each picture is about 0.058 s.