Real-time Automatic Detection Research Articles

Vision-guided crack identification and size quantification framework for dam underwater concrete structures

Remotely operated vehicles with cameras provide a non-contact inspection solution for dam underwater structural defect detection. However, manual information extraction methods suffer from problems like labor costs and high misjudgment. This study proposes an integrated dam underwater crack identification and size quantification framework using machine vision and deep learning. First, a real-time automatic detection framework for dam underwater cracks is built via You Only Look Once v5 (YOLOv5), and four different backbone detectors are introduced to balance detection accuracy and speed. Then, the Swin-Transformer module is inserted into the YOLOv5 model to enhance its feature extraction capability and small object detection capability. Next, a method for measuring the true size of cracks was constructed based on deep learning and infrared laser rangefinders. In this study, physical model experiments and actual engineering projects are combined to validate the generalization capability of the proposed crack identification and size quantification method. Experimental results show that the Swin-Transformer-based YOLOv5 model effectively balances detection accuracy and speed with a precision of 0.986, a recall of 0.979, a mean average precision of 0.985, and a frame rate of 68 frames per second in detecting underwater crack images with 768 × 576 pixels. In addition, the proposed method can accurately identify and detect cracks in complex underwater scenes, including obstacle interference, tilt shooting angle, uneven illumination, and turbid water scenarios. Moreover, the method proposed in this paper can quantify the overall size and geometric parameters of underwater cracks with relatively small errors.

Real-Time Object Detection and Tracking for Occupational Health and Safety (OHS) in Mining Operations Using Streamed CCTV Data

Abstract Occupational Safety and Health (OHS) is very important in the mining industry which is prone to accident risks. Automatic detection of violations through CCTV streaming with the concept of a perimeter area is needed to prevent accidents. The purpose of this research is to create a violation detection system without re-modeling training that can adapt to different objects and regions. The proposed violation detection model includes four computational steps, namely (i) data reading, (ii) data pre-processing, (iii) object detection, and (iv) object tracking and violation detection. The computer vision technique uses the SIFT (Scale Invariant Feature Transform) algorithm and heuristics implemented with OpenCV. We used five CCTV video recordings of mining operations as examples. The results showed an average precision of 61.14%, a precision of 80.41%, and a recall of 69.38%. The average computation time of the model is 0.3 seconds per frame with heuristics that allow the model to be used in real-time. The model can be widely used in various fields and locations without having to retrain the model to improve OHS in the mining industry and other sectors with similar risks. This study will contribute to the development of a flexible, fast, and real-time automatic detection system.

Real-Time ConvNext-Based U-Net with Feature Infusion for Egg Microcrack Detection

Real-time automatic detection of microcracks in eggs is crucial for ensuring egg quality and safety, yet rapid detection of micron-scale cracks remains challenging. This study introduces a real-time ConvNext-Based U-Net model with Feature Infusion (CBU-FI Net) for egg microcrack detection. Leveraging edge features and spatial continuity of cracks, we incorporate an edge feature infusion module in the encoder and design a multi-scale feature aggregation strategy in the decoder to enhance the extraction of both local details and global semantic information. By introducing large convolution kernels and depth-wise separable convolution from ConvNext, the model significantly reduces network parameters compared to the original U-Net. Additionally, a composite loss function is devised to address class imbalance issues. Experimental results on a dataset comprising over 3400 graded egg microcrack image patches demonstrate that CBU-FI Net achieves a reduction in parameters to one-third the amount in the original U-Net, with an inference speed of 21 ms per image (1 million pixels). The model achieves a Crack-IoU of 65.51% for microcracks smaller than 20 μm and a Crack-IoU and MIoU of 60.76% and 80.22%, respectively, for even smaller cracks (less than 5 μm), achieving high-precision, real-time detection of egg microcracks. Furthermore, on the publicly benchmarked CrackSeg9k dataset, CBU-FI Net achieves an inference speed of 4 ms for 400 × 400 resolution images, with an MIoU of 81.38%, proving the proposed method’s robustness and generalization capability across various cracks and complex backgrounds.

Temperature prediction of submerged arc furnace in ironmaking industry based on residual spatial-temporal convolutional neural network

The submerged arc furnace is widely regarded as one of the most promising ore smelting technologies. However, the real-time monitoring of the multiple physical fields including electric, thermal, and mas, through computational fluid dynamics demands significant computational resources. This paper introduces the spatial-temporal convolutional neural network algorithm to address this challenge. Initially, the influences of various working conditions on these physical fields are analyzed. Subsequently, a prediction model is developed based on the coupling of these multiple physical fields model. The spatial-temporal convolutional neural network algorithm is then employed to elucidate the main parameter distributions, enabling the automatic real-time detection of temperature variation trends and providing a theoretical foundation for intelligent furnace operation. The findings indicate that the electric field is the predominant factor causing non-uniform heat distribution, with localized overheating primarily occurring at the electrode ends. The application of the proposed model facilitates dynamic prediction of the temperature distribution, establishing relationships between historical and future time steps as well as local and global temperature variations. The reliability of the temperature prediction model is confirmed, with the model achieving an accuracy of 99.76 %, surpassing the 98.18 % accuracy of the traditional multi-layer perceptron model.

A FairMOT approach based on video recognition for real-time automatic incident detection on expressways

A FairMOT approach based on video recognition for real-time automatic incident detection on expressways

Multiobject Real-Time Automatic Detection Method for Production Quality Control of Prefabricated Laminated Slabs

Multiobject Real-Time Automatic Detection Method for Production Quality Control of Prefabricated Laminated Slabs

Improved STMask R-CNN-based defect detection model for automatic visual inspection of an optics lens.

A lens defect is a common quality issue that has seriously harmed the scattering characteristics and performance of optical elements, reducing the quality consistency of the finished products. Furthermore, the energy hotspots coming from the high-energy laser through diffraction of optical component defects are amplified step by step in multi-level laser conduction, causing serious damage to the optical system. Traditional manual detection mainly relies on experienced workers under a special light source environment with high labor intensity, low efficiency, and accuracy. The common machine vision techniques are incapable of detecting low contrast and complex morphological defects. To address these challenges, a deep learning-based method, named STMask R-CNN, is proposed to detect defects on the surface and inside of a lens in complex environments. A Swin Transformer, which focuses on improving the modeling and representation capability of the features in order to improve the detection performance, is incorporated into the Mask R-CNN in this case. A challenge dataset containing more than 3800 images (18000 defect sample targets) with five different types of optical lens defects was created to verify the proposed approach. According to our experiments, the presented STMask R-CNN reached a precision value of 98.2%, recall value of 97.7%, F1 score of 97.9%, mAP@0.5 value of 98.1%, and FPS value of 24 f/s, which outperformed the SSD, Faster R-CNN, and YOLOv5. The experimental results demonstrated that the proposed STMask R-CNN outperformed other popular methods for multiscale targets, low contrast target detection and nesting, stacking, and intersecting defects sample detection, exhibiting good generalizability and robustness, as well as detection speed to meet mechanical equipment production efficiency requirements. In general, this research offers a favorable deep learning-based method for real-time automatic detection of optical lens defects.

Artificial Intelligence based real-time automatic detection and classification of skin lesion in dermoscopic samples using DenseNet-169 architecture

The most common challenge faced by dermoscopy images is the automatic detection of lesion features. All the existing solutions focus on complex algorithms to provide accurate detections. In this research work, proposed Online Tigerclaw Fuzzy Region Segmentation with Deep Learning Classification model, an intellectual model is proposed that provides discrimination of features with classification even in fine-grained samples. This model works on four different stages, which include the Boosted Anisotropic Diffusion filter with Recursive Pixel Histogram Equalization (BADF-RPHE) in the preprocessing stage. The next step is the proposed Online Tigerclaw Fuzzy Region Segmentation (OTFRS) algorithm for lesion area segmentation of dermoscopic images, which can achieve 98.9% and 97.4% accuracy for benign and malignant lesions, respectively. In the proposed OTFRS, an accuracy improvement of 1.4% is achieved when compared with previous methods. Finally, the increased robustness of lesion classification is achieved using Deep Learning Classification –DenseNet 169 with 500 images. The proposed approach was evaluated with accuracy classifications of 100% and 98.86% for benign and malignant lesions, respectively, and a processing time of less than 18 sec. In the proposed DensetNet-169 classification technique, an accuracy improvement of 3% is achieved when compared with other state-of-art methods. A higher range of true positive values is obtained for the Region of Convergence (ROC) curve, which indicates that the proposed work ensures better performance in clinical diagnosis for accurate feature visualization analysis. The methodology has been validated to prove its effectiveness and throw light on the lives of affected patients so they can resume normalcy and live long. The research work was tested in real-time clinical samples, which delivered promising and encouraging results in skin cell detection procedures.

Automatic image-based tracking of gadolinium-filled balloon wedge catheters for MRI-guided cardiac catheterization using deep learning.

Magnetic Resonance Imaging (MRI) is a promising alternative to standard x-ray fluoroscopy for the guidance of cardiac catheterization procedures as it enables soft tissue visualization, avoids ionizing radiation and provides improved hemodynamic data. MRI-guided cardiac catheterization procedures currently require frequent manual tracking of the imaging plane during navigation to follow the tip of a gadolinium-filled balloon wedge catheter, which unnecessarily prolongs and complicates the procedures. Therefore, real-time automatic image-based detection of the catheter balloon has the potential to improve catheter visualization and navigation through automatic slice tracking. In this study, an automatic, parameter-free, deep-learning-based post-processing pipeline was developed for real-time detection of the catheter balloon. A U-Net architecture with a ResNet-34 encoder was trained on semi-artificial images for the segmentation of the catheter balloon. Post-processing steps were implemented to guarantee a unique estimate of the catheter tip coordinates. This approach was evaluated retrospectively in 7 patients (6M and 1F, age = 7 ± 5 year) who underwent an MRI-guided right heart catheterization procedure with all images acquired in an orientation unseen during training. The overall accuracy, specificity and sensitivity of the proposed catheter tracking strategy over all 7 patients were 98.4 ± 2.0%, 99.9 ± 0.2% and 95.4 ± 5.5%, respectively. The computation time of the deep-learning-based segmentation step was ∼10 ms/image, indicating its compatibility with real-time constraints. Deep-learning-based catheter balloon tracking is feasible, accurate, parameter-free, and compatible with real-time conditions. Online integration of the technique and its evaluation in a larger patient cohort are now warranted to determine its benefit during MRI-guided cardiac catheterization.

Deep learning algorithm for real-time automatic crack detection, segmentation, qualification

Cracking is one of the typical damages in concrete structures, and it is crucial to detect and quantify cracks in a timely and efficient manner. However, current research primarily focuses on either single-task recognition or dual-task recognition based on multi-step sequential approaches. Less attention has been devoted to the multi-task integration of cracks. To address the challenges of inefficient and multi-step detection in traditional concrete crack detection methods, a novel deep learning-based model, called YOLOv5-IDS, is proposed based on You Only Look Once network v5 (version 6.2) with the combination of bilateral segmentation network while introducing a dilated convolution, pyramid pooling module, and attention refinement module. Moreover, crack parameter measurement algorithms based on the micro-element method are proposed to improve accuracy and efficiency. The method proposed in this study can not only detect and segment cracks with high accuracy and efficiency, but also quickly measure crack parameters, thus developing a complete method for the process from real-time crack detection and segmentation to crack parameter measurement. The experimental results for the YOLOv5-IDS model reveal the following performance metrics. For crack detection, the mean average precision with an intersection of union threshold of 0.5 (mAP@0.5) is 84.33%, and the frames per second (FPS) is 159 f/s. For crack segmentation, the mean intersection over union (mIoU) is 94.78%, and the FPS is 8 f/s, respectively. Compared to existing methods, the proposed approach exhibits improvements in both accuracy and efficiency. Moreover, the calculation of crack parameters proves to be both precise and rapid.

A Lightweight and Partitioned CNN Algorithm for Multi-Landslide Detection in Remote Sensing Images

Landslide detection is crucial for natural disaster risk management. Deep-learning-based object-detection algorithms have been shown to be effective in landslide studies. However, advanced algorithms currently used for landslide detection require high computational complexity and memory requirements, limiting their practical applicability. In this study, we developed a high-resolution dataset for landslide-prone regions in China by extracting historical landslide remote sensing images from the Google Earth platform. We propose a lightweight LP-YOLO algorithm based on YOLOv5, with a more-lightweight backbone that incorporates our designed PartitionNet and neck equipped with CSPCrossStage. We constructed and added the vertical and horizontal (VH) block to the backbone, which explores and aggregates long-range information with two directions, while consuming a small amount of computational cost. A new feature fusion structure is proposed to boost information flow and enhance the location accuracy. To speed up the model learning process and improve the accuracy, the SCYLLA-IoU (SIoU) bounding box regression loss function was used to replace the complete IoU (CIoU) loss function. The experimental results demonstrated that our proposed model achieved the highest detection performance (53.7% of Precision, 49% of AP50 and 25.5% of AP50:95) with a speed of 74 fps. Compared to the YOLOv5 model, the proposed model achieved 4% improvement for Precision, 2.6% improvement for AP50, and 2.5% for AP50:95, while reducing the model parameters and FLOPs by 38.4% and 53.1%, respectively. The results indicated that the proposed lightweight method provides a technical guidance for achieving reliable and real-time automatic landslide detection and can be used for disaster prevention and mitigation.

Design of Aeroengine Air Leak Automatic Monitoring System Based on Infrared Thermography

The aeroengine often suffers from air leaks during the test, which will affect the performance of the aeroengine, thus affecting the test task and even the test safety. In order to make up for the low efficiency and poor real-time of traditional manual detection methods for aeroengine air leaks, a design scheme for aeroengine leakage real-time automatic detection system is proposed. The system uses infrared thermography technology to collect real-time infrared information on the surface of the aeroengine during the test and then generates thermal images. The image processing technology is used to identify the leakage fault and locate the leakage location. Through the design and analysis of the system architecture, composition unit, operation principle and key technologies of the system implementation, the designed air leak monitoring system has high feasibility. Compared with traditional air leak detection methods, the system has the advantages of real-time, automaticity, remoteness, non-contact, non-destructive, simplicity, and intuitiveness, and can significantly improve the efficiency of aeroengine air leak fault detection.

A real-time automatic pothole detection system using convolution neural networks

Detecting a pothole can help prevent damage to your vehicle and potentially prevent an accident. Different techniques, including machine learning, deep learning models, sensor methods, stereo vision, the internet of things (IoT), and black-box cameras, have already been applied to address the problem. However, studies have shown that machine learning and deep learning techniques successfully detect potholes. However, because most of these successful attempts are peculiar to the location of the study, we found no study which has addressed the peculiarity of potholes in South Africa using a tailored-trained deep learning model. In this study, we propose using a convolutional neural network (CNN), a type of deep learning model, to address this growing problem on South African roads. To achieve this, a CNN model was designed from scratch and trained with image samples obtained from the context of the study. The classifier was adapted to distinguish between a binary class which identifies the presence or absence of potholes. Results showed a significant performance enhancement at a classification accuracy of 92.72%. The outcome of this study showed that this machine learning approach holds great potential for addressing the challenge of potholes and road bumps in the region and abroad.

Realization of Autonomous Detection, Positioning and Angle Estimation of Harvested Logs

To further develop forest production, higher automation of forest operations is required. Such endeavour promotes research on unmanned forest machines. Designing unmanned forest machines that exercise forwarding requires an understanding of positioning and angle estimations of logs after cutting and delimbing have been conducted, as support for subsequent crane loading work. This study aims to improve the automation of the forwarding operation and presents a system to realize real-time automatic detection, positioning, and angle estimation of harvested logs implemented on an existing unmanned forest machine experimental platform from the AORO (Arctic Off-Road Robotics) Lab. This system uses ROS as the underlying software architecture and a Zed2 camera and NVIDIA JETSON AGX XAVIER as the imaging sensor and computing platform, respectively, utilizing the YOLOv3 algorithm for real-time object detection. Moreover, the study combines the processing of depth data and depth to spatial transform to realize the calculation of the relative location of the target log related to the camera. On this basis, the angle estimation of the target log is further realized by image processing and color analysis. Finally, the absolute position and log angles are determined by the spatial coordinate transformation of the relative position data. This system was tested and validated using a pre-trained log detector for birch with a mean average precision (mAP) of 80.51%. Log positioning mean error did not exceed 0.27 m and the angle estimation mean error was less than 3 degrees during the tests. This log pose estimation method could encompass one important part of automated forwarding operations.

An ensemble method for automatic real-time detection and evaluation of oil and gas leakage in subsea pipelines based on 3D real-time sonar system

An ensemble method for automatic real-time detection and evaluation of oil and gas leakage in subsea pipelines based on 3D real-time sonar system

IoT Based Accident Detection and Management System

Accidents on roads are a significant concern worldwide, and the response time to such incidents is crucial in saving lives. The present work proposes a real-time automatic accident detection system using the Internet of Things (IoT) based on a NodeMCU microcontroller, GPS, gyroscope, and vibration sensor. The proposed system is designed to detect accidents in real-time and notify emergency services promptly. The system uses an accelerometer-based vibration sensor to detect any sudden changes in the velocity of the vehicle. A GPS module is used to provide the location of the accident, while a gyroscope sensor provides the orientation of the vehicle The NodeMCU microcontroller is responsible for reading the data from the sensors and sending it to the cloud server. The communication module used in the system is the MQTT protocol, a lightweight messaging protocol designed for IoT devices. The microcontroller sends the data to the cloud server using MQTT, which then sends the data to the emergency services. receives the data from the microcontroller using MQTT and sends it to the emergency services using AWS Lambda. The proposed system is designed to be lightweight, reliable, and cost-effective. It uses off-the-shelf components and can be easily deployed in any vehicle. With the increasing number of vehicles on roads, the automatic accident detection system using IoT can help reduce the response time to accidents and save lives. In conclusion, the proposed system is a step towards creating a more efficient and effective accident detection system that can significantly reduce the response time to accidents and ultimately save lives. Key Word: Accident Detection, IoT, Emergency Service, Response Time.

From face detection to emotion recognition on the framework of Raspberry pi and galvanic skin response sensor for visual and physiological biosignals

The facial and physiological sensor-based emotion recognition methods are two popular methods of emotion recognition. The proposed research is the first of its kind in real-time emotion recognition that combines skin conductance signals with the visual-based facial emotion recognition (FER) method on a Raspberry Pi. This research includes stepwise documentation of method for automatic real-time face detection and FER on portable hardware. Further, the proposed work comprises experimentation related to video induction and habituation methods with FER and the galvanic skin response (GSR) method. The GSR data are recorded as skin conductance and represent the subject's behavioral changes in the form of emotional arousal and face emotion recognition on the portable device. The article provides a stepwise implementation of the following methods: (a) the skin conductance representation from the GSR sensor for arousal; (b) gathering visual inputs for identifying the human face; (c) FER from the camera module; and (d) experimentation on the proposed framework. The key feature of this article is the comprehensive documentation of stepwise implementation and experimentation, including video induction and habituation experimentation. An illuminating aspect of the proposed method is the survey of GSR trademarks and the conduct of psychological experiments. This study is useful for emotional computing systems and potential applications like lie detectors and human–machine interfaces, devices for gathering user experience input, identifying intruders, and providing portable and scalable devices for experimentation. We termed our approaches "sensovisual" (sensors + visual) and "Emosense" (emotion sensing).

Real-time Automatic Arrhythmia Detection System based on Extreme Gradient Boosting and Neural Network Algorithm

Arrhythmia and other diseases are puzzling more and more people. Accurate detection is the key to realizing intelligent diagnosis of electrocardiogram(ECG) monitoring systems. It can prevent heart disease and effectively reduce mortality. An efficient and accurate arrhythmia detection method is urgent. In this work, a real-time automatic arrhythmia detection technology based on extreme gradient boosting (XGboost) and convolutional neural network (CNN) algorithm were developed. First, ECG signals in the MIT-BIH Arrhythmia database are preprocessed: 1) EMG interference filtering; 2) Power frequency interference suppression; 3) Baseline drift correction. Secondly, We use the cyclic singular spectrum (CISSA) algorithm to decompose the ECG after pretreatment. From the original ECG and the 7 simple signals obtained from decomposition, 23 features about the time domain, frequency domain, nonlinear dynamics and statistics are extracted respectively. Finally, XGboost and CNN algorithms are used to build a classification model, and the extracted features are classified, trained and recognized to achieve automatic detection of arrhythmia. The experimental results show that XGboost and CNN algorithms can automatically detect 98.40%, 95.65% and 97.60%, 95.12% of Category 2 and Category 4 arrhythmias, respectively.

Ensemble of deep transfer learning models for real-time automatic detection of face mask.

The COVID-19 pandemic is causing a global health crisis. Public spaces need to be safeguarded from the adverse effects of this pandemic. Wearing a facemask has become an adequate protection solution many governments adopt. Manual real-time monitoring of face mask wearing for many people is becoming a difficult task. This paper applies three heterogeneous deep transfer learning models, viz., ResNet50, Inception-v3, and VGG-16, to prepare an ensemble classification model for detecting whether a person is wearing a mask. The ensemble classification model is underlined by the concept of the weighted average technique. The proposed framework is based on two phases. An off-line phase that aims to prepare a classification model by following training-testing steps to detect and locate facemasks. Then in the second online phase, it is deployed to detect real-time faces from live videos, which are captured by a web-camera. The prepared model is compared with several state-of-the-art models. The proposed model has achieved the highest classification accuracy of 99.97%, precision of 0.997, recall of 0.997, F1-score of 0.997 and kappa coefficient 0.994. The superiority of the model over state-of-the-art compared methods is well evident from the experimental results.

Self-Supervised Learning for Solar Radio Spectrum Classification

Solar radio observation is an important way to study the Sun. Solar radio bursts contain important information about solar activity. Therefore, real-time automatic detection and classification of solar radio bursts are of great value for subsequent solar physics research and space weather warnings. Traditional image classification methods based on deep learning often require considerable training data. To address insufficient solar radio spectrum images, transfer learning is generally used. However, the large difference between natural images and solar spectrum images has a large impact on the transfer learning effect. In this paper, we propose a self-supervised learning method for solar radio spectrum classification. Our method uses self-supervised training with a self-masking approach in natural language processing. Self-supervised learning is more conducive to learning the essential information about images compared with supervised methods, and it is more suitable for transfer learning. First, the method pre-trains using a large amount of other existing data. Then, the trained model is fine-tuned on the solar radio spectrum dataset. Experiments show that the method achieves a classification accuracy similar to that of convolutional neural networks and Transformer networks with supervised training.