Vision-based Detection Method Research Articles

A comprehensive approach to detecting chemical adulteration in fruits using computer vision, deep learning, and chemical sensors

Contamination of harmful additives in fruits has become a concerning norm these days. Owing to the great popularity of fruits, dishonest vendors frequently use harmful chemicals to contaminate fruits to extend their shelf life, which is extremely dangerous for the general public's health. To mitigate this issue, machine-learning algorithms like Decision Tree Classifier, Naïve Bayes and a deep learning model named “DurbeenNet” are evaluated separately. Alongside, a computer vision-based detection method coupled with a hybrid model is proposed that combines deep learning and chemical sensor. Formaldehyde Detection Sensor is used in this experiment to take reading of the sensor data. Mango, Apple, Banana, and Malta are taken as sample fruits in this study. Sensor data for both fresh and chemical-mixed fruit is newly collected using Formaldehyde Detection Sensor. The above mentioned sensor data along with the previously captures images of both fresh and chemical-mixed state are being integrated to a hybrid model. Among two machine learning algorithms naïve bayes come up with 82 % accuracy. Using both sensor data and captured image data, the proposed model “SensorNet” provides highest accuracy of 97.03 % which is substantial than “DurbeenNet” model's accuracy. Through the utilization of these fruit samples, formaldehyde detection sensor provides instantaneous detection, identifying the specific toxic substances present in the contaminated fruits.

Damage Detection in Glass Fibre Composites Using Cointegrated Hyperspectral Images.

Hyperspectral imaging (HSI) is a remote sensing technique that has been successfully applied for the task of damage detection in glass fibre-reinforced plastic (GFRP) materials. Similarly to other vision-based detection methods, one of the drawbacks of HSI is its susceptibility to the lighting conditions during the imaging, which is a serious issue for gathering hyperspectral data in real-life scenarios. In this study, a data conditioning procedure is proposed for improving the results of damage detection with various classifiers. The developed procedure is based on the concept of signal stationarity and cointegration analysis, and achieves its goal by performing the detection and removal of the non-stationary trends in hyperspectral images caused by imperfect lighting. To evaluate the effectiveness of the proposed method, two damage detection tests have been performed on a damaged GFRP specimen: one using the proposed method, and one using an established damage detection workflow, based on the works of other authors. Application of the proposed procedure in the processing of a hyperspectral image of a damaged GFRP specimen resulted in significantly improved accuracy, sensitivity, and F-score, independently of the type of classifier used.

Based on CBB-yolo rusted workpiece surface roughness detection

The surface roughness of the workpiece is one of the important indicators to measure the quality of the workpiece. Vision-based detection methods are mainly based on human-designed image feature indicators for detection, while the self-extraction method of milling surface features based on deep learning has problems such as poor perception of details, and will be affected by surface rust. In order to solve these problems, this paper proposes a visual inspection method for surface roughness of milling rusted workpieces combined with local equilibrium histogram and CBB-yolo network. Experimental results show that local equilibrium histogram can enhance the milling texture and improve the accuracy of model detection when different degrees of rust appear on the surface of the milled workpiece. The detection accuracy of the model can reach 97.9%, and the Map can reach 99.3. The inference speed can reach 29.04 frames per second. And the inspection of workpieces without rust, this method also has high detection accuracy, can provide automatic visual online measurement of milling surface roughness Theoretical basis.

Method for loose bolt positioning and prediction of bolt axial force in bolt group

Bolt-looseness reduces the stiffness of the structure and poses safety risks. The accurate monitoring of bolt axial force is crucial for ensuring the service performance of components. However, existing methods are less suitable for large-scale bolt axial force detection owing to unclear mechanisms of force interaction among bolts and limited vision-based detection methods for identifying slight angle loosening and cycle rotation. This paper proposes a bolt-set simulation model to investigate the influence of loose bolts on the axial force of tight bolts, and model the correlation between loose and tight bolts. Based on the established model, a method for accurately positioning loose bolts and predicting the axial force was developed and validated through experiments. The method's accuracy in locating the loosened bolt exceeds 99%, with the error between the predicted axial force and experimental results being less than 4%. The proposed method allows a more precise evaluation of structural health.

Automated detection of bicycle helmets using deep learning

Bicycle helmets are a main measure for injury prevention in case of a crash and are a central variable in transport safety studies. Despite this, helmet use data is only collected sporadically, as the observation of helmet use in traffic by human observers is costly and time-consuming. An automated method for the accurate registration of bicycle helmet use would enable the broad and precise registration of cyclists’ helmet use. In this paper, we develop and test a computer vision-based detection method that can be applied to traffic video data. We record bicycle traffic at two observation sites in Copenhagen, Denmark, and annotate a dataset of 4000 cyclists, registering their helmet use. We then train a state-of-the-art object detection algorithm on the detection of cyclists and helmet use. The developed model has good accuracy in registering active cyclists. For helmet use registration on the test data set, there was an underestimation of 0.52% (algorithm registered helmet use: 50.23%; actual helmet use: 50.75%). Cross-testing the algorithm, i.e., training on one observation site and applying it to another, results in a larger underestimation of bicycle helmet use between 5.28% and 6.31%. Finally, we apply the algorithm to a week of video data from two Copenhagen sites, identifying commuting-related peaks of cyclists and registering helmet use differences between the observation sites. This study shows that computer vision algorithms are a feasible method for the automated detection of bicycle helmet use. Further research needs to be conducted to make the site transfer more robust and to increase accuracy levels.

Improved Vision-Based Method for Detection of Unauthorized Intrusion by Construction Sites Workers

Improved Vision-Based Method for Detection of Unauthorized Intrusion by Construction Sites Workers

CANet: Context-aware Aggregation Network for Salient Object Detection of Surface Defects

Surface defect detection has become more and more important in the industrial manufacture and engineering application in recent years. However, due to the lack of overall perception and interaction among features layers, lots of computer vision-based detection methods cannot grab the complete details of defects when dealing with complex scenes, such as low contrast and irregular shape. Therefore, in this paper, we propose a Context-aware Aggregation Network (CANet) to accurately pop-out the defects, where we focus on the mining of context cues and the fusion of multiple context features. To be specific, embarking on the multi-level deep features extracted by encoder, we first deploy a sufficient exploration to dig the context information by deploying the weighted convolution pyramid (WCP) module, which extracts multi-scale context features, transfers the information flow between different resolution features, and fuses the features with same resolution. By this way, we can obtain the effective context pyramid features. Then, the decoder deploys the weighted context attention (WCA) module to filter the irrelevant information from context features and employs the cascaded fusion structure (CFS) to aggregate the multiple context cues in a hierarchical way. Following this way, the generated high-quality saliency maps can highlight the defects accurately and completely. Extensive experiments are performed on four public datasets, and the results firmly prove the effectiveness and superiority of the proposed CANet under different evaluation metrics.

Bolt loosening angle detection based on binocular vision

Bolt looseness detection is critical in preventing bolt connection failure. Compared to traditional sensor-based bolt looseness detection, image-based methods are low-cost and contactless and have thus become the highlight of research. However, current monocular vision-based detection methods are prone to error scaused by the camera perspective . In this paper, we present a novel bolt loosening angle detection method based on binocular vision. Key points on the bolt are detected and matched by SuperPoint Gauss network for 3D coordinates reconstruction and motion tracking. The bolt loosening angle is solved by fitting the rotation equation using random sample consensus. Experiments verify the proposed method performs well under different perspectives of camera and illumination conditions with an average error of 1.5°. Comparative test shows our method is superior to the monocular vision-based method in terms of accuracy when there is a large perspective angle. The proposed method is mark-free and robust to various working conditions, which makes it of great value for engineering application.

Multi-object detection at night for traffic investigations based on improved SSD framework

Despite significant progress in vision-based detection methods, the task of detecting traffic objects at night remains challenging. Visual information of medium and small stationary objects is deteriorated due to poor lighting conditions. And the visual information is important for traffic investigations. For meeting the needs of night traffic investigations, this study focuses on presenting a nighttime multi-object detection framework based on Single Shot MultiBox Detector (SSD). Considering the need of traffic investigations, the applicable detection framework is presented for detecting traffic objects, especially medium and small stationary objects. In the framework, the Dense Convolutional Network (DenseNet) and deconvolutional layers are introduced to enhance the feature reuse, and the effectiveness of the optimization is finally verified. In this paper, qualitative and quantitative experiments are presented. The results show that our presented framework has better detection performance for medium and small stationary objects. Moreover, the results show that presented framework has better performance for nighttime traffic investigations at intersections.

Automated detection and analysis of piglet suckling behaviour using high-accuracy amodal instance segmentation

Piglet suckling behaviour is a critical indicator of piglet liveability and health status; however, automated detection of this important behaviour is rarely reported in literatures. In this study, we proposed a two-step computer vision-based detection method for piglet suckling behaviour. In the first step, an anchor-free deep learning network was employed in instance segmentation of individual sows and piglets. Firstly the localization head detected piglets to obtain the features of the region of interest (ROI). The features extracted from ROI were passed to a novel attention graph convolution-based structure to distil element-wise features. The distilled features were further encoded by a graph convolutional network and were fed into the boundary head and the mask head for piglet contour and mask prediction, respectively. In the second step, the piglets, in adhesion with the sow, were tracked by intersection over union (IOU) which was calculated between adjacent frames. Piglet motion features were derived from the maximum, minimum, variance, and average values in IOU sequence within the 21-frame (3-s) independent processing units. The extracted motion features were input into an SVM, classifying a piglet into suckling or nonsuckling. The dataset, for training and verifying the proposed network respectively, was composed of 100 1-min and 7-fps short video clips as well as one 8-h long video episode, from seven pens of Large White sows and piglets. Our method achieved favourable detection performance with F1 score of 93.6%, Recalls of 92.1%, and Precisions of 95.2% in short video clips, which showed that detecting suckling behaviours for piglets using amodal instance segmentation was feasible. The time budgets of at least one piglet, more than half of the piglets, or all piglets exhibiting suckling behaviour were 74.0 min, 65.6 min, and 1.1 min in an 8-h long video.

Illumination and Temperature-Aware Multispectral Networks for Edge-Computing-Enabled Pedestrian Detection

Accurate and efficient pedestrian detection is crucial for the intelligent transportation system regarding pedestrian safety and mobility, e.g., Advanced Driver Assistance Systems, and smart pedestrian crosswalk systems. Among all pedestrian detection methods, the vision-based detection method is demonstrated to be the most effective in previous studies. However, the existing vision-based pedestrian detection algorithms still have two limitations that restrict their implementations, those being real-time performance as well as the resistance to the impacts of environmental factors, e.g., low illumination conditions. To address these issues, this study proposes a lightweight Illumination and Temperature-aware Multispectral Network (IT-MN) for accurate and efficient pedestrian detection. The proposed IT-MN is an efficient one-stage detector. For accommodating the impacts of environmental factors and enhancing the sensing accuracy, thermal image data is fused by the proposed IT-MN with visual images to enrich useful information when visual image quality is limited. In addition, an innovative and effective late fusion strategy is also developed to optimize the image fusion performance. To make the proposed model implementable for edge computing, the model quantization is applied to reduce the model size by 75% while shortening the inference time significantly. The proposed algorithm is evaluated by comparing it with the selected state-of-the-art algorithms using a public dataset collected by in-vehicle cameras. The results show that the proposed algorithm achieves a low miss rate and inference time at 14.19% and 0.03 seconds per image pair on GPU. Besides, the quantized IT-MN achieves an inference time of 0.21 seconds per image pair on the edge device, demonstrating the potentiality of deploying the proposed model on edge devices as a highly efficient pedestrian detection algorithm.

Non-uniform illumination image enhancement for surface damage detection of wind turbine blades

Vision-based monitoring technology has been exploited for the surface damage detection of wind turbine blades (WTBs). However, the image quality is often significantly influenced by environmental illumination conditions, imposing difficulties for obtaining high detection accuracy for large-scale WTB surfaces. To improve the image quality and guarantee reliable damage detection on WTB surfaces, this study presents an image processing method for enhancing the images captured under non-uniform illumination conditions. First, cartoon and texture maps of the WTB images are constructed by cartoon texture decomposition. Second, an illumination model is established on the cartoon map from the Gaussian scale-space, to remove the non-uniform illumination. Third, the WTB images are enhanced by utilizing a multi-directional Gabor transformation to increase the contrast between the surface damage and image background. Finally, the WTB surface damages are detected using a gradient threshold segmentation method. The experimental results indicate that the damage detection accuracy of the WTB surfaces is significantly improved by using image enhancement. The F-measure and the intersection-over-union values of the damage detection are increased by 28.05% and 41.61%, respectively, relative to those detected from the input images. Therefore, this vision-based detection method for WTB surface damage inspection under non-uniform illumination has potential application values in practice.

Defect Detection System for Smartphone Front Camera Based on Improved Template Matching Algorithm

Automatic defect detection plays crucial role in resilient manufacturing in terms of product quality and cost effectiveness. With reference to the smartphone front cameras production process, the most recurrent defects can be classified into no hole, inner hole burr, outer circle damage, hole deformation, outer circle fracture and hole position offset. Due to the fast production lines and the defects micro size, Sampling-based methods has huge uncertainty and limitation, and Machine learning-based methods are characterised by low efficiency. To tackle these issues, this paper proposes a machine vision-based detection methods of smartphone front camera based on a multi-step template matching algorithm to reduce the computational effort. Specifically, in order to improve the algorithm efficiency, the images of the smartphone front cameras, acquired using industrial image acquisition devices are pre-processed by performing Hough circle and line transformations respectively, then locate the exact defect area as a region of interest (ROI). Finally, a multi-step template matching algorithm is used to detect and classify a number of common defects. Experimental results show an excellent suitability of the proposed system in detecting front camera surface defects. A benchmarking with other available technologies highlights how the proposed system yields an improvement in the detection speed by 46%, along with an improvement in the detection accuracy by 9%. The successful industrial implementation is discussed with reference to the integration into an automatic defect detection system in a smartphone front camera manufacturing context.

Driving Behavior Analysis of Intelligent Vehicle System for Lane Detection Using Vision-Sensor

Lane detection on the road is the most essential phase of existing advanced driver assistance systems (ADAS). In the current road scheme, the lanes are parallel tracks which are marked based on the different speed limit for the vehicles. Lane markings are defined as driving regions for the vehicles in order to avoid a collision. In an Intelligent Vehicle System (IVS), these markings are being identified by various massive computing techniques and sensor-based mechanisms like radar, LiDAR, and GPS and have high operational costs. However, vision-based detection methods are gaining popularity due to its low cost and better understanding capability of the scene. The proposed study follows computer vision and image processing techniques to examine and learn significant lane features. The approach demonstrates the practical and effective usage of the lane detection model (LDM). The LDM has been designed with a vehicle prototype using Raspberry Pi, which reveals the intelligent behavior against the detected lane lines. The model is less expensive and power-efficient, which confirms the operational capability of the proposed system under a heterogeneous environment.

A Deep Learning Based Dislocation Detection Method for Cylindrical Crystal Growth Process

For the single crystal furnace used in the photovoltaic industry, growth problems occur frequently due to dislocations during the shouldering and cylindrical growth steps of the Czochralski (CZ) crystal growth. Detecting the dislocation phenomenon in the cylindrical growth step is very important for entire automation of the CZ crystal furnace, since this process usually lasts for more than 48h. The irregular nature of different patterns of dislocation would impose a big challenge for a traditional machine vision-based detection method. As almost no publications have been dedicated to detecting this phenomenon, to address this issue, after analyzing the characteristics of the silicon ingot image of this process, this paper proposes a kind of deep learning-based dislocation detection method along with tracking strategy to simulate manual inspection. The model has a good detection effect whether there is occlusion or not, the experimental results show that the detection accuracy is 97.33%, and the inference speed is about 14.7 frames per second (FPS). It can achieve the purpose of reducing energy consumption and improving process automation by monitoring this process.

Optimized visual recognition algorithm in service robots

Vision-based detection methods often require consideration of the robot’s sight. For example, panoramic images cause image distortion, which negatively affects the target recognition and spatial localization. Furthermore, the original you only look once method does not have a reasonable performance for the image recognition in the panoramic images. Consequently, some failures have been reported so far when implementing the visual recognition on the robot. In the present study, it is intended to optimize the conventional you only look once algorithm and propose the modified you only look once algorithm. Comparing the obtained results with the experiment shows that the modified you only look once method can be effectively applied in the graphics processing unit to reach the panoramic recognition speedup to 32 frames rate per second, which meets the real-time requirements in diverse applications. It is found that the accuracy of the object detection when applying the proposed modified you only look once method exceeds 70% in the studied cases.

Review on Applicability of Vision-based Detection Technology in Field Obstacle Detection

In order for robotized field operation by agricultural machineries and complete replacement of manual work in field operation, autonomous obstacle avoidance of agricultural machineries during field operation becomes a critical technology to be solved. For this technology, it is an indispensable requirement and critical premise to timely and accurately perceive static and dynamic information of field obstacles. Firstly, this paper analyzes and determines what obstacles are the target obstacles in the field operation of agricultural machineries from the two aspects - whether the obstacles hinder current operation and whether the obstacles hinder current obstacle avoidance process. Subsequently, this paper mainly discusses the applicability of obstacle detection methods based on monocular vision and binocular stereo vision in field environment: the main existing monocular vision-based methods to detect obstacles are summarized, and their restraints for detecting obstacles in the field are analyzed; considering that precise stereo matching of image pairs is the most complicated and time-consuming part of binocular vision-based detection methods, currently used typical stereo matching algorithms are summarized and compared, and their applicability in field obstacle detection is discussed. At last, the research direction is prospected, and a tendency to detect obstacles for agricultural machineries during field operation is based on multi-sensor fusion system with vision sensor as one of the main sensors.

Precise Positioning of Linear Motor Mover Directly From the Phase Difference Analysis

This article presents a new visual detection method with high precision, robustness, and processing rate to estimate the linear motor displacements. Subpixel level location is achieved by using the phase-difference matrix and a structured aperiodic fence stripe placed on the mover. Based on Fourier shifted properties, this subpixel method builds a linear relation between the mover's spatial displacements and the direct frequency-domain phase difference (not the phase correlation matrix) to calculate the translational shift in a series of stripe images; thereby, precisely determining the mover location of linear motor. Unlike the traditional phase correlation algorithm, the proposed method converts 2-D stripe images into 1-D signals via the projection technique and then uses the least squares method to linearly fit the phase difference slope, thereby acquiring subpixel shifts with less time complexity. Due to the motion parameter that is directly estimated from the phase difference, this method is also insensitive to the gray-scale change with certain anti-interference capability, thereby making it suitable for measuring the moving position of the mover. Experimental results show that the proposed vision-based detection method is efficient and applicable for real-time accurate positioning of linear motor, and the measuring root mean square error is about 0.05 mm.

An Impedance Sensing Platform for Monitoring Heterogeneous Connectivity and Diagnostics in Lab-on-a-Chip Systems.

Reliable hardware connectivity is vital in heterogeneous integrated systems. For example, in digital microfluidics lab-on-a-chip systems, there are hundreds of physical connections required between a microelectromechanical fabricated device and the driving system that can be remotely located on a printed circuit board. Unfortunately, the connection reliability cannot be checked or monitored by vision-based detection methods that are commonly used in the semiconductor industry. Therefore, a sensing platform that can be seamlessly integrated into existing digital microfluidics systems and provide real-time monitoring of multiconnectivity is highly desired. Here, we report an impedance sensing platform that can provide fast detection of a single physical connection in timescales of milliseconds. Once connectivity is established, the same setup can be used to determine the droplet location. The sensing system can be scaled up to support multiple channels or applied to other heterogeneously integrated systems that require real-time monitoring and diagnostics of multiconnectivity systems.

基于机器视觉的陶瓷瓦表面裂纹检测方法

基于机器视觉的陶瓷瓦表面裂纹检测方法