Machine Vision Method Research Articles

Semi-Supervised Training for Positioning of Welding Seams.

Robotic welding often uses vision-based measurement to find the correct placement of the welding seam. Traditional machine vision methods work well in many cases but lack robustness when faced with variations in the manufacturing process or in the imaging conditions. While supervised deep neural networks have been successful in increasing accuracy and robustness in many real-world measurement applications, their success relies on labeled data. In this paper, we employ semi-supervised learning to simultaneously increase accuracy and robustness while avoiding expensive and time-consuming labeling efforts by a domain expert. While semi-supervised learning approaches for various image classification tasks exist, we purpose a novel algorithm for semi-supervised key-point detection for seam placement by a welding robot. We demonstrate that our approach can work robustly with as few as fifteen labeled images. In addition, our method utilizes full image resolution to enhance the accuracy of the key-point detection in seam placement.

A Study on Railway Surface Defects Detection Based on Machine Vision.

The detection of rail surface defects is an important tool to ensure the safe operation of rail transit. Due to the complex diversity of track surface defect features and the small size of the defect area, it is difficult to obtain satisfying detection results by traditional machine vision methods. The existing deep learning-based methods have the problems of large model sizes, excessive parameters, low accuracy and slow speed. Therefore, this paper proposes a new method based on an improved YOLOv4 (You Only Look Once, YOLO) for railway surface defect detection. In this method, MobileNetv3 is used as the backbone network of YOLOv4 to extract image features, and at the same time, deep separable convolution is applied on the PANet layer in YOLOv4, which realizes the lightweight network and real-time detection of the railway surface. The test results show that, compared with YOLOv4, the study can reduce the amount of the parameters by 78.04%, speed up the detection by 10.36 frames per second and decrease the model volume by 78%. Compared with other methods, the proposed method can achieve a higher detection accuracy, making it suitable for the fast and accurate detection of railway surface defects.

A Study on the Application of Deep Learning Models for Real-time Defect Detection in the Manufacturing Process : Cases of Defect detection in the Label Printing Process

The global smart manufacturing market is growing rapidly as developed countries (Germany, USA, Japan) as well as late comers such as China are now recognizing the importance of “smart manufacturing” and promoting active policies to foster related ecosystems. Policies to revitalize manufacturing through the convergence of cutting-edge ICT and manufacturing technology are already in progress. Some such policies include Industry 4.0, the Advanced Manufacturing Partnership of the United States, Japan’s Industrial Revitalization Plan, and China’s Made in China 2025. As a result of the gradual shift from product standardization to product customization, the importance of machine vision in manufacturing has also been increasing. However, it is difficult to develop a standard machine vision method because there are different specifications for meeting the individual demands of different manufacturing industries. Moreover, it is difficult to apply such a standard machine vision method to artificial intelligence because defective data for learning in the manufacturing industry are not frequently generated and stored. Therefore, it is conducted manually or visually inspected by workers. This study applies the primary feature of matching models for a label printing process to efficiently detect defects with high performance and applies a deep learning model to maximize performance. Our proposed method achieved an accuracy of 97% with a feature matching model and 99.8% accuracy with the deep learning model.

A machine vision method for measurement of drill tool wear

A machine vision method for measurement of drill tool wear

Deep-Learning-Based Defect Evaluation of Mono-Like Cast Silicon Wafers

Solar cells based on mono-like cast silicon (MLC-Si) have been attracting increasing attention in the photovoltaic (PV) market due to their high energy conversion efficiency and low cost. As in the production of monocrystalline silicon (MC-Si) and polycrystalline silicon (PC-Si) cells, various defects will inevitably occur during the production process of MLC-Si cells. Although computer vision technology has been employed for defect detection in the production processes, it is still difficult to achieve high accuracy in detecting defects in PV cells using traditional machine vision methods due to defect similarity and complex background. To address this challenge, a deep-learning-based quality assessment algorithm of MLC-Si wafers is proposed. Focusing on the dislocation defects, four different deep learning models are used to conduct migration learning and selected different optimizers (ADAM and SGDM) are used to optimize the network models, achieving good results in evaluating and comparing the quality of ML-Si wafers. On this basis, an improved network model MVGG-19 based on the VGG-19 is designed to improve the prediction accuracy further. The experimental results show that the prediction error of the improved network model is reduced by 63% (compared with VGG-19) and the reasoning speed reaches 10.22 FPS, indicating good detection performance.

Overview of the machine vision methods at agrorobots

The idea of "seeing" machines developed violently after the entry of computer technology and industry 4.0 into mass production. The first task of the machine vision is the processing of images and their adjacent equipment, and its systems then include devices with digital inputs/outputs. These inputs/outputs are designed to control some kind of movement or operation of equipment, such as manipulators or automatic lines. The modern machine vision systems are a modern direction of technology that contains elements of computer technology, optics, mechanical systems and industrial automation. With their help, modern agrorobots perform orderly tasks such as visual identification, visual control, obstacle and position determination. This article presents an overview of certain applications of different machine vision systems suitable for use in agrorobots. It is known that they are autonomously powered and work in poor climatic conditions. In other cases, the machine vision in agrorobots must necessarily work in random oscillations and variable focal length, as well as with a wide assortment of objects for research. The article discusses the capabilities of different types of cameras and related software, the application of QR and laser technology for agrorobots.

Infield Apple Detection and Grading Based on Multi-Feature Fusion

A field-based apple detection and grading device was developed and used to detect and grade apples in the field using a deep learning framework. Four features were selected for apple grading, namely, size, color, shape, and surface defects, and detection algorithms were designed to discriminate between the four features using machine vision and other methods. Then, the four apple features were fused, and a support vector machine (SVM) was used for infield apple grading into three grades: first-grade fruit, second-grade fruit, and other-grade fruit. The results showed that for a single index, the accuracy of detecting the apple size, the fruit shape, the color, and the surface defects, were 99.04%, 97.71%, 98%, and 95.85%. The grading accuracies for the first-grade fruit, second-grade fruit, other-grade fruit, and the average grading accuracy based on multiple features were 94.55%, 95.71%, 100%, and 95.49%, respectively. The field experiment showed that the average grading accuracy was 94.12% when the feeding interval of the apples was less than 1.5 s and the walking speed did not exceed 0.5 m/s, meeting the accuracy requirements of field-based apple grading.

Non-Destructive Method in Determining the Barley Grain Vitreousity

Possibilities of non-destructive express-evaluation of the barley grain vitreousity using machine vision and digital image processing methods were studied. The study was carried out with the proprietary design hardware and software complex on barley samples of three different varieties provided by the LLC "Nosters". Transmittance in the near IR wavelength range was used as the objective criterion in classifying grains as vitreous, partially vitreous and better use powdery. Classification group boundaries were determined empirically by the cross-section inspection method. The optimal filming mode was experimentally selected, and the algorithm for digital processing of grain images was developed in order to determine the number of better use powdery grains in a sample. In addition to classifying grains by vitreousity, the proposed approach also makes it possible to evaluate uniformity of a sample by this indicator and, thus, to identify a grain of higher quality. It was found out that grain orientation introduces an error of not more than 5 %, and high repeatability of the results and, as a consequence, accuracy of the algorithm are characterized by the variation coefficient of 1.1 %

Geo-AI to aid disaster response by memory-augmented deep reservoir computing

It is a fact that natural disasters often cause severe damage both to ecosystems and humans. Moreover, man-made disasters can have enormous moral and economic consequences for people. A typical example is the large deadly and catastrophic explosion in Beirut on 4 August 2020, which destroyed a very large area of the city. This research paper introduces a Geo-AI disaster response computer vision system, capable to map an area using material from Synthetic Aperture Radar (SAR). SAR is a unique form of radar that can penetrate the clouds and collect data day and night under any weather conditions. Specifically, the Memory-Augmented Deep Convolutional Echo State Network (MA/DCESN) is introduced for the first time in the literature, as an advanced Machine Vision (MAV) architecture. It uses a meta-learning technique, which is based on a memory-augmented approach. The target is the employment of Deep Reservoir Computing (DRC) for domain adaptation. The developed Deep Convolutional Echo State Network (DCESN) combines a classic Convolutional Neural Network (CNN), with a Deep Echo State Network (DESN), and analog neurons with sparse random connections. Its training is performed following the Recursive Least Square (RLS) method. In addition, the integration of external memory allows the storage of useful data from past processes, while facilitating the rapid integration of new information, without the need for retraining. The proposed DCESN implements a set of original modifications regarding training setting, memory retrieval mechanisms, addressing techniques, and ways of assigning attention weights to memory vectors. As it is experimentally shown, the whole approach produces remarkable stability, high generalization efficiency and significant classification accuracy, significantly extending the state-of-the-art Machine Vision methods.

Estimation of Fruit Load in Australian Mango Orchards Using Machine Vision

The performance of a multi-view machine vision method was documented at an orchard level, relative to packhouse count. High repeatability was achieved in night-time imaging, with an absolute percentage error of 2% or less. Canopy architecture impacted performance, with reasonable estimates achieved on hedge, single leader and conventional systems (3.4, 5.0, and 8.2 average percentage error, respectively) while fruit load of trellised orchards was over-estimated (at 25.2 average percentage error). Yield estimations were made for multiple orchards via: (i) human count of fruit load on ~5% of trees (FARM), (ii) human count of 18 trees randomly selected within three NDVI stratifications (CAL), (iii) multi-view counts (MV-Raw) and (iv) multi-view corrected for occluded fruit using manual counts of CAL trees (MV-CAL). Across the nine orchards for which results for all methods were available, the FARM, CAL, MV-Raw and MV-CAL methods achieved an average percentage error on packhouse counts of 26, 13, 11 and 17%, with SD of 11, 8, 11 and 9%, respectively, in the 2019–2020 season. The absolute percentage error of the MV-Raw estimates was 10% or less in 15 of the 20 orchards assessed. Greater error in load estimation occurred in the 2020–2021 season due to the time-spread of flowering. Use cases for the tree level data on fruit load was explored in context of fruit load density maps to inform early harvesting and to interpret crop damage, and tree frequency distributions based on fruit load per tree.

Application of machine vision method in tool wear monitoring

Application of machine vision method in tool wear monitoring

A machine vision method for measurement of machining tool wear

Tool wears directly affect the quality of product and service life of tool. This paper proposes a machine vision-based measurement method for chisel edge wear of drills. Firstly, the full contour of a drill is extracted by local variance threshold segmentation. Secondly, the image is enhanced by using an adaptive contrast enhancement algorithm based on bidimensional local mean decomposition (BLMD). A threshold segmentation method is proposed to extract contour of the non-worn area. After the above two contours are superimposed, the centroid of each region in the binary image is calculated as the starting point to fill in the overflow water for tool wear extraction. Finally, the chisel edge wear can be measured directly by counting the number of pixels. The experiment in the process of drilling is performed to verify the effectiveness of the proposed method. The experimental results show that the proposed method effectively implements the measurement of tool wear.

A UAV-based machine vision method for bridge crack recognition and width quantification through hybrid feature learning

Bridge crack width is an important indicator to assess and evaluate the health condition of the bridge. In this paper, we have proposed a UAV-based machine vision method for bridge crack recognition and width quantification through hybrid feature learning. Firstly, we have configured a UAV system that can obtain bridge crack image (effective pixel 7952x5304) and GPS position, calculate image resolution, and correct measured plane simultaneously. Then, the crack recognition method combining the R-FCN network and Haar-AdaBoost suited for UAV imagery recognition is proposed, which can make full use of advanced features, shape features and gray features of bridge cracks. The time cost of our method is about 0.2 s for crack detection per one 7952 × 5304 pixels images and 4.5 s in pixel-level segmentation per one 1000 × 1000 pixels bounding box. Additionally, the real bridge crack widths are calculated and quantified by the ranging method using corresponding object distance data. Finally, a case study of the Xiangjiang-River bridge inspection is carried out to demonstrate the effectiveness of the proposed method, achieving above 90% precision in the real bridge crack width quantification.

Rock Particle Motion Information Detection Based on Video Instance Segmentation.

The detection of rock particle motion information is the basis for revealing particle motion laws and quantitative analysis. Such a task is crucial in guiding engineering construction, preventing geological disasters, and verifying numerical models of particles. We propose a machine vision method based on video instance segmentation (VIS) to address the motion information detection problem in rock particles under a vibration load. First, we designed a classification loss function based on Arcface loss to improve the Mask R-CNN. This loss function introduces an angular distance based on SoftMax loss that distinguishes the objects and backgrounds with higher similarity. Second, this method combines the abovementioned Mask R-CNN and Deep Simple Online and Real-time Tracking (Deep SORT) to perform rock particle detection, segmentation, and tracking. Third, we utilized the equivalent ellipse characterization method for segmented particles, integrating with the proportional calibration algorithm to test the translation and detecting the rotation by calculating the change in the angle of the ellipse’s major axis. The experimental results show that the improved Mask R-CNN obtains an accuracy of 93.36% on a self-created dataset and also has some advantages on public datasets. Combining the improved Mask R-CNN and Deep SORT could fulfill the VIS with a low ID switching rate while successfully detecting movement information. The average detection errors of translation and rotation are 5.10% and 14.49%, respectively. This study provides an intelligent scheme for detecting movement information of rock particles.

Gesture recognition by model matching of slope difference distribution features

Gesture recognition has been studied for many decades and remains an open problem. One possible reason is that the features extracted for gesture recognition are not effective enough. A trained person could recognize a variety of gestures robustly based on the contour of the hand. Accordingly, there must be a machine vision method that could recognize the same variety of gestures robustly based on the shape features. The key technique lies in how to define and extract the shape features effectively. In this paper, we propose to recognize of the gestures with the shape features defined and extracted by slope difference distribution (SDD). To calculate the SDD features, one-dimensional hand contour is defined as the distance distribution between the hand centroid and each point on the two-dimensional hand contour. To make the model matching robust, only the common intra-class SDD features are selected as the sparse representation of the hand contour. The proposed gesture recognition method was tested with three public datasets and achieved state-of-the-art recognition accuracy on two public datasets.

A Machine Vision Rapid Method to Determine the Ripeness Degree of Olive Lots.

The degree of olive maturation is a very important factor to consider at harvest time, as it influences the organoleptic quality of the final product, for both oil and table use. The Jaén index, evaluated by measuring the average coloring of olive fruits (peel and pulp), is currently considered to be one of the most indicative methods to determine the olive ripening stage, but it is a slow assay and its results are not objective. The aim of this work is to identify the ripeness degree of olive lots through a real-time, repeatable, and objective machine vision method, which uses RGB image analysis based on a k-nearest neighbors classification algorithm. To overcome different lighting scenarios, pictures were subjected to an automatic colorimetric calibration method—an advanced 3D algorithm using known values. To check the performance of the automatic machine vision method, a comparison was made with two visual operator image evaluations. For 10 images, the number of black, green, and purple olives was also visually evaluated by these two operators. The accuracy of the method was 60%. The system could be easily implemented in a specific mobile app developed for the automatic assessment of olive ripeness directly in the field, for advanced georeferenced data analysis.

Research on Phone Shell Detection Based on Machine Vision

With the continuous development of computer technology, machine vision technology has the incomparable advantages of manual detection, such as non-contact, non-fatigue, so it is widely used in the field of industrial detection. The traditional method of mobile phone shell detection does not meet the increasingly intelligent demand. It is of practical significance to use machine vision technology to realize mobile phone shell detection. Combined the technology of image processing technology, this paper proposes a kind of mobile phone shell detection based on machine vision method. First of all, the original image is preprocessed, and then a method combining median filter and wavelet tool is used to eliminate image noise. On this basis, image segmentation and edge extraction are carried out. The improved Canny operator is used to detect the edge of the image, which has good effect in eliminating the false edge and ensuring the integrity and continuity of the edge. Meanwhile, mathematical morphology is used to extract the characteristic parameters of the target, and the effectiveness of the detection method is verified by simulation in Matlab.

Research on defect pattern recognition of Light Guide Plate based on Deep Learning semantic Segmentation

Because the defects of the light guide plate are still extremely small under the image taken by the high-resolution industrial camera, and the characteristics of different defects are different, as well as the texture characteristics of the whole light guide plate, such as dense and uneven distribution of light guide points, the traditional image processing and detection methods require experienced visual engineers to do a lot of feature extraction algorithm programming and expensive code maintenance. The accuracy is low and the stability is poor. However, the surface defects of the light guide plate are still mainly detected by artificial visual observation, and only a few manufacturers use traditional image processing methods to detect them. For this reason, a defect detection method based on deep learning semantic segmentation is proposed. In this method, the defect features of the light guide plate are extracted by self-learning by training the neural network, so as to avoid the complicated programming work of feature extraction algorithm. First of all, the defects of the collected light guide plate are marked to make a sample set; secondly, the pre-trained pyramid scene parsing network (PSPNet) is used to retrain the labeled samples; furthermore, the defect detection of the light guide plate is realized by using the trained model. The single deep learning semantic segmentation defect detection method usually can not meet the needs of industrial applications, finally, it is necessary to combine the simple machine vision method to judge and screen all the suspected defect regions detected by the deep learning semantic segmentation method. The experimental results show that the detection rate of bright spots, dark spots and scratches is as high as 96%, which can basically meet the requirements of industrial inspection.

Chisel edge wear measurement of high-speed steel twist drills based on machine vision

Tool wear is usually the most relevant parameter of tool performance detection, which directly affects the product quality and tool life. Online tool condition monitoring can effectively reduce workpiece scrap and machine failure. Chisel edge wear is one of the main wear forms in high-speed steel (HSS) twist drill. A machine vision method for chisel edge wear measurement is proposed in this paper to improve measure accuracy and reduce testing costs. Firstly, Otsu segmentation based on Laplacian edge information is proposed for tool image segmentation. A morphology-based Canny operator is developed to perform edge detection and image registration for rough positioning of the tool wear area. Zernike moment-based sub-pixel edge detection is proposed to improve the measurement accuracy. Finally, principal curve is employed to fit sub-pixel edge points to obtain smooth edge curve of the tool wear. Finally, the chisel edge wear can be detected and measured effectively based on the proposed method. In the real machining process, the test results show that the proposed system shows high response speed and inspecting accuracy, which is more convenient and efficient than manual measurement with an electron microscope. This system can be effectively applied to the real-time monitoring of tool wear in industry.

Measurement of local strain: machine vision and finite element method

Determining the material deformation behavior including local strain data is very useful for analyzing the forming processes. In order to measure strain properties, the uniaxial tensile test was used. More than 10 important mechanical properties were obtained from the uniaxial tensile test. However, through this test, only one strain in the tensile direction can be reported. Obtaining strain at various points in the transverse and longitudinal directions can provide important information about anisotropy, the Poisson’s ratio and material heterogeneity. In this study, machine vision method along with graphical programming were used for automatic strain calculations. Also, an innovative numerical method was developed based on machine vision procedure. In this new method, a number of circles were carved as a matrix pattern on the finite element model. Removing the limitations of the machine vision method, including its sensitivity to vibration, is the most important advantage of this numerical model. The results of the finite element method were evaluated in comparison with the tensile test and the machine vision procedure.