Automatic Visual Inspection System Research Articles

A visual inspection system for detecting end gap and light leakage of the piston ring

A fully automatic visual inspection system for piston ring was proposed in this study. This system can automatically feed the piston ring, and then detect the end gap and the light leakage of the piston ring. Moreover, the piston ring can be classified after detection and the detection data can be saved for further analysis. To achieve this objective, the detection equipment was established for detecting the end gap and the light leakage, and corresponding software was developed based on Labview. The experiment was carried out and the detection performance of the system was evaluated. The result shows that the proposed system has a good accuracy for detecting the piston ring.

Hyper Parameter Optimization for Transfer Learning of ShuffleNetV2 with Edge Computing for Casting Defect Detection

A casting defect is an expendable abnormality and the most undesirable thing in the metal casting process. In Casting Defect Detection, deep learning based on Convolution Neural Network (CNN) models has been widely used, but most of these models require a lot of processing power. This work proposes a low-power ShuffleNet V2-based Transfer Learning model for defect identification with low latency, easy upgrading, increased efficiency, and an automatic visual inspection system with edge computing. Initially, various image transformation techniques were used for data augmentation on casting datasets to test the model flexibility in diverse casting. Subsequently, a pre-trained lightweight ShuffleNetV2 model is adapted, and hyperparameters are fine-tuned to optimize the model. The work results in a lightweight, adaptive, and scalable model ideal for resource-constrained edge devices. Finally, the trained model can be used as an edge device on the NVIDIA Jetson Nano-kit to speed up detection. The measures of precision, recall, accuracy, and F1 score were utilized for model evaluation. According to the statistical measures, the model accuracy is 99.58%, precision is 100%, recall is 99%, and the F1-Score is 100 %.

MobileNetV2-based Transfer Learning Model with Edge Computing for Automatic Fabric Defect Detection

In textile manufacturing, fabric defect detection is an essential quality control step and a challenging task. Earlier, manual efforts were applied to detect defects in fabric production. Human exhaustion, time consumption, and lack of concentration are the main problems in the manual defect detection process. Machine vision systems based on deep learning play a vital role in the Industrial Internet of things (IIoT) and fully automated production processes. Deep learning centered on Convolution Neural Network (CNN) models have been commonly used in fabric defect detection, but most of these models require high computing resources. This work presents a lightweight MobileNetV2-based Transfer Learning model to assist defect detection with low power consumption, low latency, easy upgrade, more efficiency, and an automatic visual inspection system with edge computing. Firstly, different image transformation techniques were performed as data augmentation on four fabric datasets for the model's adaptability in various fabrics. Secondly, fine-tuning hyperparameters of the MobileNetV2 with transfer learning gives a lightweight, adaptable and scalable model that suits the resource-constrained edge device. Finally, deploy the trained model to the NVIDIA Jetson Nano-kit edge device to make its detection faster. We assessed the model based on its accuracy, sensitivity rate, specificity rate, and F1 measure. The numerical simulation reveals that the model accuracy is 96.52%, precision is 96.52%, recall is 96.75%, and F1-Score is 96.52%.

A Novel Industrial Application of CNN Approach: Real Time Fabric Inspection and Defect Classification on Circular Knitting Machine

Fabric Automatic Visual Inspection (FAVI) system provides reliable performance on fabric defects inspection. This study presents a machine vision system developed to adapt in circular knitting machines where fabric defects can be automatically controlled and detected defects can be classified. The knitted fabric surface are detected during real-time manufacturing. For the classification process, three different transfer learning architectures (ResNet-50, AlexNet, GoogLeNet) have been applied. The five common knitted fabric defects were recognized with the artificial intelligence-based software and classified with an average success rate of 98% using ResNet-50 architecture. The success rates of the trained networks were compared.

UzADL: Anomaly detection and localization using graph Laplacian matrix-based unsupervised learning method

Visual inspection is an essential quality control process in industrial businesses. It is usually automated due to its tedious procedure. An automated visual inspection (AVI) attempts to detect items with abnormal patterns based on image data. Recent developments in computer vision models, especially the introduction of deep convolutional neural networks, has extensively improved the accuracy and speed of AVI systems. However, supervised learning approaches for AVI necessitate a large number of annotated data, while the unsupervised ones lack accuracy and interpretability as well as require an extensive amount of time for training and inference. Therefore, in this study, we propose an unsupervised learning-based computationally inexpensive, efficient, and interpretable model UzADL for AVI to address the aforementioned problems. This system has three principal stages. First, unlabeled images are annotated using a pseudo-labeling algorithm. Second, the obtained instances are trained during a training process stage. Third, identified abnormal instances’ defective regions are explicitly visualized using an anomaly interpretation technique. Owing to an elaborate unsupervised learning method based on the pseudo-labeling algorithm using graph Laplacian matrix that allows transforming defect detection into a classification task, the proposed system has rapid convergence ability and significantly outperforms existing deep learning-based AVI methods. In the experiments conducted with three real-life fabric material databases NanoTWICE, MVTec anomaly detection (MVTec AD), and DWorld datasets UzADL outperformed other methods in terms of accuracy and speed when assessed using several evaluation metrics.

Artificial intelligence-based automatic visual inspection system for built heritage

PurposeHeritage is the latent part of a sustainable built environment. Conservation and preservation of heritage is one of the United Nations' (UN) sustainable development goals. Many social and natural factors seriously threaten heritage structures by deteriorating and damaging the original. Therefore, regular visual inspection of heritage structures is necessary for their conservation and preservation. Conventional inspection practice relies on manual inspection, which takes more time and human resources. The inspection system seeks an innovative approach that should be cheaper, faster, safer and less prone to human error than manual inspection. Therefore, this study aims to develop an automatic system of visual inspection for the built heritage.Design/methodology/approachThe artificial intelligence-based automatic defect detection system is developed using the faster R-CNN (faster region-based convolutional neural network) model of object detection to build an automatic visual inspection system. From the English and Dutch cemeteries of Surat (India), images of heritage structures were captured by digital camera to prepare the image data set. This image data set was used for training, validation and testing to develop the automatic defect detection model. While validating this model, its optimum detection accuracy is recorded as 91.58% to detect three types of defects: “spalling,” “exposed bricks” and “cracks.”FindingsThis study develops the model of automatic web-based visual inspection systems for the heritage structures using the faster R-CNN. Then it demonstrates detection of defects of spalling, exposed bricks and cracks existing in the heritage structures. Comparison of conventional (manual) and developed automatic inspection systems reveals that the developed automatic system requires less time and staff. Therefore, the routine inspection can be faster, cheaper, safer and more accurate than the conventional inspection method.Practical implicationsThe study presented here can improve inspecting the built heritages by reducing inspection time and cost, eliminating chances of human errors and accidents and having accurate and consistent information. This study attempts to ensure the sustainability of the built heritage.Originality/valueFor ensuring the sustainability of built heritage, this study presents the artificial intelligence-based methodology for the development of an automatic visual inspection system. The automatic web-based visual inspection system for the built heritage has not been reported in previous studies so far.

Automated Identification of Wood Veneer Surface Defects Using Faster Region-Based Convolutional Neural Network with Data Augmentation and Transfer Learning

In the lumber and wood processing industry, most visual quality inspections are still done by trained human operators. Visual inspection is a tedious and repetitive task that involves a high likelihood of human error. Currently, new automated solutions with high-resolution cameras and visual inspection algorithms are being tested, but they are not always fast and accurate enough for real-time industrial applications. This paper proposes an automatic visual inspection system for the location and classification of defects on the wood surface. We adopted a faster region-based convolutional neural network (faster R-CNN) for the identification of defects on wood veneer surfaces. Faster R-CNN has been successfully used in medical image processing and object tracking before, but it has not yet been applied for wood panel surface quality assurance. To improve the results, we used pre-trained AlexNet, VGG16, BNInception, and ResNet152 neural network models for transfer learning. The results of the experiments using a synthetically augmented dataset are presented. The best average accuracy of 80.6% was obtained using the pretrained ResNet152 neural network model. By combining all the defect classes, a 96.1% accuracy of finding wood panel surface defects was achieved.

Automatic visual inspection system for quality control of the sandwich panel and detecting the dipping and buckling of the surfaces

In this paper, an automated inspection system is proposed for detecting the location and measuring the size of existing dipping or buckling on the sandwich panel surface using an RGB camera and an inexpensive linear laser. The proposed method, by observing the radiated laser beams on the sandwich panel surface, can localize and calculate the level of dipping and buckling with acceptable accuracy while being robust to vibrations of moving sandwich panel on the production line conveyor. After a complete processing of the panel by the system, a three-dimensional (surface plot) or two-dimensional (heat map) output is produced to assist the production line supervisor to easily inspect the surface quality of the sandwich panels. Our experimental results show that the proposed system can detect and measure the surface defects including dipping and buckling with a high accuracy and performance.

Automatic segmentation method for live fish eggs microscopic image analysis

Automatic visual inspection (AVI) is a popular tool for object detection and analysis in many fields. In fisheries, AVI systems designed for fish egg development analysis are used for fish egg assessments in commercial and experimental fish production systems and in ecological monitoring. The first step in AVI systems is to subdivide an image into meaningful regions. The accuracy of this segmentation directly influences the success of subsequent image analysis operations such as feature selection and classification. At present, a lot of image segmentation techniques are available, but there is no universal segmentation technique suitable for all kinds of images.This paper presents a cascading automatic segmentation method for microscopic digitised images of live common carp (Cyprinuscarpio) eggs in Petri dishes filled with water. The proposed method involves five main steps:1) image pre-processing, 2) image initial segmentation by Otsu’s method, 3) segmentation post-processing based on morphological operations, 4) image edge detection and morphological operations to remove incomplete eggs, and finally 5) a Watershed method was used to resolve the problem of separating adjoining fish eggs. Testing the method in Matlab software verified that fish eggs were recognized, segmented and counted with 100% accuracy compared to manual observation in all 96 test cases.

A unified framework for automated inspection of handheld safety critical devices in production assemblies

Due to portability and ease of usage, handheld devices are widely used in various safety critical industrial applications. A failure in such devices either can endanger life of human(s) or can lead to substantial economic/environmental damage. Therefore, production inspection of handheld safety critical devices requires rigorous verification and validation prior to a field deployment to gain certain level of assurance. The research is aimed at devising a unified inspection framework for automated inspection of hardware, software and mechanical characteristics of handheld safety critical devices in production assemblies for smooth integration of all the required tools and technologies. The framework provides system and process level integration of all the stages of the inspection lifecycle for an end-to-end automated production inspection system. The testbeds are geared towards for reusability, flexibility and configurability of all the resources across various inspection stages. AVI (Automated Visual Inspection) system which exhibit specific characteristics is a core component of the proposed framework. The specific characteristics of the AVI system, a generalized architecture of the handheld safety critical devices for which the proposed framework is applicable. Also an integrated production inspection lifecycle for production assemblies of handheld safety critical devices is presented in this research paper.

Detection of Bond Pad Discolorations at Outgoing Wafer Inspections

Deployment of an automatic visual inspection system in semiconductor industry has become increasingly popular than ever not only due to its relatively high value as a yield analysis tool of outgoing products but more importantly for the prevention of defect escapee. A lot of studies are done on the application of in-line defect scan but the application of outgoing wafer inspection at post-fab environment has been very limited and rarely found in literature. With rapid growth of automotive application in worldwide industry, the importance of quality of the wafer at die level has never been so critical. This paper provides a method for detection of bond pad discolorations at outgoing quality check especially in semiconductor industry. An effective method for detection of the bond pad discolorations was proposed. The advantages and disadvantages of the detection method are discussed. Factors that are affecting the performances of the detection method are also described and analyzed.

A Hierarchical Extractor-Based Visual Rail Surface Inspection System

Rail inspection based on visual inspection system (VIS) has drawn much attention recently, since VIS is automatic, fast, nondestructive, and objective. However, visual rail inspection is still a challenge to accurately identify possible defects involving a large variety of visual appearances. This paper presents an automatic visual inspection system to address these difficulties. Rail surface images are first obtained by the system, and then fed into the proposed hierarchical inspection framework containing coarse extractor and fine extractor. Specifically, the coarse extractor handles this defect inspection problem by exploring characteristics of rail background instead of defect itself, and focuses on finding the background modes. The fine extractor, which integrates the longitudinal context information and transversal prior information, is presented to largely reduce the impacts of other irregulars. The proposed method is evaluated on labeled Type-I and Type-II rail surface discrete defects data sets and a practical rail line. The experimental results demonstrate that our method outperforms the state-of-the-art works.

Processing depth distance data to increase precision of multiple infrared sensors in the automatic visual inspection system

Processing depth distance data to increase precision of multiple infrared sensors in the automatic visual inspection system

インフラ設備外観自動点検システムのためのベクトル量子化を利用した無人飛行体の計測位置決定

Maintenance of infrastructures is a major problem. In order to streamline it, automation of inspections is researched. For complete automation, the measuring position determination method of inspection machines is necessary. This paper presents a method of measuring position determination using vector quantization for automatic visual inspection systems using Unmanned Aerial Vehicles (UAVs). In the proposed method, it is assumed that the 3D model of the target infrastructure, the specification of measurement and UAV’s features are given. Under those assumptions, the 3D model is regarded as the cluster of vectors and the measuring positions are determined using vector quantization. In this paper, the proposed method is compared with the conventional method by a computer simulation assuming visual inspections.

Automatic visual inspection: An approach with multi-instance learning

One of the industrial applications of computer vision is automatic visual inspection. In the last decade, standard supervised learning methods have been used to detect defects in different kind of products. These methods are trained with a set of images where every image has to be manually segmented and labeled by experts in the application domain. These manual segmentations require a large amount of high quality delineations (on pixels), which can be time consuming and often a difficult task. Multi-instance learning (MIL), in contrast to standard supervised classifiers, avoids this task and can, therefore, be trained with weakly labeled images. In this paper, we propose an approach for the automatic visual inspection that uses MIL for defect detection. The approach has been tested with data from three artificial benchmark datasets and three real-world industrial scenarios: inspection of artificial teeth, weld defect detection and fishbone detection. Results show that the proposed approach can be used with weakly labeled images for defect detection on automatic visual inspection systems. This approach is able to increase the area under the receiver-operating characteristic curve (AUC) up to 6.3% compared with the naïve MIL approach of propagating the bag labels.

Application of statistical modeling of image spatial structures to automated visual inspection of product quality

Automated visual inspection (AVI) attracts increasing interest in product quality control both academic and industrial communities, particularly on mass production processes, because product qualities of most types can be characterized with their corresponding surface visual attributes. However, many product images in AVI systems are comprised of stochastically accumulative fragmentations (particles) of local homogeneity, without distinctive foregrounds and backgrounds, which brings great challenges in computer analysis, e.g., rice images, fabric images, and consequently, in the intelligent identification of the product qualities. A method of Weibull distribution (WD)-based statistical modeling of image spatial structures (ISSs) to inspect automatically the product quality is presented. The ISS, obtained with multi-scale and omnidirectional Gaussian derivative filters (OGDFs), was demonstrated to be subject to a representative WD model of integral form based on the theory of sequential fragmentation in advance. The WD-model parameters (WD-MPs) of the ISS, with essential human perceptual significance, were extracted as the visual features for product quality identification. The classification performance of the proposed product quality inspection method, namely, the proposed WD-MP features integrated with an introduced spline regression (SR) classifier in this study, was verified on two case studies in the field of the AVI of product quality, namely, automated rice quality classification, and intelligent fabric quality assessment in the corresponding assembly lines of industrial scale. Experimental results indicate that the proposed WD-MP features can effectively characterize the statistical distribution profiles of ISS of complex grain images, piled with a large number of stochastically accumulative fragmentations. The proposed method provides an effective tool for grain image modeling and analysis and consequently lays a foundation for the intelligent perception of product qualities on assembly lines.

計測機器の特性を考慮したインフラ設備外観点検システムのための無人飛行体の軌道計画

計測機器の特性を考慮したインフラ設備外観点検システムのための無人飛行体の軌道計画

An automatic visual inspection system for cone surface defects

Limited to the impact of human visual sensitivity and individual subjective judgments, traditionally manual detection method is unable to accurately and reliably capture the cone surface defects, and will easily cause a lot of missing and false detection. In this paper, we propose a novel machine vision system for cone surface defect inspection. Specifically, we apply gradient and regional consistency detection algorithms to check the surface defects. And then the detection results will be sent via the serial port to the mechanical separation system, and the defective cones will be peeled off the production line by motor. The proposed system can improve the defect detection efficiency and product quality, also can reduce production costs. Experimental results demonstrate that the proposed approach can obtain superior performance over traditional manual method and has high detection efficiency.

Further development of adaptable automated visual inspection—part I: concept and scheme

The issue of adaptability in the design of automated visual inspection (AVI) systems has been a major road block in the application of AVI technology to manufacturing industry. The objective of an adaptable AVI system is to reduce system tuning time while maintaining the quality of inspection. In the authors’ previous work (Int J Adv Manuf Technol 59:655–667, 2012), we have proposed a novel adaptable AVI scheme, with which an AVI system can be trained online to accommodate new patterns. As such, the system can be deployed to undertake new inspection tasks or adapt to new operation conditions without excessive offline training efforts. The research presented in this two-part article is our further development of the adaptable AVI scheme, namely A2VIS. In this research, we focus on elaborating the concept, designing the structure, implementing the functions, and evaluating the performance of A2VIS in a systemic perspective. In part I, the proposed A2VIS is presented as a generic scheme at the conceptual level, with respect to the system components and structure, operating procedure, and performance measures. The key element in the design of the A2VIS is an adaptable inspection model that integrates subject localization, feature extraction, and classification functions as trainable submodels. In part I of this paper, we developed a generic tool for an adaptable AVI system without specifying the use of particular machine vision and machine learning techniques. Interested readers may choose techniques for implementing the proposed scheme according to their own preferences. In part II of this article, we focus on the system implementation and evaluation of A2VIS using such techniques as template matching, principal component analysis (PCA), and support vector machine (SVM).

Automatic Fastener Classification and Defect Detection in Vision-Based Railway Inspection Systems

The detection of fastener defects is an important task in railway inspection systems, and it is frequently performed to ensure the safety of train traffic. Traditional inspection is usually operated by trained workers who walk along railway lines to search for potential risks. However, the manual inspection is very slow, costly, and dangerous. This paper proposes an automatic visual inspection system for detecting partially worn and completely missing fasteners using probabilistic topic model. Specifically, our method is able to simultaneously model diverse types of fasteners with different orientations and illumination conditions using unlabeled data. To assess the damages, the test fasteners are compared with the trained models and automatically ranked into three levels based on the likelihood probability. The experimental results demonstrate the effectiveness of this method.