Automatic Surface Inspection Research Articles

Multiple Guidance Network for Industrial Product Surface Inspection With One Labeled Target Sample.

Most automatic product surface inspection methods in industry are data-hungry and task-specific. It is difficult to collect adequate labeled samples in practice due to factors including expensive data annotation cost, inadequate samples for some categories, and limitations on the initial production stage. In this article, a multiple guidance network (MGNet) is proposed to address these issues. In the network, the feature extraction machine (FEM) produces four feature maps of different functions to enhance the inspection ability of the algorithm. Also, the probability map generation (PMG) module is designed for coarse positioning of objects. Moreover, the structures of the mutual guidance and historical guidance (HG) guarantee that the network can fully utilize the information of the auxiliary dataset. Only one support sample containing the labeled objects is required for reference, and the network can determine whether the same labeled objects exist in the query images and locate them. For a comprehensive evaluation of MGNet, three experiments are carried out using three real-world datasets. Experiment results verify that the proposed method is promising for industrial product surface inspection with one labeled target sample.

Preview the Products on Display at Display Week 2023

Preview the Products on Display at Display Week 2023

Accelerated In Situ Inspection of Release Coating and Tool Surface Condition in Composites Manufacturing Using Global Mapping, Sparse Sensing, and Machine Learning

The interfacial adhesion, friction, and resulting tool-part interaction during composites manufacturing contribute to the formation of residual stresses and process-induced deformations (PIDs). Tool-part interaction and PIDs are highly sensitive to processing variabilities, one of which is the aging of the release coating and the surface condition of production tools. Unfortunately, due to a lack of available tool inspection methods, manufacturers often attempt to mitigate the aging of release coating based on know-how, leading to cost-deficient tool preparation schedules, lower end-part quality, and in some cases, higher levels of PIDs. This paper presents an in-situ inspection method to evaluate the physicochemical properties of release coating and the surface condition of large production tools by utilizing global mapping, sparse sensing, and machine learning (ML). ML methods are used in conjunction with multiple automated measurement techniques to quickly identify the condition of release coating or contamination on production tool surfaces in manufacturing environments. Results in this paper demonstrate that during autoclave processing, aerospace-grade release coatings undergo significant chemical changes, but may remain highly abhesive for more than twenty autoclave processing cycles. Using the proposed novel inspection technology, dry ice blasting (DIB) is also demonstrated to be an effective method for non-abrasively removing cured resin contamination and release coating from a tool surface. Overall, this paper demonstrates how the proposed inspection method can be integrated into a manufacturing process for automatic surface inspection of large tools to improve production efficiency and potentially mitigate PIDs in composites manufacturing.

Ferrite Beads Surface Defect Detection Based on Spatial Attention Under Weakly Supervised Learning

Ferrite beads automatic surface inspection is an important means to improve the quality and ensure proper operation. Although the deep learning based defect inspection methods reveal powerful performance, these methods often require a large amount of expensive annotation data for training, which limits the practical application of deep learning based inspection methods. To solve this problem, we propose a weakly supervised learning defect detection algorithm to achieve both high accuracy identification and effective localization of defects while using only image-level labels. In order to better utilize the texture location information of defects, we present a spatial association module(SAM) based on shallow features to improve the network performance. Then a training enhancement method is proposed to improve the detection ability, in which the guide crop and object ignore algorithms are used to extract the main defect area and background area in the image respectively to assist in generating optimal decisions. Finally, we put forward an optimal inference method to improve the completeness of localization without sacrificing accuracy, so as to provide a more reasonable and effective visual explanation for defect recognition. On the ferrite bead dataset, the proposed method uses less than 200 defect samples with only image-level labels in the training process to achieve a classification average precision of 97.1% with good stability and reliability, which has been employed in the ferrite bead inspection machine. To further verify the superiority and generalization, the proposed method is evaluated on several datasets for industrial quality inspection: Deutsche Arbeitsgemeinschaft fuer Mustererkennung(DAGM), KolektorSDD, and KolektorSDD2 and achieve the best average precision of defect classification of 100%, 100% and 99.9%, respectively.

Automatic defect detection of metro tunnel surfaces using a vision-based inspection system

Due to the impact of the surrounding environment changes, train-induced vibration, and human interference, damage to metro tunnel surfaces frequently occurs. Therefore, accidents caused by the tunnel surface damage may happen at any time, since the lack of adequate and efficient maintenance. To our knowledge, effective maintenance heavily depends on the all-round and accurate defect inspection, which is a challenging task, due to the harsh environment (e.g., insufficient illumination, the limited time window for inspection, etc.). To address these problems, we design an automatic Metro Tunnel Surface Inspection System (MTSIS) for the efficient and accurate defect detection, which covers the design of hardware and software parts. For the hardware component, we devise a data collection system to capture tunnel surface images with high resolution at high speed. For the software part, we present a tunnel surface image pre-processing approach and a defect detection method to recognize defects with high accuracy. The image pre-processing approach includes image contrast enhancement and image stitching in a coarse-to-fine manner, which are employed to improve the quality of raw images and to avoid repeating detection for overlapped regions of the captured tunnel images respectively. To achieve automatic tunnel surface defect detection with high precision, we propose a multi-layer feature fusion network, based on the Faster Region-based Convolutional Neural Network (Faster RCNN). Our image pre-processing and the defect detection methods also promising performance in terms of recall and precision, which is demonstrated through a series of practical experimental results. Moreover, our MTSIS has been successfully applied on several metro lines.

DeepInspection: Deep learning based hierarchical network for specular surface inspection

• Image fusion based on deflectometry principle can enhance the contrast between defective and non-defective regions. • A benchmark dataset of specular surface defects was constructed for algorithms evaluation and comparison. • An end-to-end attention-based network framework for automatic surface inspection is proposed. • Proposed framework can maintain performance dealing with extremely unbalanced pixel classes. • Quantitative and qualitative tests show superior performance compared to the state-of-art methods. Automated defect detection on specular vehicle surface with limited features (of up to 0.7 mm in diameter or width) and extremely unbalanced pixel classes is a still challenge of product quality control in automotive industry. The traditional defect inspection on specular surface is usually performed by inspectors, which is subjective, unstable and unquantified. Also, due to the limited features of isolated defect regions and hand-crafted feature extraction models may not be able to coordinated with each other, it is difficult for traditional methods to achieve comparable learning performance with the deep network. To alleviate these problems, a novel end-to-end attention-based fully convolutional neural network framework -DeepInspection is proposed for automated defect inspection on specular surface. Specifically, a sequence fusion algorithm through the principle of deflectometry is introduced to enhance the contrast between defective regions (pixels with lower intensity) and non-defective regions (pixels with higher intensity). Then, an attention-based deep convolutional neural network (DCNN) with Atrous Spatial Pyramid Pooling (ASPP) was proposed to capture local-to-global feature presentation from pre-processed fused images. To verify the robustness and effectiveness of the proposed method, a benchmark dataset called DeepInspection160 with 160 manually labeled images is established. Although the defective pixels only account for 0.561% in the DeepInspection160 dataset, the proposed DeepInspection framework still surpasses several state-of-the-art specular surface inspection methods which achieves F1 score over 0.7513 (pixel level) and 0.8055 (individual connected components) on the proposed challenging dataset.

Automatic surface inspection for S-PVC using a composite vision-based method.

Appearance defect inspection is crucial for quality control in the context of Industry 4.0. This research introduces a joint surface defect inspection and classification framework for polyvinyl chloride (PVC) pipe based on the low-cost visual sensors and high-efficiency computer vision algorithms. First, we build a robust imaging system to acquire the surface of PVC (S-PVC) by considering its characteristics and the illumination condition into the modeling process. Second, we adopt the region of interest method to eliminate the background interference captured in the S-PVC imaging and design an efficient S-PVC defect inspection and classification method. Third, we build an automatic machine prototype to evaluate the efficiency of the proposed method. Experimental results demonstrate that our framework has the advantages of low latency, high precision, and robustness.

Defective samples simulation through adversarial training for automatic surface inspection

Deep learning has shown great potential in machine vision. However, introducing deep learning to automated surface inspection is still a challenging task because it depends heavily on the quality of training samples, especially the defective samples. In this paper, a general defective samples simulation method based on generative adversarial nets (GAN) is proposed to deal with the limitation of defective samples in production. Under the GAN framework, the simulative network with encoder–decoder architecture is proposed. Further, the simulative network and discriminative network are trained adversarially under the proposed regional training strategy, which gives priority to the translation of the defective area. Finally, the defect-free area is refined through wavelet fusion. This method requires a small number of defective training samples, and can generate simulative defects of specified shapes and types and meanwhile obtain the pixel-wise ground truth. The simulative samples can be used directly for training of deep learning based automated surface inspection tasks. We conduct experiments on four datasets. The experimental results show that our method can generate defective samples of higher quality than general image translation methods. Applying this method to surface defect inspection can significantly improve the effect of defect inspection model based on deep learning.

Quality Control system for a hot-rolled metal surface

The modern ideas about of quality of products are based on the principle of the absolute satisfaction of requirements of recommendations of the buyer. A presence of surface defects of steel-smelting and rolling origin is peculiar to the production of hot-rolling mill. The automatic surface inspection system (ASIS) includes two digital line video cameras for the filming of the upper and lower surfaces of the flat bar, block of illumination of the upper and lower surfaces of the flat bar, computer equipment. A system that secures 100 % control of the surface of rolled metal (of the upper and lower side) detects automatically and classifies the sheet defects in the real time mode was mounted in the domestic practice in the first time in 2003 on hot rolling mill 2000 JSC «Novolipetsk Iron & Steel Corporation» (NISC). The whole assortment of the mill 2000 was divided for the five groups by the outward appearance of the surface. The works on the identification of defects of hot-rolled metal and widening of data base of knowledge of ASIS were continued after the carrying out of guarantee tests. More than 10 thousand images of defects were added to the data base during the year.

Optimization of an Electromagnetic Technology in ArcelorMittal Gent for Improving Products Quality in Steel Industry

AbstractA multi-mode electromagnetic stirring technology (MM-EMS) delivered by Danieli–Rotelec Company has been implemented in ArcelorMittal Gent CC2. This solution is able to control the liquid steel flows in CC mould by braking, accelerating or stirring them through the use of three different modes: EMLS, EMLA and EMRS. This paper presents the methodology followed in order to optimize the working of this new technology for improving the quality of products in terms of slivers defects. Different tools were developed and used by ArcelorMittal Maizières R&D to reach this objective: Numerical simulation with MagnetoHydroDynamics (MHD) tool to simulate the fluid flows in mould under the influence of magnetic fields, Submeniscus Velocity Control (SVC) tool to directly measure the liquid steel flows in the mould and finally some quality investigation tool like Automatic Surface Inspection System to detect defects directly on coils and qualify quality of products. Numerical and experimental approaches were associated to finally deliver to ArcelorMittal Gent steelplant an optimized actuator efficient for reducing defects in slabs and improving the final quality of products.

Digital Factory Approach for Flexible and Efficient Manufacturing Systems in the Aerospace Industry

This paper is focused on the employment of Digital Factory methodologies with the aim of improving the flexibility and the efficiency of a real manufacturing cell dedicated to the precision manufacturing of mechanical components for the aerospace industry. An automation upgrade of the cell is proposed, involving the introduction of an industrial robot for part handling, loading and unloading, and a new automatic surface inspection system (ASIS) for the final part inspection. The research activities are carried out within one of the projects of the Italian National Technology Cluster on Intelligent Factory. In this framework, the digital validation of the real manufacturing system is performed through Discrete Event Simulation to analyze the performance of the manufacturing system in terms of productivity and utilization of resources under different alternative scenarios, avoiding the risks associated with physical experimentation on the real system.

Use of Artificial Intelligence in Classification of Mill Scale Defects

The subject of the work was to design and implement classification model for various kinds of mill scales recognition at typical hot rolling mill (HRM). Input data were measured by the automatic surface inspection system, which provided numerous features describing single image, considered as a defect. The data were analyzed by using feature selection methods. Afterwards, determined most important subsets of features, were applied to build scale classification models. The most efficient model and its key variables were used to distinguish different kinds of scales. Implementation of the model in production system allowed to show a gain of overall mill scales classification accuracy. In the paper, the main attention is put on the research methodology and artificial intelligence (AI) methods used to classify scale defects.

Enhanced anomaly detection in wire ropes by combining structure and appearance

Automatic visual surface inspection is a challenging task, which has become important for quality assurance in the last years. Wire rope inspection is a special problem within this field. Usually, the huge and heavy ropes cannot be detached. Thus, an inspection of the ropes must be conducted, while the ropes are in use. The rope surface exhibits various appearance characteristics so that the existing, purely appearance-based approaches tend to fail.We explicitly integrate information about the object geometry, which we obtain by aligning a sequence of 2d rope images with a perfectly regular 3d model of the rope. The rendering equation is used to link object geometry to the observed rope appearance. Based on the connection between geometry and surface appearance we build a probabilistic appearance model which serves as representation for normal surface variations. A robust localization of rope surface defects is achieved by means of anomaly detection. The presented approach has no need for knowledge about the illumination setting or the reflectance properties of the material.An evaluation on real-world data from ropeways leads to an accuracy comparable to that of a human expert. With an accuracy of 95% and a false-alarm-rate of 1.5% the approach outperforms all other existing approaches.

An Enhanced Automatic Surface and Structural Flaw Inspection and Categorization using Image Processing Both for Flat and Textured Ceramic Tiles

As the price of ceramic tiles depend on its limpidness and precision of surface texture, color and shape, it’s in fact a great challenge to control surface eminence and uphold production rate in the field of industrial fabrication of ceramic tiles. Under consideration these criteria, in this research paper, we have proposed an enhanced automatic surface flaw detection and categorization procedure that is able to guarantee the quality of ceramic tiles as well as production rate in industrial fabrication. Our proposed model plays an important role for automatic revealing of surface flaw during production and packaging. This proposed model includes an automatic categorization technique using computer vision that helps us to make sense about the pattern of surface defect within a very short time and also helps to make quick decision about the recovery process. Moreover, it also ensures the quality of tiles automatically during packaging procedure so that the defected tiles may not be mixed up with the fresh tiles.

이동평균 영상처리기법을 이용한 금속판재 표면품질 정밀 측정시스템 연구

산업자동화를 위한 영상처리기술을 활용하여 금속 표면 검사 공정의 육안 검사를 대체하기 위한 장치개발이 요구된다. 본 논문에서는 압연 공정의 금속 판재 표면의 결함을 탐지하기 위하여 이동평균 영상처리기법을 적용한 정밀측정시스템을 제안하였다. 금속 판재의 압연 처리 과정에서 발생하는 표면 패턴이 결함으로 판정될 수 있다. 이러한 경우에 대하여 제안한 시스템을 적용하여 금속 판재 표면의 실제 결함의 수를 측정할 수 있었고 FFT방법에 의한 측정치에 비하여 양호한 결과를 얻었다. It has been highly required to develope an automatic metal surface inspection system, specifically using image processing techniques, which can replace the visual inspection method in the steel industry. In this paper, we propose a precisional surface measurement system using the moving average image processing technique. When the surface patterns which are generated in the rolling process of metal plates are recognized as defects, the proposed system can measure the actual number of defects. It has been proved that our system shows better results than the conventional FFT method.

Automatic surface inspection of vehicles

One major challenge presently faced by the automotive industry is the automated investigation of large-scale objects with specular or partially specular surfaces. There are a few measurement systems on the market which allow for the two-dimensional inspection of reflective surfaces, but there are currently no processes which use the available technology to measure large areas. The newly established company adomea GmbH aims to resolve this problem by further developing the fringe reflection method.

Automatic surface inspection for directional textures using nonnegative matrix factorization

A global image restoration scheme using nonnegative matrix factorization (NMF) is proposed in this paper. This NMF-based image restoration scheme can be used for inspecting the defects in directional texture surfaces automatically. Decomposing the gray level of image pixels into an ensemble of row vectors, we first reduce the data set from original data space into a lower-dimensional NMF space. The repetitive and periodical primitives are well reconstructed by two lower-dimensional basis and weight matrices with nonnegative elements, named nonnegative matrix approximation (NMA). Then the local defects will be revealed by applying image subtraction between the original image and the NMA. As a consequence, the directional textures are eliminated, and only local defects are preserved if they initially are embedded in the surface. A supervised heuristic, elbow of residual curve rule, is devised which helps users to determine a proper basis space size of a specific image. Experiments on a variety of directional texture surfaces are given to demonstrate the effectiveness and robustness of the proposed method.

New Structured Illumination Technique for the Inspection of High-Reflective Surfaces: Application for the Detection of Structural Defects without any Calibration Procedures

We present a novel solution for automatic surface inspection of metallic tubes by applying a structured illumination. The strength of the proposed approach is that both structural and textural surface defects can be visually enhanced, detected, and well separated from acceptable surfaces. We propose a machine vision approach and we demonstrate that this technique is applicable in an industrial setting. We show that recording artefacts drastically increases the complexity of the inspection task. The algorithm implemented in the industrial application and which permits the segmentation and classification of surface defects is briefly described. The suggested method uses perturbations from the stripe illumination to detect, segment, and classify any defects. We emphasize the robustness of the algorithm against recording artefacts. Furthermore, this method is applied in 24 h/7 day real-time industrial surface inspection system.

Surface Inspection System Using the Real-Time Image Processing

We have developed an innovative surface inspection system for automated quality control for steel productions in POSCO. We had ever installed the various kinds of surface inspection systems. such as a linear CCD and a laser typed surface inspection system at cold rolled strip production lines. But these systems cannot fulfill the sufficient detection and classification rate. and real time processing performance. In order to increase detection and classification rate. we have used the Dark. Bright and Transition field illumination and area type CCD camera and for the real time image processing parallel computing has been used. In this paper we introduce the automatic surface inspection system and real time image processing technique using the object detection, defect detection, classification algorithm and its application to automatic quality control system.

Automatic surface inspection system : experience at Usinor and outloock

Since about 10 years, Usinor has been extensively developing and implementing automatic surface inspection systems (designated as ASIS). Research work enhanced by industrial developments in partnership with Matra, has produced systems capable of detecting and classifying defects according to the expectations of the users. The data obtained are transformed by a computer-aided post-treatment into automatic grading of coil quality and into guidelines to the operator for the evaluation of its compliance with specifications.