Visual Inspection System Research Articles

Multiobjective Optimization Strategy for Enhancing the Efficiency and Quality of Organic Thin-Film Manufacturing with Electrohydrodynamic Atomization Coating.

Electrohydrodynamic atomization coating technology is well-suited for micro-/nanoscale thin-film additive manufacturing. However, there are still some challenges in quality control and parameter adjustment during the coating process. Especially when coating on nonconductive and nonhydrophilic substrates, film quality and thickness uniformity are difficult to control. This paper proposes an optimization strategy for enhancing the efficiency and quality of thin-film manufacturing on nonconductive, nonhydrophilic glass substrates. In this paper, a visual inspection system was developed for in situ inspection and identification of droplet deposition states in the substrate surface. Then, the statistical relationship between the operating parameters and the quality of the deposition state was analyzed by response surface methodology. On this basis, machine learning models and intelligent recommendation frameworks for small data sets were developed to rapidly optimize operating parameters and improve the quality of thin-film coating. Optimization strategy developed by applying the principles of statistical modeling, analysis of variance, and global optimization are more efficient and less costly than traditional parameter screening methods. The experimental results show that optimum deposition quality can be obtained with the recommended operating parameters. And, validation results show a 12.8% improvement in film thickness uniformity. At the same time, no mura defects appeared on the thin-film surface. The proposed optimization strategy can improve the efficiency and quality of additive manufacturing of micro and nano thin films and is beneficial for advancing industrial applications of the electrohydrodynamic atomization coating.

The importance of image quality in AI visual inspection systems

AbstractThe use of AI for visual inspection is currently making great strides in sectors like automotive, metals and machinery manufacturing. However, those investing in the technology not only expect a system that will be superior to manual inspection, but an error rate close to zero in defect detection. Let us look at the importance of image quality and why the variability in the production environment is the single greatest challenge that AI engineers must face.

Advanced Defect Detection in Wrap Film Products: A Hybrid Approach with Convolutional Neural Networks and One-Class Support Vector Machines with Variational Autoencoder-Derived Covariance Vectors

This study proposes a novel approach that utilizes Convolutional Neural Networks (CNNs) and Support Vector Machines (SVMs) to tackle a critical challenge: detecting defects in wrapped film products. With their delicate and reflective film wound around a core material, these products present formidable hurdles for conventional visual inspection systems. The complex task of identifying defects, such as unwound or protruding areas, remains a daunting endeavor. Despite the power of commercial image recognition systems, they struggle to capture anomalies within wrap film products. Our research methodology achieved a 90% defect detection accuracy, establishing its practical significance compared with existing methods. We introduce a pioneering methodology centered on covariance vectors extracted from latent variables, a product of a Variational Autoencoder (VAE). These covariance vectors serve as feature vectors for training a specialized One-Class SVM (OCSVM), a key component of our approach. Unlike conventional practices, our OCSVM does not require images containing defects for training; it uses defect-free images, thus circumventing the challenge of acquiring sufficient defect samples. We compare our methodology against feature vectors derived from the fully connected layers of established CNN models, AlexNet and VGG19, offering a comprehensive benchmarking perspective. Our research represents a significant advancement in defect detection technology. By harnessing the latent variable covariance vectors from a VAE encoder, our approach provides a unique solution to the challenges faced by commercial image recognition systems. These advancements in our study have the potential to revolutionize quality control mechanisms within manufacturing industries, offering a brighter future for product integrity and customer satisfaction.

Stochastic geometry models for texture synthesis of machined metallic surfaces: sandblasting and milling

Training defect detection algorithms for visual surface inspection systems requires a large and representative set of training data. Often there is not enough real data available which additionally cannot cover the variety of possible defects. Synthetic data generated by a synthetic visual surface inspection environment can overcome this problem. Therefore, a digital twin of the object is needed, whose micro-scale surface topography is modeled by texture synthesis models. We develop stochastic texture models for sandblasted and milled surfaces based on topography measurements of such surfaces. As the surface patterns differ significantly, we use separate modeling approaches for the two cases. Sandblasted surfaces are modeled by a combination of data-based texture synthesis methods that rely entirely on the measurements. In contrast, the model for milled surfaces is procedural and includes all process-related parameters known from the machine settings.

Design of hexapod robot equipped with omnidirectional vision sensor for defect inspection of pipeline’s inner surface

Abstract Defect detection of inner surface of precision pipes is a crucial aspect of ensuring production safety. Currently, pipeline defect detection primarily relies on recording video for manual recognition, with urgent need to improve automation, quantification and accuracy. This paper presents a hexapod in-pipe robot with carrying capacity designed to transport the omnidirectional vision sensor to specified location within unreachable pipelines. The feasibility of the robot’s mechanical design and sensor load-carrying module is analyzed using theory calculations, motion simulations and finite element method. To address the challenges of small pixel ratio and weak background changes in panoramic images, a tiny defect segmentor (TDS) based on ResNet is proposed for detecting tiny defects on the inner surface of pipelines. The hardware and software systems are implemented, and the motion performance of the pipeline robot is validated through experiments. The results demonstrate that the robot achieves stable movement at a speed of over 0.1m/s and can adapt to pipe diameter ranging from of 110 to 130 mm. The novelty of the robot lies in providing stable control of the loaded vision sensor, with control precision of the rotation angle and the displacement recorded at 1.84% and 0.87%, respectively. Furthermore, the proposed method achieves a detection accuracy of 95.67% for tiny defects with a diameter less than 3 mm and provides defect location information. This pipeline robot serves as an essential reference for development of in-pipe 3D vision inspection system.

Next-generation Vision Inspection Systems: a pipeline from 3D model to ReCo file

Next-generation Vision Inspection Systems: a pipeline from 3D model to ReCo file

Intelligent Inspection Method and System of Plastic Gear Surface Defects Based on Adaptive Sample Weighting Deep Learning Model.

After injection molding, plastic gears often exhibit surface defects, including those on end faces and tooth surfaces. These defects encompass a wide range of types and possess complex characteristics, which pose challenges for inspection. Current visual inspection systems for plastic gears suffer from limitations such as single-category defect inspection and low accuracy. There is an urgent industry need for a comprehensive and accurate method and system for inspecting defects on plastic gears, with improved inspection capability and higher accuracy. This paper presents an intelligent inspection algorithm network for plastic gear defects (PGD-net), which effectively captures subtle defect features at arbitrary locations on the surface compared to other models. An adaptive sample weighting method is proposed and integrated into an improved Focal-IoU loss function to address the issue of low inspection accuracy caused by imbalanced defect dataset distributions, thus enhancing the regression accuracy for difficult defect categories. CoordConv layers are incorporated into each inspection head to improve the model's generalization capability. Furthermore, a dataset of plastic gear surface defects comprising 16 types of defects is constructed, and our algorithm is trained and tested on this dataset. The PGD-net achieves a comprehensive mean average precision (mAP) value of 95.6% for the 16 defect types. Additionally, an online inspection system is developed based on the PGD-net algorithm, which can be integrated with plastic gear production lines to achieve online full inspection and automatic sorting of plastic gear defects. The entire system has been successfully applied in plastic gear production lines, conducting daily inspections of over 60,000 gears.

Effects of visual neuroplasticity training on visual perception, visual quality, and macular blood flow in patients with strabismus.

To observe the effects of visual neuroplasticity training on visual perception, visual quality, and macular blood flow in patients with concomitant strabismus postoperatively. In total, 108 patients underwent binocular strabismus correction operation, and some patients underwent neuroplasticity training. All patients underwent clinical ophthalmic examination, including measurement of best-corrected visual acuity, spherical equivalent, axis length, optical coherence tomography angiography, optical quality analysis system, and visual perception examinations. A total of 78 patients received neuroplasticity training for 1 month postoperatively, and 30 patients did not receive training. All patients underwent a visual perception examination preoperatively and at 1 day, 1 week, and 1 month postoperatively. Macular blood flow and visual quality were examined preoperatively and at 1 month postoperatively. Postoperative visual perception was better than preoperative visual perception (P < 0.05). After neuroplasticity training, the visual perception of the trained subjects was better than that of the untrained subjects (P < 0.05), and the blood flow in the macular area of the trained patients was lower than that of the untrained subjects (P < 0.05). The visual quality of the untrained subjects was lower than that of the trained patients (P < 0.05). Visual inspection system could accurately evaluate binocular visual perception in patients with concomitant strabismus. After surgical alignment of the strabismus patient, training can stimulate and integrate the formation of stereovision in a short period of time, maintain the visual quality of patients after surgery, and provide conditions for the formation of binocular visual signals and binocular stereovision, but in the short term, it will lead to the decrease of macular blood vessel density and perfusion density. However, the long-term effects of training have not been proven.

An UAV system for visual inspection and wall thickness measurements in ship surveys

This article presents the development of a UAV system designed to perform visual inspections and thickness measurements in ships’ cargo tanks. That includes the conception and evolution of the different components and subsystems of this UAV system and several tests, including some carried out in actual operating conditions. It also presents a near-wall positioning and propulsion system designed and developed to improve stability and avoid collisions. The paper also describes an articulated arm designed around an EMAT probe for wall thickness measurement. The measurement chain developed for this purpose includes all the electronics and computing necessary to send data in real-time to a ground station. Flying near walls poses challenges that compromises accuracy when approaching each measurement target points. Therefore, the wall approach and departure flight maneuvers are fully automatic. That facilitates and speeds up the inspection task that may require measurements on thousands of locations specified by the surveyor. The proposed UAV system aims to automate and improve the long and expensive surveys and inspections in the marine industry by reducing their time and costs.

Automated fabric inspection system development aided with convolutional autoencoder-based defect detection

Industrial automatic fabric inspection system, a critical technology in the industry, enhances both total production quantity and quality compared to conventional inspection techniques. This study aims to create a reliable and effective real-time automated visual inspection system for fabrics, focusing on defect detection. The goals of the study can be stated as; installing a system with advanced technology for capturing and processing images swiftly, the development and deployment of a system capable of autonomously learning and scanning fabrics in use, and the creation of a smart framework for accurate fabric defect detection and classification. We focus on the development of unsupervised fabric defect detection using a convolutional autoencoder model, and defect classification using a convolutional neural network model, which takes input as the feature vector generated by the convolutional autoencoder. The experimental outcomes have displayed significant success rates in both detecting defects and classifying them, confirming the effectiveness of the framework in real-time visual inspection systems.

Utilizing Deep Learning for Defect Inspection in Hand Tool Assembly.

The integrity of product assembly in the precision assembly industry significantly influences the quality of the final products. During the assembly process, products may acquire assembly defects due to personnel oversight. A severe assembly defect could impair the product's normal function and potentially cause loss of life or property for the user. For workpiece defect inspection, there is limited discussion on the simultaneous detection of the primary kinds of assembly anomaly (missing parts, misplaced parts, foreign objects, and extra parts). However, these assembly anomalies account for most customer complaints in the traditional hand tool industry. This is because no equipment can comprehensively inspect major assembly defects, and inspections rely solely on professionals using simple tools and their own experience. Thus, this study proposes an automated visual inspection system to achieve defect inspection in hand tool assembly. This study samples the work-in-process from three assembly stations in the ratchet wrench assembly process; an investigation of 28 common assembly defect types is presented, covering the 4 kinds of assembly anomaly in the assembly operation; also, this study captures sample images of various assembly defects for the experiments. First, the captured images are filtered to eliminate surface reflection noise from the workpiece; then, a circular mask is given at the assembly position to extract the ROI area; next, the filtered ROI images are used to create a defect-type label set using manual annotation; after this, the R-CNN series network models are applied to object feature extraction and classification; finally, they are compared with other object detection models to identify which inspection model has the better performance. The experimental results show that, if each station uses the best model for defect inspection, it can effectively detect and classify defects. The average defect detection rate (1-β) of each station is 92.64%, the average misjudgment rate (α) is 6.68%, and the average correct classification rate (CR) is 88.03%.

A visual inspection and diagnosis system for bridge rivets based on a convolutional neural network

AbstractRivets are critical mechanical fasteners in steel bridges, and rivet defects may cause catastrophic failure. This study proposes a convolutional neural network (CNN)‐based inspection system for fast rivet identification and diagnosis. Rivet states are classified as normal, rusted, loose, and missing. A CNN‐based training workflow was introduced to develop a reliable rivet diagnosis system. A multiscale moving window searching technique was proposed to solve the challenge of small rivet identification. A continuous dataset enrichment strategy was applied, which improves training efficiency and minimizes training time. The model performance was assessed based on a historical bridge in Gjerstad. The proposed multiscale moving window searching technique significantly enhances the rivet identification rate to 96.3%. The classification accuracy and model robustness were evaluated, and conditions leading to unidentified rivets were discussed and summarized.

Training of a Neural Network System in the Task of Detecting Blue Stains in a Sawmill Wood Inspection System

This article presents the operation of an automatic pine sawn timber inspection system, which was developed at the Woodinspector company and is offered commercially. The vision inspection system is used to detect various wood defects, including knots, blue stain, and mechanical damage caused by worms. A blue stain is a defect that is difficult to detect based on the color of the wood, because it can be easily confused with wood defects or dirt that do not impair its strength properties. In particular, the issues of detecting blue stain in wood, the use of artificial neural networks, and improving the operation of the system in production conditions are discussed in this article. While training the network, 400 boards, 4 m long, and their cross-sections of 100 × 25 [mm] were used and photographed using special scanners with laser illuminators from four sides. The test stages were carried out during an 8-hour workday at a sawmill (8224 m of material was scanned) on material with an average of 10% blue stain (every 10th board has more than 30% of its length stained blue). The final learning error was assessed based on defective boards detected by humans after the automatic selection stage. The system error for 5387 boards, 550 m long, which had blue staining that was not detected by the scanner (clean) was 0.4% (25 pieces from 5387), and 0.1 % in the case of 3412 boards, 610 mm long, on which there were no blue stains, but were wrongly classified (blue stain). For 6491 finger-joint boards (180–400 mm), 48 pieces were classified as class 1 (clean), but had a blue stain (48/6491 = 0.7%), and 28 pieces did not have a blue stain, but were classified as class 2 (28/3561 = 0.7%).

Visual inspection system for crack defects in metal pipes

Visual inspection system for crack defects in metal pipes

Tool Damage Detection Method Based on Improved Threshold Segmentation Algorithm

In order to solve the problem that the current tool wear defects are difficult to be collected by the visual inspection system, an Otsu threshold segmentation algorithm based on particle swarm optimization is proposed.to detect tool wear The algorithm improved update strategy for inertia coefficients which effectively expanding the search scope of the algorithm and shortens the running time of the algorithm. By adding a perturbation equation to the particle swarm, solved the problem of traditional particle swarm optimization algorithms easily falling into local optima. Finally, an experimental platform is built to verify the effectiveness of the algorithm. This inspection method can achieve the identification of tool damage areas and the measurement of tool damage amount, and has advantages such as high recognition accuracy and fast running speed compared to traditional Otsu algorithm, Canny algorithm ,local threshold segmentation and so on. The research results have certain reference value for the actual tool defect detection system.

CAD-based Autonomous Vision Inspection Systems

Automated industrial Visual Inspection Systems (VIS) are typically customized for specific applications, limiting their flexibility. They are characterized by a demanding setup, high capital investments, and significant knowledge barriers. In this paper, we propose an alternative architecture for the visual inspection of 3D printed parts or complex assemblies using a robotic arm equipped with hand-eye sensors and controllable lighting system. The core of the proposed Flexible Vision Inspection System (FVIS) is the self-extraction of 3D text annotations from STandard for the Exchange of Product model (STEP) AP242 files. The system self-selects and parametrizes the most suitable inspection algorithm, including lighting settings. Additionally, it autonomously performs self-localization, self-referencing of physical products, and self-planning of robot inspection path based on CAD information. This framework, characterized by self-X, cost-effective, non-invasive, and plug-and-play architecture has the potential to disrupt the business model of vision inspection, enabling an as-a-service solution aligned with the next generation of flexible manufacturing.

Automated Machine Vision Inspection System for Applying Plastic Lids on Wet Wipes Packs

Computer vision techniques have been widely used in industrial manufacturing for automation, monitoring, quality assessment and inspection. Specifically, some techniques are being deployed to detect and identify products’ defects instead of human operators. In this paper, a vision system is proposed to inspect and assess the quality of applying plastic lids on wet wipes packs at a manufacturing company in Jordan. Currently, the manufacturing company uses automatic lid applicator that has no visual control system and thus, a human operator is needed to inspect every pack. This process includes detecting the position and orientation of each lid to assure that it has been applied correctly at the center of the pack above specific label. Automating this inspection process would save time and efforts and increase the productivity. The proposed inspection system has the following modules: (1) Pack and lid detection and segmentation using YOLOv8s-Seg algorithm; (2) distance to border (DtB) extraction between the pack centroid and lid boundary; and (3) inspection module using linear Support Vector Machine (SVM). A segmentation dataset of 319 different images of wet wipes packs was constructed using the Segment Anything Model (SAM). The proposed inspection has been tested on different wet wipes packs. Experimental results successfully demonstrated the efficiency of the inspection system.

Adaptive robotic system for the inspection of aerospace slat actuator mount.

Introduction: Robotics uptake in the aerospace industry is low, mainly due to the low-volume/high-accuracy production that aerospace manufacturers require. Furthermore, aerospace manufacturing and assembly sites are often unstructured environments not specifically suitable for robots to operate in. Methods: This paper introduces a robotic visual inspection system using off-the-shelf components able to inspect the mounting holes for wing slat actuators without the need for fixed-coordinate programming; the part just needs to be left within reach of the robot. Our system sets one of the opposed pairs of mounting holes as a reference (the "datum") and then compares the tilt of all other pairs of mounting holes with respect to it. Under the assumption that any deviation in the mounting hole tilt is not systematic but due to normal manufacturing tolerances, our system will either guarantee the correct alignment of all mounting holes or highlight the existence of misaligned holes. Results and Discussion: Computer-vision tilt measurements are performed with an error of below 0.03° using custom optimization for the sub-pixel determination of the center and radius of the mounting holes. The error introduced by the robot's motion from the datum to each of the remaining hole pairs is compensated by moving back to the datum and fixing the orientation again before moving to inspect the next hole pair. This error is estimated to be approximately 0.05°, taking the total tilt error estimation for any mounting hole pair to be 0.08° with respect to the datum. This is confirmed by manually measuring the tilt of the hole pairs using a clock gauge on a calibrated table (not used during normal operation).

Practical Applications of a Set-Based Camera Deployment Methodology.

This work establishes a complete methodology for solving continuous sets of camera deployment solutions for automated machine vision inspection systems in industrial manufacturing facilities. The methods presented herein generate constraints that realistically model cameras and their associated intrinsic parameters and use set-based solving methods to evaluate these constraints over a 3D mesh model of a real part. This results in a complete and certifiable set of all valid camera poses describing all possible inspection poses for a given camera/part pair, as well as how much of the part's surface is inspectable from any pose in the set. These methods are tested and validated experimentally using real cameras and precise 3D tracking equipment and are shown to accurately align with real imaging results according to the hardware they are modelling for a given inspection deployment. In addition, their ability to generate full inspection solution sets is demonstrated on several realistic geometries using realistic factory settings, and they are shown to generate tangible, deployable inspection solutions, which can be readily integrated into real factory settings.

Visual clustering network-based intelligent power lines inspection system

The huge differences in the visual shapes of multi-angle shooting objects leads to the poor performance of deep neural network (DNN). In this paper, an object detection model named TRUST-TECH-based visual clustering network (TTVCNet) for power line inspection is constructed. First, a TRUST-TECH-based visual clustering method (TTVCM) for multi-view-shape unsupervised clustering is proposed and can learn the difference in visual shape according to the object's views, which is the core of TTVCNet. Then, a Cascaded R–CNN object detection model based on TTVCNet is constructed for the power line inspection. Moreover, we apply the bilinear interpolation method and feature enhancement fusion techniques to this object detection model to solve the problem of small sample detection and semantic loss. In this paper, TTVCNet is applied to the MS-COCO 2017 dataset, and the test accuracy is improved up to 65.3%, especially the recognition accuracy of multi-view-shape is greatly improved. In the contrast experiment of self-made power line inspection dataset, the recognition accuracy of TTVCNet has been greatly improved, and the overall recognition accuracy is 86.3%.