Optical Inspection System Research Articles

Evaluating sub-pixel functional defects of a display using an arbitrary resolution camera

An automatic optical inspection system to evaluate functional defects of sub-pixel elements in image displays is proposed, which is based on a single full-field imaging sensor. In order to circumvent sampling aliasing, the display is addressed with a set of grid patterns whose defocused optical images imitate a lower resolution display. The cross-talk between the sensor and the display primary colors is corrected by means of a linear transformation between their color-spaces. Dark and bright dots are located and quantified by applying a notch multi-band Fourier filter to the sensor images. The technique is simple, robust, and easily scalable, since only one image sensor is required, no moving parts are used, and the sensor can be reused to analyze increasing resolution displays.

Physically based computer graphics for realistic image formation to simulate optical measurement systems

Abstract. Physically based image synthesis methods, a research direction in computer graphics (CG), are capable of simulating optical measuring systems in their entirety and thus constitute an interesting approach for the development, simulation, optimization, and validation of such systems. In addition, other CG methods, so-called procedural modeling techniques, can be used to quickly generate large sets of virtual samples and scenes thereof that comprise the same variety as physical testing objects and real scenes (e.g., if digitized sample data are not available or difficult to acquire). Appropriate image synthesis (rendering) techniques result in a realistic image formation for the virtual scenes, considering light sources, material, complex lens systems, and sensor properties, and can be used to evaluate and improve complex measuring systems and automated optical inspection (AOI) systems independent of a physical realization. In this paper, we provide an overview of suitable image synthesis methods and their characteristics, we discuss the challenges for the design and specification of a given measuring situation in order to allow for a reliable simulation and validation, and we describe an image generation pipeline suitable for the evaluation and optimization of measuring and AOI systems.

The Application of Manhole Optical Cable Inspection System

The manhole optical cable inspection system based on RFID is a kind of technology which processes data through RFID, GPS, GIS, database and network technique. It will make the daily maintenance more scientific, standard and automatic. This system can increase service restoration speed and save management costs which effectively improve the electronic level of optical cable resource management and bring convenience to track workers’ daily work. The application results provide effective mode for manhole optical cable management.

Simulation of microscopic metal surfaces based on measured microgeometry

Abstract Lack of training data is one of the main problems when realizing optical surface inspection systems. In the best case, provision of enough representative training samples is difficult and most of the time expensive. In some cases, it is not possible at all. Here we present an alternative method where the surface defects are simulated. Thereby, we focus on metal surfaces in the microscale where diffraction phenomena start to play a major role. Ray tracing and scalar diffraction approximation methods are applied and compared.

Stent optical inspection system calibration and performance.

Implantable medical devices, such as stents, have to be inspected 100% so no defective ones are implanted into a human body. In this paper, a novel optical stent inspection system is presented. By the combination of a high numerical aperture microscope, a triple illumination system, a rotational stage, and a CMOS camera, unrolled sections of the outer and inner surfaces of the stent are obtained with high resolution at high speed with a line-scan approach. In this paper, a comparison between the conventional microscope image formation and this new approach is shown. A calibration process and the investigation of the error sources that lead to inaccuracies of the critical dimension measurements are presented.

A Machine Vision Based Automatic Optical Inspection System for Measuring Drilling Quality of Printed Circuit Boards

In this paper, we develop and put into practice an automatic optical inspection (AOI) system based on machine vision to check the holes on a printed circuit board (PCB). We incorporate the hardware and software. For the hardware part, we combine a PC, the three-axis positioning system, a lighting device, and charge-coupled device cameras. For the software part, we utilize image registration, image segmentation, drill numbering, drill contrast, and defect displays to achieve this system. Results indicated that an accuracy of 5 $\mu \text{m}$ could be achieved in errors of the PCB holes allowing comparisons to be made. This is significant in inspecting the missing, the multi-hole, and the incorrect location of the holes. However, previous work only focuses on one or other feature of the holes. Our research is able to assess multiple features: missing holes, incorrectly located holes, and excessive holes. Equally, our results could be displayed as a bar chart and target plot. This has not been achieved before. These displays help users to analyze the causes of errors and immediately correct the problems. In addition, this AOI system is valuable for checking a large number of holes and finding out the defective ones on a PCB. Meanwhile, we apply a 0.1-mm image resolution, which is better than others used in industry. We set a detecting standard based on 2-mm diameter of circles to diagnose the quality of the holes within 10 s.

Automated optical inspection system for surface mount device light emitting diodes

Surface-mount device light emitting diode (SMD-LED) is characterized by small size, wide viewing angle and light weight. It becomes the main package type of LED gradually. The traditional visual inspection is likely to cause misrecognition due to personal subjectivity and different defect recognition standards. Therefore, this study develops an automatic SMD-LED defect detection system, which is characterized by non-contact inspection, defect recognition standardization and upgrading product quality. It detects the common and important defects in LED package components, including missing component, no chip, wire shift and foreign material. In this study the gray scale characteristic of histogram is used as the rapid sieving analysis indicator of missing component defect, and then the component and solder joint are positioned by using fast normalized cross-correlation, and the maximum correlation coefficient value is used as judgment indicator of no chip defect. In order to overcome the difficult identification as the weld line is subject to light rays, the improved Michelson-like contrast (MLC) enhancement is proposed, and the segmentation threshold is selected by entropy information to segment the weld line successfully. Furthermore, in order to overcome the effect of the tolerance of component size and internal electrode and unfixed weld line position resulted from lead frame process on foreign material detection result, the multiscale adaptive Fourier analysis (MAFA) is proposed in the concept of texture anomaly detection for foreign material defect detection. The result proves that the proposed method can segment the defect effectively and correctly compared with the phase-only transform (PHOT) and multiscale phase-only transform (MPHOT), and it can be used in other fields of texture anomaly detection. The overall recognition rate of this system is 98.25%, contributing to the large market demand and high component quality of LED industry.

A Design Method for LEDs Arrays Structure Illumination

Capturing high quality image is very important in the active automatic optical inspection system. Besides camera, image quality mainly is affected by the illumination. A design method of structure illumination, which is comprised of light emitting diodes (LEDs) arrays, is proposed in this paper. And especially, these LEDs arrays are not in the same plane. First, the single LED irradiating model is built based on analyzing its irradiance properties. Second, the case of two LEDs irradiating is set as an example to analyze superposition properties of the same color light for any specified point's light brightness based on the built single LED irradiating model, and then the irradiating model of LEDs is developed. Third, the structure illumination design method of LEDs ring arrays is illustrated based on the proposed design model of illumination uniformity. Especially, the illumination design method is decomposed into two subproblems to simplify the design process, which can reduce the design difficulty. Experimental results show that the light intensity deviation is less than 3.78% by the proposed illumination design method, which validates the effectiveness of the proposed method.

Multiscale measurement of air foils with data fusion of three optical inspection systems

Measuring the geometry of machine parts with complex geometries creates new challenges towards measurement systems. Due to cost, fast inspection is nevertheless desirable. Additionally the characterization of a parts health and functionality requires geometric information in different scales. We have developed a set of optical measurement systems which together meet those requirements: A borescopic fringe projection system, a macroscopic fringe projection system using newly developed algorithms for fast inspection of turbine blades and a low coherence Michelson interferometer (LCI). The first can detect geometric variances in hard to reach areas, e.g. inside machines or in between parts with highly complex geometries like blade integrated discs (blisks). Using fully adaptable fringe patters, the second system can locate geometric variances with a single fringe pattern. These patterns lead to high sensitivity and high measurement speed. Afterwards the LCI further inspects the micro structure of defects and characterizes the surface structure of the air foil. The presented algorithms provide fast 3D reconstruction with a nanometer resolution and, compared to other systems, a large measurement range. Together, these three inspection systems are capable to detect and quantify defects or geometric variances of industrial parts in different scales. This information improves the prediction of the reliability of a part and helps extending its lifetime to reduce the maintenance costs of machines. As an example a turbine blade was measured with all three systems and the results are visualized in one dataset. Means to merge the measurements are discussed.

Exploring combined dark and bright field illumination to improve the detection of defects on specular surfaces

An automatic optical inspection system for detecting local defects on specular surfaces is presented. The system uses an image display to produce a sequence of structured diffuse illumination patterns and a digital camera to acquire the corresponding sequence of images. An image enhancement algorithm, which measures the local intensity variations between bright- and dark-field illumination conditions, yields a final image in which the defects are revealed with a high contrast. Subsequently, an image segmentation algorithm, which compares statistically the enhanced image of the inspected surface with the corresponding image for a defect-free template, allows separating defects from non-defects with an adjusting decision threshold. The method can be applied to shiny surfaces of any material including metal, plastic and glass. The described method was tested on the plastic surface of a car dashboard system. We were able to detect not only scratches but also dust and fingerprints. In our experiment we observed a detection contrast increase from about 40%, when using an extended light source, to more than 90% when using a structured light source. The presented method is simple, robust and can be carried out with short cycle times, making it appropriate for applications in industrial environments.

Design and control of a cascaded piezoelectric actuated two-degrees-of-freedom positioning compliant stage

A novel piezoelectric driven compliant stage and its associated PC-based X–Y axes control schemes have been developed as a potential carrier of automatic optical inspection (AOI) systems and other possible applications in precision positioning. The design is based on compliant structure with a mechanical amplifier mechanism. Two orthogonal stages are integrated into a final 2-DoF design. This approach offers the advantages of directly using a well-development design with minor modifications and the performance can be more precisely controlled. However, this also brings concerns in coupling between two motion axes due to manufacturing and alignment errors and this issue is examined experimentally. By integrating feedback control with the stage, it is possible to perform precision positioning and vibration suppression for improving the dynamic performance. This cascaded structure design can effectively reduce the system complexity and can be further extended for additional degrees of freedom. Based on the test results, the designed stage can achieve a closed loop bandwidth up to 100Hz and a steady state resolution less than 50nm using a model reference sliding mode controller. In addition, a shaping-control integration approach is also demonstrated for providing faster positioning while maintaining the robustness against possible external disturbances. Meanwhile, the experimental results indicate that the coupling due to manufacturing and assembly errors exists but this can be effectively reduced by proper trajectory planning with incorporating of dual-axis control. In summary, this study has realized a structurally-simple and low-cost positioning system and is able to achieve high-precise motion. It is hope that this study can be further expanded to longer stroke and higher precise positioning system, and integrates with cutting edge technologies for developing more superior precise instrument in the future.

An Intelligent Image Processing System for Real-Time Detection of Surface Flaws

This paper presents an intelligent parallel image processing system, PVCS (Parallel Visual Computing System), to handle in-line surface defect detection for large objects. PVCS is a parallel expansion of a previously-reported non-destructive Compute Unified Device Architecture (CUDA)-enabled optical inspection system to accommodate large test objects. PVCS is heterogeneous both in terms of hardware and software. From the hardware perspective, PVCS consists of multiple CPUs and GPUs; from the software perspective, PVCS adopts the Message Passing Interface (MPI) and CUDA programming models. A parallel prototype system, consisting of three CPUs and two GPUs, is used to inspect a simulated object with an area eight times greater than that of the real object in our previous work. Given the same resolution requirements, the simulation results show that PVCS can obtain the correct number of defects within a large-size image at a satisfactory processing rate.

Design and Control of an Optical Alignment System Using a XXY Stage Integrated with dual CCDs

The aim of this study is to investigate assembly and positioning tasks of a micro-component using a co-plane XXY stage integrated with an automatic optical inspection (AOI) system instead of manual alignment. An artificial neural algorithm is added for alignment to enhance the alignment accuracy of the system. Therefore, a servo alignment system is proposed to use the AOI system to guide the XXY stage using vision technology. The image identification system is used to make the specimen align the target specimen using dual CCDs. Since the XXY stage has no positioning feedback of the optical scale or encoder, the artificial neural network (ANN) system is used to map the relative relationship between the image position and the actual XXY stage's position.

An Optical Inspection System Development for Defects on Multi-surfaces of Chips

We proposed a new architecture with multiple pick and place heads to suck inspected chips and to be moved synchronously for acquiring the images of different surfaces on different chips at their corresponding acquisition stations. The computation loading of defect inspection for each chip also can be distributed because of sequential image acquisition. Considering the different computing loading of 6 surfaces of a chip, image acquisition for the front and back sides of chips is allocated at the first and second stations, and then 4 edge sides of chips. The modification and calibration of position and orientation for each opto-mechanical device will be simpler and easier when the characteristics of inspected chips changed. Thus system can accomplish the inspection for 6 surfaces of each chip and determine its quality in real-time before that chip is put onto target wafer. Developed inspection system has advantages and contributions of real-time inspection with high throughput during chip sorting process, and easily adjustment and calibration for all opto-mechanical devices to inspectdifferent types of chips. The first phase of development was accomplished for an AOI system to inspect defects on 5 surfaces of chips. System has been integrated in customer’s chip sorter for testing. The inspecting accuracy for chip backside is better than 95%.

A New IC Solder Joint Inspection Method for an Automatic Optical Inspection System Based on an Improved Visual Background Extraction Algorithm

In the field of automatic optical inspection (AOI), defect recognition for an integrated circuit (IC) solder joint is a long-standing task. Inspired by a visual background extraction (ViBe) algorithm, an object detection method in computer vision, we propose a new inspection method for IC solder joints with an improved ViBe algorithm. To the best of our knowledge, we are the first to consider the defect inspection problem as an object detection problem. We build a solder joint model using the ViBe model updating scheme. Then, we compare the solder joint image with the well-trained model to detect potential defects. Finally, we introduce a frequency map method and define a metric named defect degree to evaluate the qualities of the solder joints. Experimental results show that our method is universal, accurate, and easily debugged compared with the other existing methods.

Eddy Current Volume Heating Thermography and Phase Analysis for Imaging Characterization of Interface Delamination in CFRP

Imaging inspection is highly demanded in the optimization of industry processes. Optical imaging inspection is not applicable for inside defects, while infrared (IR) imaging inspection can provide information about internal structure of objects. Eddy current thermography is an emerging IR imaging inspection technique for conductive materials or objects. This paper presents eddy current volume heating thermography (ECVHT) and phase analysis for delamination inspection in carbon fiber reinforced plastics (CFRPs) based on the previously proposed eddy current pulsed phase thermography (ECPPT). The proposed method has been verified through experimental studies under both transmission and reflection modes. After discrete Fourier transform (DFT) of temperature responses, the phasegram and phase spectra can be used to image and characterize interface delamination in CFRP due to elimination of nonuniform heating effect and carbon fiber structures. With the whole temperature response processed by DFT, carbon fiber structures and delamination can be differentiated due to periodic oscillation of phase spectra. With temperature response in cooling phase processed by DFT, some characteristic features can be extracted to construct the new phase images according to the shape of phase spectra. In all, using ECVHT and phase analysis, imaging characterization for delamination can get much better performance than conventional visible optical inspection system and eddy current pulsed thermography.

Model-based defect detection on structured surfaces having optically unresolved features.

In this paper, we demonstrate, both numerically and experimentally, a method for the detection of defects on structured surfaces having optically unresolved features. The method makes use of synthetic reference data generated by an observational model that is able to simulate the response of the selected optical inspection system to the ideal structure, thereby providing an ideal measure of deviation from nominal geometry. The method addresses the high dynamic range challenge faced in highly parallel manufacturing by enabling the use of low resolution, wide field of view optical systems for defect detection on surfaces containing small features over large regions.

Simulation Model of Surface Heterogeneity, Using Gielis' Superformula for Adaptive Methods of Image Analysis

The article presents a developed orginal model designed to increase the efficiency of the processes of the design of adaptive image analysis methods used for detection of surface defects in materials. The model was created as part of activities related to the Strategic Program – Specialized knowledge bases and expert systems for the simulation of complex processes. The methods presented in this paper make it possible to accelerate design process, including testing of algorithms developed for a wide range of surface defects, such as: cracks, discoloration, loss of materials, geometric distortion, or even the presence of defect agglomerations caused by corrosion, without the need for the acquisition of the physical image of the actual objects. This in many cases can be a significant problem for engineers who design automatic optical inspection systems, because the acquisition of test objects with specified defects which are characterized by a fixed range of values of selected parameters is not always possible. This paper presents a formal model designed to generate material defects on the surface of three-dimensional virtual objects, which is equivalent to the acquisition of actual data from vision systems. The model takes into account various surface characteristics such as their texture or roughness by using mapping by the Blinn method. The results of the use of the system developed for the classification of products represented in digital images for which image analysis algorithms have been based on so-called artificial intelligence in the form of dedicated neural networks are presented. As described in this paper, artificial neural networks are an example of adaptive models, and provide the ability to solve problems for which there are no deterministic models. The models, however, require the use of learning processes (training) with the use of extensive data sets, which in this case were generated with the use of the developed solution model.

Automatic Color Inspection for Colored Wires in Electric Cables

In this paper, an automatic optical inspection system for checking the sequence of colored wires in electric cable is presented. The system is able to inspect cables with flat connectors differing in the type and number of wires. This variability is managed in an automatic way by means of a self-learning subsystem and does not require manual input from the operator or loading new data to the machine. The system is coupled to a connector crimping machine and once the model of a correct cable is learned, it can automatically inspect each cable assembled by the machine. The main contributions of this paper are: (i) the self-learning system; (ii) a robust segmentation algorithm for extracting wires from images even if they are strongly bent and partially overlapped; and (iii) a color recognition algorithm able to cope with highlights and different finishing of the wire insulation. We report the system evaluation over a period of several months during the actual production of large batches of different cables; tests demonstrated a high level of accuracy and the absence of false negatives, which is a key point in order to guarantee defect-free productions.

Quick and efficient light control for conventional automatic optical inspection (AOI) systems

This study discusses a method to determine optimal illumination quickly in order to increase the process efficiency of automatic optical inspection (AOI) systems. Optimum searching methods were applied to find optimal illumination of a single color to enhance the quality of a monochrome image and reduce searching time. The problem was defined as optimizing 1D searching between light intensity and image sharpness. The image sharpness is the degree of distinctness of the image and is a non-linear function of the input voltage, which can be used to adjust the light intensity. We considered conventional optimum search methods, such as steepest descent, conjugate gradient, Newton’s method, bisection, and golden section. These derivative methods and direct searches were tested for a sample pattern using single color lights under coaxial illumination. The iteration for the optimal condition was 6.7% of full scanning in average. The optimal illumination was obtained using low input voltages, implying that a fine image could be obtained by applying appropriate electric energy. Additionally, we checked the possibility of applying these search methods to automatic lighting in industrial machine vision.