Optical Inspection Research Articles

Multidimensional computed measurement for highly accurate PCBA defect detection

Accurate defect detection in industrial automated optical inspection (AOI) is crucial as it directly affects product quality and production efficiency. Although numerous techniques have been developed for industrial defect detection, most of them rely on single-texture image data. This dependence limits the accuracy and robustness of the defect detection due to inadequate optical source information. To overcome the problem of low accuracy owing to the lack of 3D topographic information, a multidimensional information fusion (MIF) module is proposed that fuses texture image and depth image features. The MIF module includes tailored mechanisms to fully extract complementary semantic information from space and channel dimensions. A hierarchical fusion strategy further improves feature integration by enabling higher-layer feature fusion via lower-layer Transformer blocks and efficiently removing redundant features. Afterward, feature extraction is performed on the fused feature map, and output is obtained. To enhance the detection accuracy, the position information mask (PIM) module is introduced for post-processing. The PIM module uses surface mount devices (SMDs) position data from Gerber files to create a position information mask. The mask helps filter out defects that are often misidentified owing to missing design information. The results of the comparative experiments demonstrate that the average accuracy of our method on the printed circuit board assembly (PCBA) defect dataset is 99.93%, which is 5.64% higher than that of conventional YOLOV5. Furthermore, a comprehensive ablation study is conducted to elucidate the contribution of the proposed MIF and PIM modules. It demonstrated that the present model serves as a valuable reference for PCBA surface defect detection.

Low-cost integrated circuit packaging defect classification system using edge impulse and ESP32CAM

Defects in integrated circuit (IC) packaging are inevitable. Several factors can cause defects in IC packaging such as material quality, errors in machine and human handling operations, and non-optimized processes. An automated optical inspection (AOI) is a typical method to find defects in the IC manufacturing field. Nevertheless, AOI requires human assistance in the event of uncertain defect classification. Human inspection often misses very tiny defects and is inconsistent throughout the inspection. Therefore, this study proposed a low-cost IC packaging defect classification system using edge impulse and ESP32-CAM. The method involves training a deep learning model (i.e., convolutional neural network (CNN)) using a dataset of non-defective and defective ICs on Edge Impulse. For defective ICs, the top surface of the ICs is deliberately scratched to imitate the cosmetic defects. ICs with scratch-free on their top surfaces are considered non-defective ICs. A successfully trained model using Edge Impulse is subsequently deployed on ESP32-CAM. The model is optimized to fit the limited resources of the ESP32-CAM. By using the built-in camera in ESP32-CAM, the trained model can perform a real-time image classification of non-defective/defective ICs. The proposed system achieves 86.1% prediction accuracy by using a 1,571 image dataset of defective and non-defective ICs.

Nanophotonic inspection of deep-subwavelength integrated optoelectronic chips.

Artificial nanostructures with ultrafine and deep-subwavelength features have emerged as a paradigm-shifting platform to advanced light-field management, becoming key building blocks for high-performance integrated optoelectronics and flat optics. However, direct optical inspection of integrated chips remains a missing metrology gap that hinders quick feedback between design and fabrications. Here, we demonstrate that photothermal nonlinear scattering microscopy can be used for direct imaging and resolving of integrated optoelectronic chips beyond the diffraction limit. We reveal that the inherent coupling among deep-subwavelength nanostructures supporting leaky resonances allows for the pronounced heating effect to access reversible nonlinear modulations of the confocal reflection intensity, yielding optical resolving power down to 80 nm (~λ/7). The versatility of this approach has been exemplified by imaging silicon grating couplers and metalens with minimum critical dimensions of 100 nm, as well as central processing unit chip with 45-nm technology, unfolding the long-sought possibility of in situ, nondestructive, high-throughput optical inspection of integrated optoelectronic and nanophotonic chips.

Siamese-RCNet: Defect Detection Model for Complex Textured Surfaces with Few Annotations

The surface texture of objects in industrial scenes is complex and diverse, and the characteristics of surface defects are often very similar to the surrounding environment and texture background, so it is difficult to accurately detect the defect area. However, when deep learning technology is used to detect complex texture surface defects, the detection accuracy is not high, due to the lack of large-scale pixel-level label datasets. Therefore, a defect detection model Siamese-RCNet for complex texture surface with a small number of annotations is proposed. The Cascade R-CNN target detection network is used as the basic framework, making full use of unlabeled image feature information, and fusing the nonlinear relationship learning ability of Siamese network and the feature extraction ability of the Res2Net backbone network to more effectively capture the subtle features of complex texture surface defects. The image difference measurement method is used to calculate the similarity between different images, and the attention module is constructed to weight the feature map of the feature extraction pyramid, so that the model can focus more on the defect area and suppress the influence of complex background texture area, so as to improve the accuracy of detection. To verify the effectiveness of the Siamese-RCNet model, a series of experiments were carried out on the DAGM2007 dataset of weakly supervised learning texture surface defects for industrial optical inspection. The results show that even if only 20% of the labeled datasets are used, the mAP@0.5 of the Siamese-RCNet model can still reach 96.9%. Compared with the traditional Cascade R-CNN and Faster R-CNN target detection networks, the Siamese-RCNet model has high accuracy, can reduce the workload of manual labeling, and provides strong support for practical applications.

Автоматизация визуального контроля фотошаблонов для изделий микроэлектроники

The choice of a specific research topic was caused by a request for the development and implementation of automatic optical inspection at a semiconductor plant. The purpose of the work was to develop requirements, conduct technical design and create an optical control system for geometry deviations from a photomask drawing and a conductive pattern obtained using a manual stencil printer using relatively inexpensive equipment. As a result of the research, a pilot design sample of an automated optical control system using relatively inexpensive equipment was created and a computational algorithm was developed that ensures increased performance of the visual image recognition system, which allows to significantly reduce the percentage of defects. The work describes the implemented algorithm for obtaining images by an optical system and extracting images from a drawing file. Regardless of the method of obtaining an image (optical system, scanning electron microscope), there remains interest in choosing a criterion for comparing the obtained image with the reference one. The paper studies quantitative empirical metrics - Mean Square Error and Peak Signal-to-Noise Ratio for various noise reduction methods Block-Matching and 3D filtering and the classical spatial filtering method of Gaussian blur. The sensitivity of the structural similarity index metric to structural distortions of images after noise reduction is checked, taking into account the minimum values of the structural elements of metal-ceramic housings. Based on the Rosner test applied to the obtained values of the structural similarity metric, images containing defects are identified. The user interface provides for the output of the image area with a defect to the operator's screen.

PCB Defect Classification with Data Augmentation-Based Ensemble Method for Sustainable Smart Manufacturing

In the rapidly evolving field of printed circuit board (PCB) manufacturing, automated optical inspection (AOI) systems play a critical role but often face challenges such as computational inefficiencies, high costs, and limited defect data. To address these issues, we propose an ensemble methodology that combines lightweight models with custom data augmentation techniques to enhance defect classification accuracy in real-time production environments. Our approach mitigates overfitting in small datasets by generating diverse models through advanced data augmentation and employing feature-specific validation strategies. These models are integrated into an ensemble framework, achieving complementary results that improve classification accuracy while reducing computational overhead. We validate the proposed method using two datasets: the general classification dataset CIFAR-10 and an on-site real-world PCB dataset. With our approach, the average accuracy on CIFAR-10 improved from 97.6% to 98.2%, and the accuracy on the PCB dataset increased from 81% to 89%. These results demonstrate the method’s effectiveness in addressing data scarcity and computational challenges in real-world manufacturing scenarios. By improving quality control and reducing waste, our method optimizes production processes and contributes to sustainability through cost savings and environmental benefits. The proposed methodology is versatile, scalable, and applicable to a range of defect classification tasks beyond PCB manufacturing, making it a robust solution for modern production systems.

Broadband S-Parameter-Based Characterization of Multilayer Ceramic Capacitors Submitted to Mechanical Stress Through Bending Tests on a PCB.

A full characterization of multilayer ceramic capacitors including variations in capacitance, series resistance, and series inductance is accomplished by measuring their RF response while being submitted to mechanical stress. This allows for the first time quantifying the degradation of the device's RF performance when cracks form within its structure. In this regard, the main challenge is designing an interface for measuring the high-frequency response of a capacitor using a vector network analyzer as a bending test on a PCB in progress, which is achieved here by using a microstrip-based test fixture. The results indicate that there is an overestimation of its response to microwave stimuli when considering only the degradation impact as a reduction in capacitance. Capacitors of representative sizes and capacitances are analyzed to show the usefulness of the proposal, whereas the validity of the results is verified by observing the correlation with measurements collected using microprobes and performing optical inspections of cross-sectioned samples.

Detection of matrix cracks in composite laminates using embedded fiber-optic distributed strain sensing

We used an optical frequency-domain reflectometry fiber Bragg grating (FBG) distributed strain measurement system to detect matrix cracks in carbon fiber-reinforced plastics. Load/unload tests were performed on cross-ply laminates under ambient-, high-, and low-temperature conditions using embedded 100 mm gauge FBG sensors. We measured the strain distributed along the gauge and examined the crack initiation using optical microscopy and soft X-ray inspection. A peak appeared in the distributed strain corresponding to the occurrence of cracks. The strain distribution during crack initiation was calculated in each temperature range via finite element analysis and compared with the measurement results. We determined that the number and location of cracks can be determined by extracting the peaks from the distributed strain. However, the peaks in the distributed strain do not correspond to the occurrence of cracks that are close to each other, warranting a measurement method with a higher strain resolution.

Quasi-visualizable detection of deep sub-wavelength defects in patterned wafers by breaking the optical form birefringence

Abstract In integrated circuit (IC) manufacturing, fast, nondestructive, and precise detection of defects in patterned wafers, realized by bright-field microscopy, is one of the critical factors for ensuring the final performance and yields of chips. With the critical dimensions of IC nanostructures continuing to shrink, directly imaging or classifying deep-subwavelength defects by bright-field microscopy is challenging due to the well-known diffraction barrier, the weak scattering effect, and the faint correlation between the scattering cross-section and the defect morphology. Herein, we propose an optical far-field inspection method based on the form-birefringence scattering imaging of the defective nanostructure, which can identify and classify various defects without requiring optical super-resolution. The technique is built upon the principle of breaking the optical form birefringence of the original periodic nanostructures by the defect perturbation under the anisotropic illumination modes, such as the orthogonally polarized plane waves, then combined with the high-order difference of far-field images. We validated the feasibility and effectiveness of the proposed method in detecting deep subwavelength defects through rigid vector imaging modeling and optical detection experiments of various defective nanostructures based on polarization microscopy. On this basis, an intelligent classification algorithm for typical patterned defects based on a dual-channel AlexNet neural network has been proposed, stabilizing the classification accuracy of λ/16-sized defects with highly similar features at more than 90%. The strong classification capability of the two-channel network on typical patterned defects can be attributed to the high-order difference image and its transverse gradient being used as the network’s input, which highlights the polarization modulation difference between different patterned defects more significantly than conventional bright-field microscopy results. This work will provide a new but easy-to-operate method for detecting and classifying deep-subwavelength defects in patterned wafers or photomasks, which thus endows current online inspection equipment with more missions in advanced IC manufacturing.

Split Hopkinson bar and flash X-ray characterization of ultra high performance concrete reinforced by steel fibers subjected to dynamic compression

AbstractExcellent mechanical properties of ultra high performance concrete make it suitable for use in special applications, where the material is subjected to dynamic phenomena such as impacts, explosions, or earthquakes. This paper presents a novel experimental approach that integrates a Split Hopkinson Pressure Bar with a flash X-ray system and high-speed optical imaging to investigate the dynamic behavior of steel fiber reinforced UHPC under high strain rate uni-axial compression. In-situ Flash X-ray radiography emerges as a particularly effective tool, providing clear visualization of deformation response and overcoming challenges associated with flying debris encountered in optical inspection. Moreover, computed tomography and scanning electron microscopy appear as a vital technique for analyzing micro-structure and fiber distribution and orientation. The combined approach offers a promising method to study the dynamic behavior of steel fiber reinforced ultra high performance concrete and also holds promise for analyzing more complex modes of deformation and material interactions, providing valuable insights for enhancing the design and performance of critical infrastructure subjected to dynamic loading events.

3D micromorphological reconstruction and roughness characterization of wood surface based on sequence images

A novel non-contact optical inspection technique is proposed to precisely reconstruct the three-dimensional surface structure of wood and evaluate its roughness. The feasibility of using image fusion techniques to characterize the 3D roughness of wood surfaces is explored by combining high-resolution serial image acquisition techniques with advanced data processing methods. The work focuses on examining various cut surfaces of three distinct wood species: Larch, Catalpa, and Toona sinensis. The proposed 3D roughness parameters are verified using 3D reconstruction and precise measurements. The experimental results demonstrate that the proposed method exhibits strong adaptability in measuring wood surface roughness and achieves a high degree of consistency with the results obtained from traditional digital microscope systems. This study not only validates the possible use of multilayer sequence image fusion technology in measuring wood roughness, but also establishes a solid basis for the creation of 3D assessment criteria for wood surface roughness and associated inspection systems.

AI-Driven Quality Control in PCB Manufacturing: Enhancing Production Efficiency and Precision

This paper investigates the application of Artificial Intelligence (AI) in enhancing quality control within Printed Circuit Board (PCB) manufacturing processes, focusing on how AI-driven technologies improve production efficiency and precision. The research addresses traditional quality control methods, such as manual inspection and Automated Optical Inspection (AOI), highlighting their limitations in keeping up with the complexities and demand for high-precision PCBs in modern electronics. The study delves into how AI technologies, particularly machine learning, computer vision, and predictive analytics, are being leveraged to overcome these limitations. By automating defect detection, improving accuracy, and enabling real-time analysis, AI systems not only streamline the quality control process but also significantly reduce human error and production costs. AI-driven quality control systems are shown to increase defect detection rates, reduce inspection times, and enhance overall production throughput. The paper includes a comparative analysis between traditional and AI-based quality control methods, revealing a notable improvement in both detection accuracy and production speed when AI systems are employed. Additionally, cost efficiency is explored, demonstrating how AI systems reduce waste, minimize rework, and lower operational costs. The potential challenges of AI implementation, such as the high initial costs, data requirements, and integration with legacy systems, are also discussed. The paper concludes that despite these challenges, AI offers a transformative solution to the growing demands of the PCB manufacturing industry, with the ability to scale effectively and adapt to future technological advancements. Ultimately, this research underscores the significant role of AI in revolutionizing PCB quality control by providing a more efficient, precise, and cost-effective approach, paving the way for further innovation in the electronics manufacturing sector.

Optical Design Study with Uniform Field of View Regardless of Sensor Size for Terahertz System Applications

The focal length in a typical optical system changes with the angle of view, according to the size of the sensor. This study proposed an optical terahertz (THz) system application where the focal length changed while the angle of view was fixed; thus, the image height was variable and responded to various sensor sizes. Therefore, it is possible to respond to various sensors with one optical system when the inspection distance is fixed. The fundamental optical system was designed by arranging the refractive power, which was determined according to the sensor size using the Gaussian bracketing method. A zoom optical system that changed the image height by fixing the angle of view and changed the focal length by moving the internal lens group was designed. THz waves exhibit minimal change in the refractive index depending on the wavelength. Moreover, their long-wavelength characteristics facilitate the development of millimeter-level pixel sizes. Therefore, the root mean square size of the maximum spot was 0.329 mm, which corrected the aberration to less than 1 mm (smaller than the pixel size). Further, a lighting analysis at 3 and 6 m locations confirmed the expansion of the lighting area by the magnification of the sensor size. After turning off certain light sources, we checked the contrast ratio via lighting analysis and confirmed that the size of one pixel was clearly distinguishable. Consequently, this newly designed optical system performed appropriately as an optical inspection system for THz system applications.

Extraction of isotropic and anisotropic components for optical surface micro-metrology based on the two-dimensional power spectral density analysis

Extraction of surface characteristics is essential during surface processing and optical inspections. In this work, we propose a new extraction method by dividing the global feature within multiple directions into isotropic and anisotropic components and combining noise filtration, two-dimensional component extraction, and fast reconstruction. The mathematical descriptions of global and local features were derived. The noise by holes, scratches, and finite sampling points was restrained by Bearing Ratio analysis, Hough transform, and Welch window operation. The isotropic and anisotropic surface components were extracted in the two-dimensional power spectral density domain, reconstructed in the two-dimensional Fourier domain, and inversed in Cartesian coordinates. Five surfaces with anisotropic structural characteristics ranging from zero to two dimensions were analyzed. The general applicability was proved according to the consistency between surface processing methods and extracted results. A chemical mechanical polishing experiment was designed and accomplished to verify the sensitivity of the extraction method. The subtle variation in surface morphology was captured on the reconstructed surfaces near the polishing end-point, confirming its detectability on weak anisotropic components. This process-oriented surface extraction method can achieve qualified results without transcendental knowledge of surface conditions and offers openness to various surface evaluation criteria by statistical roughness indicators, which supports surface inspection for multiple surface processing techniques.

Optical inspection of stator slots for electric motors

An optical non-contact inspection system was developed for measuring the slots in stator lamination stacks. To avoid passing go/no-go gage blocks through the slots, a machine vision system is instead used to measure the stator core slots and identify the presence of burrs within the slots. Utilizing telecentric optics along with an alignment monitoring system configured to monitor and orient the stator core, the core slots can be oriented relative to the imaging axis for further metrology measurements. Among these measurements, the smallest opening dimensions (slot width and depth) of each slot due to misalignment of laminations and the detection of burrs along the edges of the slots throughout the length of the lamination stack are critical for full stator assembly. Advanced image processing algorithms were developed to obtain sub-pixel accuracy which is required to measure the slots. This, used in conjunction with a robust vision calibration technique, increases the feasibility of building a device that can be implemented as a production inspection system. Experiments show the reliability of the computer vision approach and how it can be used in the inspection of slots in lamination stacks.

Faster R-CNN-CA and thermophysical properties of materials: An ancient marquetry inspection based on infrared and terahertz techniques

The demand for non-invasive inspection (NII) is ever-increasing in the field of cultural heritage conservation. NII is a two-step procedure, first of data acquisition and second of defect detection. Stand-alone imaging techniques such as infrared thermography (IRT) are often insufficient for performing a complete remote analysis and diagnosis of historic structures and art pieces that are of very high cultural value. On this point, an emerging optical inspection method, terahertz time-domain spectroscopy (THz-TDS), is herein employed to provide more details of deeper defects. The imaging results from THz-TDS and IRT are compared and analyzed by employing advanced image processing methods. Next, to achieve automatic inspection of the test sample, which is an ancient marquetry, a Faster R-CNN with coordinate attention (Faster R-CNN-CA) is proposed and fitted with data from two different sources. Worth noting is that, in order to populate sufficient data for training, samples are simulated using finite element analysis and finite difference time domain method. The experiments demonstrate that the mean average precision of the Faster R-CNN-CA model improves by 6.09% over the traditional Faster R-CNN model.

Low velocity impact resistance of hybrid CFRP‐elastomer‐metal laminates: Influence of stacking sequence and impact conditions on damage mechanisms

AbstractFiber‐metal laminates (FMLs) are generally regarded as excellent lightweight materials for advanced structure design. To enhance the mechanical properties, the common FMLs can be optimized using carbon fibers. However, the combination of carbon fibers with aluminum induces interfacial challenges. Preventing galvanic corrosion with elastomeric interlayers is an effective solution. The lay‐up configuration greatly effects the impact damage resistance of hybrid CFRP‐elastomer‐metal laminates (HyCEMLs). In this work, micro‐CT scans and optical micrographic inspections on HyCEMLs are conducted after impact tests to ascertain the microstructural origins behind the mechanical performance changes. In addition, finite element models of different HyCEML configurations are built to complement the limited experimental data. The damage mechanisms of HyCEML with different configurations under various impact conditions are further compared. The numerical results suggest that impact energy is a more informative measure in terms of damage mode and size than impact velocity and momentum. Results also indicate that when the thickness for each sub‐laminate of HyCEML is maintained the same, hybrid laminates with aluminum stacked outside is beneficial for delaying the occurrence of matrix cracking and delaminations, and enhances HyCEML's resistance to global deformation. These findings will contribute to engineering hybrid laminates with desired impact performance for lightweight load‐bearing structures.Highlights The hybrid laminate with elastomeric interlayers is a forward‐looking solution in impact applications. Impact energy is a more informative measure in terms of assessing the damage mode and extent in HyCEMLs. The influence of stacking sequence on damage mechanisms of HyCEMLs is evaluated. Microstructural origins behind variations of hybrid laminates in the impact resistance are revealed.

Replicating biological 3D root and hyphal networks in transparent glass chips

Replicating the complex 3D microvascular architectures found in biological systems is a critical challenge in tissue engineering and other fields requiring efficient mass transport. Conventional microfabrication techniques often face limitations in creating extensive hierarchical networks, especially within bulk materials. Here, we report a versatile bioinspired approach to generate optimized 3D microvascular networks within transparent glass matrix by transcribing the natural growth patterns of plants and fungi. Plant seeds or fungal spores are first cultivated on nanoparticle-based culture media. Subsequent heat treatment removes the biological species while sintering the surrounding compound into a solidified chip with replica root/hyphal architectures as open microchannels. A diverse range of architectures, including the hierarchical branching of plant roots and the intricate networks formed by fungal hyphae, can be faithfully replicated. The resultant glass microvascular networks exhibit high chemical and thermal stability, enabling applications under harsh conditions. Fluid flow experiments validate the functionalities of the fabricated channels. By co-cultivating plants and fungi, hierarchical multi-scale architectures mimicking natural vascular systems are achieved. This bioinspired manufacturing technique leverages autonomous biological growth for architectural optimization, offering a complementary approach to existing microfabrication methods. The transparent nature of the glass chips allows for direct optical inspection, potentially facilitating integration with imaging components. This versatile platform holds promise for various engineering applications, such as microreactors, heat exchangers, and advanced filtration systems.

Optimizing Automated Optical Inspection: An Adaptive Fusion and Semi-Supervised Self-Learning Approach for Elevated Accuracy and Efficiency in Scenarios with Scarce Labeled Data.

In the field of automatic optical inspection (AOI), this study presents innovative strategies to enhance object detection accuracy while minimizing dependence on large annotated datasets. We initially developed a defect detection model using a dataset of 3579 images across 32 categories, created in collaboration with a major Taiwanese panel manufacturer. This model was evaluated using 12,000 ambiguously labeled images, with improvements achieved through data augmentation and annotation refinement. To address the challenges of limited labeled data, we proposed the Adaptive Fused Semi-Supervised Self-Learning (AFSL) method. This approach, designed for anchor-based object detection models, leverages a small set of labeled data alongside a larger pool of unlabeled data to enable continuous model optimization. Key components of AFSL include the Bounding Box Assigner, Adaptive Training Scheduler, and Data Allocator, which together facilitate dynamic threshold adjustments and balanced training, significantly enhancing the model's performance on AOI datasets. The AFSL method improved the mean average precision (mAP) from 43.5% to 57.1% on the COCO dataset and by 2.6% on the AOI dataset, demonstrating its effectiveness in achieving high levels of precision and efficiency in AOI with minimal labeled data.

Application of optical homogeneity detection technology in glass quality control based on visual inspection technology

Abstract Glass is an advanced material with a wide range of applications. Improving the quality of glass products supports and promotes various industries. This paper discusses the streaks affecting the quality of glass products and focuses on implementing optical homogeneity detection technology for glass. Based on the relationship between optical homogeneity and mechanical properties, this detection technology analyzes the causes of streaks in the production process of glass products and deduces the causes for the streak characteristics through the discussion of the working principle of the homogeneity detection technology, the detection device, the detection steps and the use of technology. In this study, visual inspection technology is introduced into the optical homogeneity inspection technology instead of the traditional manual inspection method, which can make the inspection results faster and more accurate and can find a quick and effective solution to the streaks.