Visual Inspection Tasks Research Articles

Localisation of a mobile robot for bridge bearing inspection

Abstract Bridge bearings are a critical component of a bridge and require regular visual inspection to ensure the safe operation of the bridge throughout its life. However, the bearings are often located in spaces that are difficult or hazardous to reach, which can impact how often the bearings are inspected. In addition, these spaces are small and offer significant challenges for tele-operation due to line-of-sight restrictions; hence, some level of autonomy is required to make robotic inspection possible. In this work, a robotic solution to bridge bearing inspection is presented, and localisation methods are assessed as the first, and most, important step towards automation. Robot localisation is performed in both a lab environment and a real bridge bearing environment. In this paper, Adaptive Monte-Carlo Localisation is considered for localisation in a known map and gave comparable results to Hector-SLAM, with all results less than a defined error threshold of 10 cm. A combination of both of these methods are proposed to give a more robust approach that gives errors lower than the defined threshold in the real bridge. The experiments also show the need to provide an accurate starting point for each inspection within the bearing, for which we notionally suggest the use of a docking station that could also be used for improved autonomy, such as charging. In addition, proof-of-concept approaches for visual inspection tasks, such as geometry changes and foreign object detection are presented to show some of the proposed benefits of the system presented in this work.

An ergonomics prototype of adjustable chin stands aid for visual mechanical inspection at electronic manufacturing-based company in Kuantan, Malaysia

Chin stands aid is a device designed to reduce fatigue on the chin during the Visual Mechanical Inspection (VMI) task for operators in TT Electronic Sdn Bhd, Kuantan, Malaysia. It is also used to reduce cycle time and also improve employee well-being in terms of comfort. In this project, a 3D model of chin stands aid with an ergonomics approach is created using SOLIDWORKS software. Two different concepts were designed and the best one is chosen based on the Pugh concept selection method, concept screening and also concept scoring. After the selection of concepts is done, a prototype of chin stands aid will be developed and a simulation of the prototype is performed. The simulation has been executed by using Workbench ANSYS software as a tool. Stress analysis, deformation analysis, and fatigue analysis have been done to know the strength and lifespan of the product. The prototype also has been tested to know the functionality and also comfortability for the user to use the chin stands aid.

PENGARUH PENCAHAYAAN, POSISI KERJA DAN GENDER TERHADAP PRODUKTIVITAS VISUAL INSPECTION TASK

<p><em>Visual inspection task is the inspection activity of a product with the eye position centered over the moving conveyor, the faster the product is produced, the more critical the time of the inspection operator. The conditions in the long term will result in a decrease in productivity (Manuaba, 1992). The research objective to be achieved, among others: (1) Determining the optimal level of productivity based on the studies measure of performance checks on experimental design in every visual inspection task beverage packaging cup "Panther". (2) Determine the influence of lighting, working position and gender on the productivity of visual inspection task on beverage packaging. The object of research is part operator checks on beverage packaging cup "panther". Here are 10 people who have IQ scores of 90-120 aged 20-25 years. The research method was experimental method. The independent variables consist of lighting levels, working position, and gender. Variable manipulated into 12 groups experimental design. Data processing is done by checking the performance measurement approach. Analysis of experimental data using Analysis of Variance (ANOVA), and productivity. The study states that: (1) the highest labor productivity LkB200 experimental design (male, standing working position, the lighting level of 200 Lux) for 0.833. But the productivity of all design experiments have not reached optimal productivity, (2) Areas of different interactions in experimental design factor is the level of lighting in the working position. These factors affect the number of detected defects and work productivity.</em></p><div class="page" title="Page 1"><div class="layoutArea"><div class="column"><p><span>Keywords: </span><span>Lighting, Measurement Time, Productivity, Performance Measures Inspection, Analysis of Variance (ANOVA) </span></p></div></div></div><p><em><br /></em></p>

Design and implementation of visual inspection system handed in tokamak flexible in-vessel robot

In-vessel viewing system (IVVS) is a fundamental tool among the remote handling systems for ITER, which is used to providing information on the status of the in-vessel components. The basic functional requirement of in-vessel visual inspection system is to perform a fast intervention with adequate optical resolution. In this paper, we present the software and hardware solution, which is designed and implemented for tokamak in-vessel viewing system that installed on end-effector of flexible in-vessel robot working under vacuum and high temperature. The characteristic of our in-vessel viewing system consists of two parts: binocular heterogeneous vision inspection tool and first wall scene emersion based augment virtuality. The former protected with water-cooled shield is designed to satisfy the basic functional requirement of visual inspection system, which has the capacity of large field of view and high-resolution for detection precision. The latter, achieved by overlaying first wall tiles images onto virtual first wall scene model in 3D virtual reality simulation system, is designed for convenient, intuitive and realistic-looking visual inspection instead of viewing the status of first wall only by real-time monitoring or off-line images sequences. We present the modular division of system, each of them in smaller detail, and go through some of the design choices according to requirements of in-vessel visual inspection task.

Visual Inspection Reliability for Precision Manufactured Parts.

Sandia National Laboratories conducted an experiment for the National Nuclear Security Administration to determine the reliability of visual inspection of precision manufactured parts used in nuclear weapons. Visual inspection has been extensively researched since the early 20th century; however, the reliability of visual inspection for nuclear weapons parts has not been addressed. In addition, the efficacy of using inspector confidence ratings to guide multiple inspections in an effort to improve overall performance accuracy is unknown. Further, the workload associated with inspection has not been documented, and newer measures of stress have not been applied. Eighty-two inspectors in the U.S. Nuclear Security Enterprise inspected 140 parts for eight different defects. Inspectors correctly rejected 85% of defective items and incorrectly rejected 35% of acceptable parts. Use of a phased inspection approach based on inspector confidence ratings was not an effective or efficient technique to improve the overall accuracy of the process. Results did verify that inspection is a workload-intensive task, dominated by mental demand and effort. Hits for Nuclear Security Enterprise inspection were not vastly superior to the industry average of 80%, and they were achieved at the expense of a high scrap rate not typically observed during visual inspection tasks. This study provides the first empirical data to address the reliability of visual inspection for precision manufactured parts used in nuclear weapons. Results enhance current understanding of the process of visual inspection and can be applied to improve reliability for precision manufactured parts.

Integrating new classes on the fly in evolving fuzzy classifier designs and their application in visual inspection

In this paper, we address the problem of integrating new classes on the fly into on-line classification systems. The main focus is on visual inspection tasks, although the concepts proposed in this paper can easily be applied to any other on-line classification systems. We use evolving fuzzy classifiers (EFCs), which can adapt their structure and update their parameters in incrementally due to embedded on-line adaptable classifier learning engines. We consider two different model architectures — classical single model and an all-pairs approach that uses class information to decompose the classification problem into several smaller sub-problems. The latter technique is essential for establishing new classes quickly and efficiently in the classifier, and for reducing class imbalance. Methodological novelties are (i) making appropriate structural changes in the EFC whenever a new class appears while operating in a single-pass incremental manner and (ii) estimating the expected change in classifier accuracy on the older classes. The estimation is based on an analysis of the impact of new classes on the established decision boundaries. This is important for operators, who are already familiar with an established classifier, the accuracy of which is known. The new concepts are evaluated in a real-world visual inspection scenario, where the main task is to classify event types which may occur dynamically on micro-fluidic chips and may reduce their quality. The results show stable performance of established classifiers and efficient (low number of samples requested) as well as fast integration (steeply rising accuracy curves) of new event types (classes).

Multi-Point Stereovision System for Contactless Dimensional Measurements

The paper focuses on the design of a fast and reliable multi-point vision-based measurement system able to be flexible, low cost and accurate for quality control in industrial robotics applications. The paper discusses a new method for integrating visual quality control in highly dynamic manufacturing lines, where products are added or removed from production. The structure of the vision-based measurement system is described. In particular, the stereo system is created by the movement of a single camera mounted on a six-axis manipulator. The visual software is structured in three phases: single camera and stereo calibration, addition of the visual inspection tasks for the object to be measured, stereo measurement. The feasibility of the proposed solution has been tested in a real industrial application with strong requirements on robot speed. The comparison of experimental results on a target with a simple and well-defined shape has shown that the proposed solution provides better results, in terms of accuracy and measurement speed, when compared to commercial libraries and RGBD vision systems. Besides, the robot motion policy adopted by the solution proposed here guarantees a continuous movement without the need for stop&go phases.

Effects of Auditory Cues for Linear and Differential Magnification Methods on Visual Inspection Performance

AbstractThe unfavorable consequence of biased attention to peripheral objects was reported with the use of a newly proposed differential magnification method by the authors. Given the favorable influence of auditory cueing on attention orientation, this study investigated the effectiveness of auditory cues in directing participants’ attention from the peripheral to foveal areas of the differential magnifying lens for improving inspection performance. Thirty‐nine Chinese undergraduates performed a simulated inspection task using a particular magnification method (between‐participant factor). Three within‐participant factors of auditory cue interval, target difficulty, and magnification power were tested. Objective performance measures of speed and accuracy and subjective evaluation using NASA task load index were collected and analyzed. The results showed that positive effects of auditory cues were found for the differential magnification methods but not for the traditional linear magnification method. Significant effects of target difficulty and magnification power were also found. It was concluded that the aid of auditory cues, which did not share the same attentional resource with the visual inspection task, proved to be an appropriate reminder for the differential magnification methods to improve participants’ visual inspection performance. Attention should also be paid to selecting an appropriate auditory cue interval for search tasks of different target difficulties. The findings of this study have implications for the design of video magnifiers in the context of multimodal human–machine interfaces.

Relative saliency model over multiple images with an application to yarn surface evaluation.

Saliency models have been developed and widely demonstrated to benefit applications in computer vision and image understanding. In most of existing models, saliency is evaluated within an individual image. That is, saliency value of an item (object/region/pixel) represents the conspicuity of it as compared with the remaining items in the same image. We call this saliency as absolute saliency, which is uncomparable among images. However, saliency should be determined in the context of multiple images for some visual inspection tasks. For example, in yarn surface evaluation, saliency of a yarn image should be measured with regard to a set of graded standard images. We call this saliency the relative saliency, which is comparable among images. In this paper, a study of visual attention model for comparison of multiple images is explored, and a relative saliency model of multiple images is proposed based on a combination of bottom-up and top-down mechanisms, to enable relative saliency evaluation for the cases where other image contents are involved. To fully characterize the differences among multiple images, a structural feature extraction strategy is proposed, where two levels of feature (high-level, low-level) and three types of feature (global, local-local, local-global) are extracted. Mapping functions between features and saliency values are constructed and their outputs reflect relative saliency for multiimage contents instead of single image content. The performance of the proposed relative saliency model is well demonstrated in a yarn surface evaluation. Furthermore, the eye tracking technique is employed to verify the proposed concept of relative saliency for multiple images.

A Survey on Class Imbalance Learning on Automatic Visual Inspection

The supervised machine learning has been showing very useful for the automatic visual inspection task. However, little has been considered about use traditional machine learning techniques on a domain where the classes are imbalanced. This problem corresponds to dealing with the situation where one class outnumbers the other. Traditional machine learning algorithms trained with imbalance datasets can be biased towards the majority class, thus producing poor predictive accuracy over the minority class. In this paper, we present different approaches to address the class imbalance problem and how these approaches have been used in the context of automatic visual inspection. The literature shows there are few works that consider the class imbalance problem on automatic visual inspection task and it shows that the one class classification technique is the most used.

Deposition uniformity inspection in IC wafer surface

This paper focuses on the task of automatic visual inspection of color uniformity on the surface of integrated circuits (IC) wafers arising from the layering process. The oxide thickness uniformity within a given wafer with a desired target thickness is of great importance for modern semiconductor circuits with small oxide thickness. The non-uniform chemical vapor deposition (CVD) on a wafer surface will proceed to fail testing in Wafer Acceptance Test (WAT). Early detection of non-uniform deposition in a wafer surface can reduce material waste and improve production yields. The fastest and most low-priced inspection method is a machine vision-based inspection system. In this paper, the proposed visual inspection system is based on the color representations which were reflected from wafer surface. The regions of non-uniform deposition present different colors from the uniform background in a wafer surface. The proposed inspection technique first learns the color data via color space transformation from uniform deposition of normal wafer surfaces. The individual small region statistical comparison scheme then proceeds to the testing wafers. Experimental results show that the proposed method can effectively detect the non-uniform deposition regions on the wafer surface. The inspection time of the deposited wafers is quite compatible with the atmospheric pressure CVD time.

Controlling a system for underwater visual inspection

Nowadays, the ocean plays a fundamental role in the global economy, mainly due to oil extraction industry. It makes the environment be populated with human-made structures that needs to be inspected and maintained. In this context, this paper details a system for online detect an underwater cable-like target using computer vision algorithms, as well as an automatic control of a vehicle to tracking it. This system could be used to assist a human operator during visual inspection tasks. This work is concluded with a series of tests and analyses aiming to the system validation.

Development and evaluation of specific lighting scenarios and ergonomic approaches to optimise visual inspection tasks

The purpose of the several studies reported here was to improve the lighting conditions at industrial visual testing and inspection workplaces beyond the specifications given in technical standards. In this context, special emphasis was laid on the examination of the effects of directional and diffuse lighting as well as of sample-specific lighting scenarios with respect to the ability to detect three-dimensional (3D) details. The sample-specific lighting scenarios were designed according to ergonomic principles and adapted to different material surfaces. Landolt rings of various sizes were laser-engraved on several surface types, which are representative of the surfaces encountered in industrial final inspection tasks. The sample surfaces were fixed in a special object mount, which allowed defined and reproducible test arrangements. In a first step, the luminance distribution on the samples and the contrast between the Landolt rings and the background of the samples were measured depending on different illumination types and inclinations of the surface types. Additionally, laboratory tests were carried out in which the test subjects’ visual acuity was measured while detecting 3D details (e.g. the Landolt rings) on the surfaces of different materials illuminated in different ways. It turned out that all results were strongly dependent on the inclination of the samples rather than on the lighting scenarios applied. In a further study, the time-related performance in detecting surface flaws on samples under different lighting scenarios was measured. For this purpose, the average error quota was assessed, i.e. the proportion of wrongly identified Landolt rings among the overall number of optotypes and the response time to the identification of a presented optotype were determined. Both parameters gave evidence of the long-term advantage of the ergonomic sample-specific lighting scenarios. In a last step, the various illuminations were evaluated through a structured interview of the test subjects. The subjective assessment of the illuminations also indicated a high dependency on the geometric arrangement between light source and the sample under examination. Especially, in the context of long-term inspections, the subjects preferred the sample-specific lighting scenarios. The diffuse lighting was also assessed predominantly and positively. The illumination with directional lighting and with the lighting according to technical standards was, mostly, rated disadvantageously.

A Sampling Inspection Plan with Human Error: Considering the Relationship between Visual Inspection Time and Human Error Rate

Objective: The aim of this study is to design a sampling inspection plan with human error which is changing according to inspection time. Background: Typical sampling inspection plans have been established typically based on an assumption of the perfect inspection without human error. However, most of all inspection tasks include human errors in the process of inspection. Therefore, a sampling inspection plan should be designed with consideration of imperfect inspection. Method: A model for single sampling inspection plans were proposed for the cases that visual inspection error rate is changing according to inspection time. Additionally, a sampling inspection plan for an optimal inspection time was proposed. In order to show an applied example of the proposed model, an experiment for visual inspection task was performed and the inspection error rates were measured according to the inspection time. Results: Inspection error rates changed according to inspection time. The inspection error rate could be reflected on the single sampling inspection plans for attribute. In particular, inspection error rate in an optimal inspection time may be used for a reasonable single sampling plan in a practical view. Conclusion: Human error rate in inspection tasks should be reflected on typical single sampling inspection plans. A sampling inspection plan with consideration of human error requires more sampling number than a typical sampling plan with perfect inspection. Application: The result of this research may help to determine more practical sampling inspection plan rather than typical one.

Use of saccadic latency for visual inspection system

In some industrial inspection processes, multiprocess-handling workers are required to enter the results of visual inspection tests without using their hands because of efficiency or hygiene reasons. We have developed a hands-free visual inspection system by using saccadic latency, a temporal characteristic of saccadic eye movements. The proposed system is free from the Midas touch problem (i.e., the difficulty in developing an eye-typing interface owing to the difficulty in differentiating between intentional blinks and gazes and natural ones). For verifying the system, an experiment was conducted in which ten subjects performed a visual inspection task. The average defect detection rates were 99.4%, and no Midas touch–related errors were observed. Results of the error analysis showed that redesigning the system interface would lead to a further enhancement of the system performance. © 2011 Wiley Periodicals, Inc. © 2012 Wiley Periodicals, Inc.

Onboard object recognition for planetary exploration

Machine learning techniques have shown considerable promise for automating common visual inspection tasks such as the detection of human faces in cluttered scenes. Here, we examine whether similar techniques can be used (or adapted) for the problem of automatically locating geologic landforms in planetary images gathered by spacecraft. Beyond enabling more efficient and comprehensive ground analysis of down-linked data, we are aiming toward perceptive spacecraft that use onboard processing to autonomously analyze their collected imagery and take appropriate actions. In our current study, we have employed various supervised learning algorithms, including neural networks, ensemble methods, support vector machines (SVM), and continuously-scalable template models (CSTM) to derive detectors for craters from ground-truthed images. The resulting detectors are evaluated on a challenging set of Viking Orbiter images of Mars containing roughly one thousand craters. The SVM approach with normalized image patches provides detection and localization performance closest to that of human labelers and is shown to be substantially superior to boundary-based approaches such as the Hough transform. However, the run-time cost in applying the SVM solution in the standard way (spatial scanning in which the SVM is applied to each patch of the image on a window-by-window basis) is too high due both to the number of support vectors required and the number of test vectors generated by sliding a window across the data. We have developed an implementation using FFTs and the overlap-and-add technique, which can be used to efficiently apply SVMs to sensor data in resource-constrained environments such as on a spacecraft. The technique allows exact computation of the SVM decision function over an image using minimal RAM (typically less than 5% of the size of the image) and only ${\mathcal{O}}(n_{s} (\log_{2} d + 11))$ real multiplications per pixel, where n s is the number of support vectors and d is the dimensionality of the vectors compared with ${\mathcal{O}}(n_{s} d)$ real multiplications per pixel for spatial scanning. Our approach is complementary to reduced set methods providing (in theory) a multiplicative gain in performance.

Data mining for quality control: Burr detection in the drilling process

Drilling process is one of the most important operations in aeronautic industry. It is performed on the wings of the aeroplanes and its main problem lies with the burr generation. At present moment, there is a visual inspection and manual burr elimination task subsequent to the drilling and previous to the riveting to ensure the quality of the product. These operations increase the cost and the resources required during the process. The article shows the use of data mining techniques to obtain a reliable model to detect the generation of burr during high speed drilling in dry conditions on aluminium Al 7075-T6. It makes possible to eliminate the unproductive operations in order to optimize the process and reduce economic cost. Furthermore, this model should be able to be implemented later in a monitoring system to detect automatically and on-line when the generated burr is out of tolerance limits or not. The article explains the whole process of data analysis from the data preparation to the evaluation and selection of the final model.

The relationship between stimulus-response mappings and the detection of novel stimuli in a simulated luggage screening task

Automaticity research suggests that consistent mapping leads to better skill acquisition than varied mapping. Other research demonstrates that in some situations varied mapping leads to better transfer. The effect of stimulus-response mappings in complex visual inspection tasks, wherein transfer stimuli might differ from training stimuli, has seldom been studied. Therefore, the effects of consistency vs variability of practice on learning and transfer in a simulated luggage screening task were compared. Consistent mapping led to faster and more accurate initial skill acquisition. However, during transfer, varied mapping led to higher levels of sensitivities and confidence and fewer deviations from optimal response criteria. Consistent mapping assists initial skill acquisition; however, in tasks where the physical identity of transfer stimuli might differ from those used in training, varied mapping leads to more efficient transfer. The results provide an important starting point for training individuals to achieve optimal transfer of learning in complex tasks. This research demonstrates that a combination of variables influence transfer of learning in real-world visual inspection tasks wherein transfer conditions may not be identical to training conditions. Acquisition of skills during training is facilitated by consistent stimulus-response mappings; however, in order to ensure optimal transfer of skills to situations involving novel stimuli, training should incorporate varied mapping of stimulus-response elements.

Efficient Heuristics for Discriminative Structure Learning of Bayesian Network Classifiers

We introduce a simple order-based greedy heuristic for learning discriminative structure within generative Bayesian network classifiers. We propose two methods for establishing an order of N features. They are based on the conditional mutual information and classification rate (i.e., risk), respectively. Given an ordering, we can find a discriminative structure with O(Nk+1) score evaluations (where constant k is the tree-width of the sub-graph over the attributes). We present results on 25 data sets from the UCI repository, for phonetic classification using the TIMIT database, for a visual surface inspection task, and for two handwritten digit recognition tasks. We provide classification performance for both discriminative and generative parameter learning on both discriminatively and generatively structured networks. The discriminative structure found by our new procedures significantly outperforms generatively produced structures, and achieves a classification accuracy on par with the best discriminative (greedy) Bayesian network learning approach, but does so with a factor of ~10-40 speedup. We also show that the advantages of generative discriminatively structured Bayesian network classifiers still hold in the case of missing features, a case where generative classifiers have an advantage over discriminative classifiers.

Towards incremental classifier fusion

To process the large amounts of data industrial systems are producing nowadays, machine learning techniques have shown their usefulness in many applications. As the amounts of data being generated are getting huge, the need for machine learning methods which can deal with them in an appropriate way - i.e. methods which can be adapted incrementally - becomes very important. Ensembles of classifiers have been shown to be able to improve the predictive accuracy as well as the robustness of single classification methods. In this work novel incremental variants of several well-known classifier fusion methods (Fuzzy Integral, Decision Templates, Dempster-Shafer Combination and Discounted Dempster-Shafer Combination) are presented. Furthermore, a novel incremental classifier fusion method called Incremental Direct Cluster-based ensemble will be introduced, which exploits an evolving clustering approach. A flexible and interactive framework for on-line learning will be introduced, in which the ensemble methods are adapted incrementally in a sample-wise manner together with their base classifiers. The performance of this framework and the proposed incremental classifiers fusion methods therein are evaluated on five real-world visual quality inspection tasks, captured on-line from an industrial CD imprint production process, together with five data sets from the UCI repository.