Industrial Vision Systems Research Articles

A Study on Elements of Machine Vision and Machine Learning in Industrial Vision Systems

The article presents an analysis of issues regarding the current state and prospects of using machine learning elements in industrial applications of vision systems. The analysis is preceded by a short introduction to the subject of machine vision as well as a presentation of the results of research on industrial 3D vision systems, leading to conclusions that affect the prospects for further development of vision systems with elements of machine learning.

Generic FPGA Pre-Processing Image Library for Industrial Vision Systems.

Currently, there is a demand for an increase in the diversity and quality of new products reaching the consumer market. This fact imposes new challenges for different industrial sectors, including processes that integrate machine vision. Hardware acceleration and improvements in processing efficiency are becoming crucial for vision-based algorithms to follow the complexity growth of future industrial systems. This article presents a generic library of pre-processing filters for execution in field-programmable gate arrays (FPGAs) to reduce the overall image processing time in vision systems. An experimental setup based on the Zybo Z7 Pcam 5C Demo project was developed and used to validate the filters described in VHDL (VHSIC hardware description language). Finally, a comparison of the execution times using GPU and CPU platforms was performed as well as an evaluation of the integration of the current work in an industrial application. The results showed a decrease in the pre-processing time from milliseconds to nanoseconds when using FPGAs.

Modelling the Interaction of Illuminations with a Sensor Matrix Used in Vision Systems

The article discusses a method of modelling the interaction of industrial illuminators with sensor arrays used in industrial vision systems cameras. The research used a model containing a light source and a sensor matrix. As part of the research, the average intensity of electromagnetic radiation in the visible range on the surface of the sensor matrix was measured, as well as its analysis and interpretation using a model of illuminators. The light source is described based on the measurement of an authentic industrial illuminator, for which a photometric solid was determined using a photogoniometer and spectral irradiance with a spectroradiometer. A theoretical model of the matrix was prepared, enabling the selection and control of the parameters of its work. As part of the conducted research, the impact of changing the lighting model parameters, such as the photometric solid, spectral irradiance, and number of rays, and the effect of changing the matrix parameters, such as the dimensions, number of pixels, on the irradiance measurement, were checked. The results of simulation tests are presented for selected sets of models of the light source and for the model of sensors’ matrix. The summary presents the impact of changes in parameters adopted for models on the simulation results.

Viewpoint Planning for Range Sensors Using Feature Cluster Constrained Spaces for Robot Vision Systems.

The efficient computation of viewpoints for solving vision tasks comprising multi-features (regions of interest) represents a common challenge that any robot vision system (RVS) using range sensors faces. The characterization of valid and robust viewpoints is even more complex within real applications that require the consideration of various system constraints and model uncertainties. Hence, to address some of the challenges, our previous work outlined the computation of valid viewpoints as a geometrical problem and proposed feature-based constrained spaces (C-spaces) to tackle this problem efficiently for acquiring one feature. The present paper extends the concept of C-spaces to consider multi-feature problems using feature cluster constrained spaces (GC-spaces). A GC-space represents a closed-form, geometrical solution that provides an infinite set of valid viewpoints for acquiring a cluster of features satisfying diverse viewpoint constraints. Furthermore, the current study outlines a generic viewpoint planning strategy based on GC-spaces for solving vision tasks comprising multi-feature scenarios effectively and efficiently. The applicability of the proposed framework is validated on two different industrial vision systems used for dimensional metrology tasks.

Design and Deployment of a Generic Software for Managing Industrial Vision Systems

Computer vision systems have become a key element of modern companies. However, its implantation not only requires specialized hardware components but also software to control the correct performance of all these components. The purpose of this article is to present the design and deployment of a software created to manage the different computer vision systems of any company. First, using our experience in mechanization and industrialization of company processes, a general company specification will be proposed as well as the main software requirements that have to be satisfied. Then, a three-level modular design composed of the core, a configuration module, and user interfaces with functionalities capable of satisfying the defined requirements will be presented. Special attention will be given to time restrictions and components’ synchronization. In addition, the different tests that have been carried out to control the correct performance of the software will be shown. The development process will end with a generic, modular, and scalable software able to fit different industrial scenarios by simply modifying a set of input parameters. To illustrate the correct performance of the proposal, the details of its installation in four real companies with different needs will be presented. The proposed work has a practical use in industry and it also provides a thorough description of the main components involved in computer vision systems of real company environments and how to manage them. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The aim of the article is to provide a software solution to manage the computer vision systems of a company. The proposed software is simple enough to be controlled by company operators with no need for great expertise on the technical aspects, but rather on the industrial workflow to be controlled. To develop such a software, it has been necessary to know about all the elements involved in the industrial scenarios controlled by computer vision systems as well as their relationships. This article presents all these elements providing a global and complete view of the problem with special attention to synchronization, which is fundamental to satisfy industrial time restriction requirements. The software is presented from a technical point of view, but also from a practical one, since different examples of its application in real cases are given. These examples illustrate the adaptability of the proposal to different scenarios and also exemplify how the different elements are modified to fit the different situations. In addition, this article provides a comprehensive overview of real industrial situations, focusing on the most important issues faced during real installations.

Novel object motion estimation method for industrial vision systems in aligning machines

Industrial vision systems have been adopted in many manufacturing processes for monitoring manufacturing procedures. With the recent proposed concept of industrial information integration engineering (IIIE) (Da Xu, 2020 [1]), the adoption of industrial vision systems involves a wider IIIE context as a data acquisition measurement, and thus higher level intelligence like control automation is expected. Industrial vision systems have been used for object alignment in aligning machines. By coping with a pneumatic device, misaligned objects can be detected by vision systems and rejected. Since alignment efficiency depends on the appropriateness of the blown timing/pressure, which is casually manifested in the motion of the observed blown objects, a motion estimation method for industrial vision systems must be adopted. Although conventional motion estimation methods in this field typically use pre-prepared templates, this paper proposed a novel object motion estimation method based on the properties of industrial images. A bounding box within a confined observation area is first initialized for each object, and then motion estimation is achieved by updating the bounding box following the expectation–maximization principle. Experiments revealed that our proposed method achieved robust and continuous motion estimation of the parts and reduced the processing time compared to conventional template matching methods.

Industrial objects recognition in intelligent manufacturing for computer vision

PurposeThe overall goal of this research is to develop algorithms for feature-based recognition of 2D parts from intensity images. Most present industrial vision systems are custom-designed systems, which can only handle a specific application. This is not surprising, since different applications have different geometry, different reflectance properties of the parts.Design/methodology/approachComputer vision recognition has attracted the attention of researchers in many application areas and has been used to solve many ranges of problems. Object recognition is a type of pattern recognition. Object recognition is widely used in the manufacturing industry for the purpose of inspection. Machine vision techniques are being applied in areas ranging from medical imaging to remote sensing, industrial inspection to document processing and nanotechnology to multimedia databases. In this work, recognition of objects manufactured in mechanical industry is considered. Mechanically manufactured parts have recognition difficulties due to manufacturing process including machine malfunctioning, tool wear and variations in raw material. This paper considers the problem of recognizing and classifying the objects of such mechanical part. Red, green and blue RGB images of five objects are used as an input. The Fourier descriptor technique is used for recognition of objects. Artificial neural network (ANN) is used for classification of five different objects. These objects are kept in different orientations for invariant rotation, translation and scaling. The feed forward neural network with back-propagation learning algorithm is used to train the network. This paper shows the effect of different network architecture and numbers of hidden nodes on the classification accuracy of objects as well as the effect of learning rate and momentum.FindingsOne important finding is that there is not any considerable change in the network performances after 500 iterations. It has been found that for data smaller network structure, smaller learning rate and momentum are required. The relative sample size also has a considerable effect on the performance of the classifier. Further studies suggest that classification accuracy is achieved with the confusion matrix of the data used. Hence, with these results the proposed system can be used efficiently for more objects. Depending upon the manufacturing product and process used, the dimension verification and surface roughness may be integrated with proposed technique to develop a comprehensive vision system. The proposed technique is also highly suitable for web inspections, which do not require dimension and roughness measurement and where desired accuracy is to be achieved at a given speed. In general, most recognition problems provide identity of object with pose estimation. Therefore, the proposed recognition (pose estimation) approach may be integrated with inspection stage.Originality/valueThis paper considers the problem of recognizing and classifying the objects of such mechanical part. RGB images of five objects are used as an input. The Fourier descriptor technique is used for recognition of objects. ANN is used for classification of five different objects. These objects are kept in different orientations for invariant rotation, translation and scaling. The feed forward neural network with back-propagation learning algorithm is used to train the network. This paper shows the effect of different network architecture and numbers of hidden nodes on the classification accuracy of objects as well as the effect of learning rate and momentum.

Magic Line: An Integrated Method for Fast Parts Counting and Orientation Recognition Using Industrial Vision Systems

Magic Line: An Integrated Method for Fast Parts Counting and Orientation Recognition Using Industrial Vision Systems

Unmanned machine vision system for automated recognition of mechanical parts

PurposeThe field of machine vision, or computer vision, has been growing at fast pace. The growth in this field, unlike most established fields, has been both in breadth and depth of concepts and techniques. Machine vision techniques are being applied in areas ranging from medical imaging to remote sensing, industrial inspection to document processing and nanotechnology to multimedia databases. The goal of a machine vision system is to create a model of the real world from images. Computer vision recognition has attracted the attention of researchers in many application areas and has been used to solve many ranges of problems. The purpose of this paper is to consider recognition of objects manufactured in mechanical industry. Mechanically manufactured parts have recognition difficulties due to manufacturing process including machine malfunctioning, tool wear and variations in raw material. This paper considers the problem of recognizing and classifying the objects of such parts. RGB images of five objects are used as an input. The Fourier descriptor technique is used for recognition of objects. Artificial neural network (ANN) is used for classification of five different objects. These objects are kept in different orientations for invariant rotation, translation and scaling. The feed forward neural network with back-propagation learning algorithm is used to train the network. This paper shows the effect of different network architecture and numbers of hidden nodes on the classification accuracy of objects.Design/methodology/approachThe overall goal of this research is to develop algorithms for feature-based recognition of 2D parts from intensity images. Most present industrial vision systems are custom-designed systems, which can only handle a specific application. This is not surprising, since different applications have different geometry, different reflectance properties of the parts.FindingsClassification accuracy is affected by the changing network architecture. ANN is computationally demanding and slow. A total of 20 hidden nodes network structure produced the best results at 500 iterations (90 percent accuracy based on overall accuracy and 87.50 percent based on κ coefficient). So, 20 hidden nodes are selected for further analysis. The learning rate is set to 0.1, and momentum term used is 0.2 that give the best results architectures. The confusion matrix also shows the accuracy of the classifier. Hence, with these results the proposed system can be used efficiently for more objects.Originality/valueAfter calculating the variation of overall accuracy with different network architectures, the results of different configuration of the sample size of 50 testing images are taken. Table II shows the results of the confusion matrix obtained on these testing samples of objects.

Towards increased intelligence and automatic improvement in industrial vision systems

Robots and in-process inspection systems equipped with machine vision solutions are used for increased flexibility and quality in automated manufacturing. Although vision systems have found wide industrial use, there are still problems regarding optimization of vision system robustness and capabilities. This paper presents a comprehensive case study of vision system functions, techniques and capabilities in an automotive 1-tiers supplier. Based on the study, the paper further describes a method for systematic improvement of industrial vision systems on a continuous basis. This is proposed to be done by establishing a data store and data analysis system, based on training machine learning models in an off-line mode using the historical data, as well as on on-line stream processing.

Review of techniques for 2D camera calibration suitable for industrial vision systems

In camera calibration the goal is to determine a set of camera parameters that describe the mapping between 3-D reference coordinates and 2-D image ones. Correction for image distortion in cameras is an important topic in order to obtain accurate information from the vision systems. A review of calibration techniques for the vision systems is presented among the different methods for camera calibration found in literature. The advantages and limitations of these techniques are also discussed. This work analyzes the techniques that can be useful in modern industrial contest with the aim of knowing the effective distortion of the lens and the possibility to determinate the measurement uncertainty of the vision system.

On-line knowledge acquisition and enhancement in robotic assembly tasks

Industrial robots are reliable machines for manufacturing tasks such as welding, panting, assembly, palletizing or kitting operations. They are traditionally programmed by an operator using a teach pendant in a point-to-point scheme with limited sensing capabilities such as industrial vision systems and force/torque sensing. The use of these sensing capabilities is associated to the particular robot controller, operative systems and programming language. Today, robots can react to environment changes specific to their task domain but are still unable to learn skills to effectively use their current knowledge. The need for such a skill in unstructured environments where knowledge can be acquired and enhanced is desirable so that robots can effectively interact in multimodal real-world scenarios. In this article we present a multimodal assembly controller (MAC) approach to embed and effectively enhance knowledge into industrial robots working in multimodal manufacturing scenarios such as assembly during kitting operations with varying shapes and tolerances. During learning, the robot uses its vision and force capabilities resembling a human operator carrying out the same operation. The approach consists of using a MAC based on the Fuzzy ARTMAP artificial neural network in conjunction with a knowledge base. The robot starts the operation having limited initial knowledge about what task it has to accomplish. During the operation, the robot learns the skill for recognising assembly parts and how to assemble them. The skill acquisition is evaluated by counting the steps to complete the assembly, length of the followed assembly path and compliant behaviour. The performance improves with time so that the robot becomes an expert demonstrated by the assembly of a kit with different part geometries. The kit is unknown by the robot at the beginning of the operation; therefore, the kit type, location and orientation are unknown as well as the parts to be assembled since they are randomly fed by a conveyor belt.

Statistical Learning Algorithms to Compensate Slow Visual Feedback for Industrial Robots

Vision guided robots have become an important element in the manufacturing industry. In most current industrial applications, vision guided robots are controlled by a look-then-move method. This method cannot support many new emerging demands which require real-time vision guidance. Challenge comes from the speed of visual feedback. Due to cost limit, industrial robot vision systems are subject to considerable latency and limited sampling rate. This paper proposes new algorithms to address this challenge by compensating the latency and slow sampling of visual feedback so that real-time vision guided robot control can be realized with satisfactory performance. Statistical learning methods are developed to model the pattern of target's motion adaptively. The learned model is used to recover visual measurement from latency and slow sampling. The imaging geometry of the camera and all-dimensional motion of the target are fully considered. Tests are conducted to provide evaluation from different aspects.

An Analysis of Uncertainty and Robustness of Waterjet Machine Positioning Vision System

The paper presents a new Automatic Waterjet Positioning Vision System (AWPVS) and investigates components of workpiece positioning accuracy. The main purpose of AWPVS is to precisely identify the position and rotation of a workpiece placed on a waterjet machine table. Two webcams form a basis for the system, and constitute its characteristics. The proposed algorithm comprises various image processing techniques to assure a required identification precision. To validate the PVS identification quality, synthetic images were applied under various conditions. The analysis ascertains dependence of an object detection rate and accuracy on a size of cropping frame. Experimental results of the proposed PVS prototype prove that a combination of the vision algorithm and webcams is an alternative to dedicated expensive industrial vision systems. The two main components of AWPVS uncertainty, a machine component and PSV component are discerned and estimated.

Mixed-color Illumination and Quick Optimum Search for Machine Vision

Quick search for optimal illumination using mixed color is proposed, and the possibility for use in industrial vision systems is considered. Multidimensional formulation of the optimum search methods is reviewed to use multiple inputs for various color sources and one output of image sharpness. The voltage inputs are used to adjust the intensity of each color source and the optimal illumination is found by the maximum of the sharpness. The optimum search methods reveal that the number of iterations for the optimal condition in mixed color illumination was under 0.1% the number required by equal section search. The searching paths and convergence were considered in both of the input coordinate and the color coordinate. The results show that the optimum search methods make it possible to find the optimal lighting conditions quickly and accurately.

기하학적 영상왜곡의 보정을 위한 제어영역 감소 방법

In the industrial vision systems, image correction has great influence on the overall performance of measurement or inspection. The overall area of distorted image is usually splitted into small control areas, and each area is corrected by its control points. The performance of correction methods using control points can be improved by reduction of control areas because the computational time and memory highly depend on the number of control areas. We develop a merging algorithm that reduces control areas and preserves the correction accuracy. The algorithm merges the splitted control areas by use of quad tree method. Experimental results are presented to verify the usefulness of the proposed method.

Monocular 3D scene reconstruction at absolute scale

In this article we propose a method for combining geometric and real-aperture methods for monocular three-dimensional (3D) reconstruction of static scenes at absolute scale. Our algorithm relies on a sequence of images of the object acquired by a monocular camera of fixed focal setting from different viewpoints. Object features are tracked over a range of distances from the camera with a small depth of field, leading to a varying degree of defocus for each feature. Information on absolute depth is obtained based on a Depth-from-Defocus approach. The parameters of the point spread functions estimated by Depth-from-Defocus are used as a regularisation term for Structure-from-Motion. The reprojection error obtained from bundle adjustment and the absolute depth error obtained from Depth-from-Defocus are simultaneously minimised for all tracked object features. The proposed method yields absolutely scaled 3D coordinates of the scene points without any prior knowledge about scene structure and camera motion. We describe the implementation of the proposed method both as an offline and as an online algorithm. Evaluating the algorithm on real-world data, we demonstrate that it yields typical relative scale errors of a few per cent. We examine the influence of random effects, i.e. the noise of the pixel grey values, and systematic effects, caused by thermal expansion of the optical system or by inclusion of strongly blurred images, on the accuracy of the 3D reconstruction result. Possible applications of our approach are in the field of industrial quality inspection; in particular, it is preferable to stereo cameras in industrial vision systems with space limitations or where strong vibrations occur.

Remarks on Hardware Implementation of Image Processing Algorithms

Remarks on Hardware Implementation of Image Processing AlgorithmsImage processing in industrial vision systems requires both real-time speed and robustness. Modern computers, which fulfill the first demand, are sensitive to hard industrial environment conditions and require considerable amounts of energy. Programmable logic chips are available, which can realize many simple, still time-consuming operations in a parallel or a pipelined manner. The paper discusses particular features of the pipelined architecture and presents selected techniques of implementing early image processing procedures in hardware.

Automated vision system for localizing structural defects in textile fabrics

Quality control is one of the basic issues in textile industry. Analysis of texture content in digital images plays an important role in the automated visual inspection of textile images to detect their defects. In this paper, a system for automated visual inspection of textiles is discussed. A detailed system configuration is presented and a fault detection algorithm is proposed. Industrial vision systems must operate in real-time, produce a low false alarm rate and be flexible to accommodate variations in inspection sites. This was the rationale behind developing a detection algorithm which employs simple statistical features (mean, variance, median). The intent was to utilize such features to make the calculations simple and fast for the system to be suitable for real-time applications. The performance of the system was evaluated on plain fabrics with different types of textile flaws. The results indicate that the system can detect flaws which vary drastically in physical dimension and nature with a very low false detection rate.

A survey on industrial vision systems, applications and tools

The state of the art in machine vision inspection and a critical overview of real-world applications are presented in this paper. Two independent ways to classify applications are proposed, one according to the inspected features of the industrial product or process and the other according to the inspection independent characteristics of the inspected product or process. The most contemporary software and hardware tools for developing industrial vision systems are reviewed. Finally, under the light of recent advances in image sensors, software and hardware technology, important issues and directions for designing and developing industrial vision systems are identified and discussed. (C) 2003 Elsevier Science B.V. All rights reserved.