Development Of Machine Vision System Research Articles

DEVELOPMENT OF A MACHINE VISION SYSTEM FOR DAMAGE AND OBJECT DETECTION IN TUNNELS USING CONVOLUTIONAL NEURAL NETWORKS

Abstract. Tunnel inspection, i.e. detection of damages and defects on concrete surfaces, is essential for monitoring structural reliability and health conditions of transport facilities, thus providing safe and sustainable urban transportation infrastructures. In this study, an innovative visual-based system is developed for damage and object detection tasks in roadway tunnels based on deep learning techniques. The main components of the developed Machine Vision System such as industrial cameras, flash-based light sources, controller, the synchronization unit and corresponding software programs are designed to collect high-resolution images with sufficient quality from dimly lit tunnel environments in normal traffic flows with an operating speed of 30–50 km/h. Unlike recent studies, the training data includes multiple types of damage such as cracks, spalling, rust, delamination and other surface changes. Furthermore, 10 classes of common tunnel objects including traffic signs, traffic cameras, traffic lights, ventilation ducts, various sensors and cables are labeled for object detection. As state-of-the-art Convolutional Neural Networks, DeepLab and U-Net are trained and evaluated using accuracy metrics for image segmentation. The results highlight the most important parameters of the discussed Machine Vision System as well as the performance of DeepLab and U-Net for object and damage detection.

Tree Trunk Detection of Eastern Red Cedar in Rangeland Environment with Deep Learning Technique

Uncontrolled spread of eastern red cedar invades the United States Great Plains prairie ecosystems and lowers biodiversity across native grasslands. The eastern red cedar (ERC) infestations cause significant challenges for ranchers and landowners, including the high costs of removing mature red cedars, reduced livestock forage feed, and reduced revenue from hunting leases. Therefore, a fleet of autonomous ground vehicles (AGV) is proposed to address the ERC infestation. However, detecting the target tree or trunk in a rangeland environment is critical in automating an ERC cutting operation. A tree trunk detection method was developed in this study for ERC trees trained in natural rangeland environments using a deep learning-based YOLOv5 model. An action camera acquired RGB images in a natural rangeland environment. A transfer learning method was adopted, and the YOLOv5 was trained to detect the varying size of the ERC tree trunk. A trained model precision, recall, and average precision were 87.8%, 84.3%, and 88.9%. The model accurately predicted the varying tree trunk sizes and differentiated between trunk and branches. This study demonstrated the potential for using pretrained deep learning models for tree trunk detection with RGB images. The developed machine vision system could be effectively integrated with a fleet of AGVs for ERC cutting. The proposed ERC tree trunk detection models would serve as a fundamental element for the AGV fleet, which would assist in effective rangeland management to maintain the ecological balance of grassland systems.

Development of an Intelligent Imaging System for Ripeness Determination of Wild Pistachios.

Rapid, non-destructive, and smart assessment of the maturity levels of fruit facilitates their harvesting and handling operations throughout the supply chain. Recent studies have introduced machine vision systems as a promising candidate for non-destructive evaluations of the ripeness levels of various agricultural and forest products. However, the reported models have been fruit-specific and cannot be applied to other fruit. In this regard, the current study aims to evaluate the feasibility of estimating the ripeness levels of wild pistachio fruit using image processing and artificial intelligence techniques. Images of wild pistachios at four ripeness levels were recorded using a digital camera, and 285 color and texture features were extracted from 160 samples. Using the quadratic sequential feature selection method, 16 efficient features were identified and used to estimate the maturity levels of samples. Linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and an artificial neural network (ANN) were employed to classify samples into four ripeness levels, including initial unripe, secondary unripe, ripe, and overripe. The developed machine vision system achieved a correct classification rate (CCR) of 93.75, 97.5, and 100%, respectively. The high accuracy of the developed models confirms the capability of the low-cost visible imaging system in assessing the ripeness of wild pistachios in a non-destructive, automated, and rapid manner.

Estimation of dust concentration by a novel machine vision system

The dust phenomenon is one of the main environmental problems that it reversely affects human health and economical and social activities. In the present research, a novel algorithm has been developed based on image processing to estimate dust concentration. An experimental setup was implemented to create airborne dust with different concentration values from 0 to 2750 µg.m−3. The images of the different dust concentration values were acquired and analyzed by image processing technique. Different color and texture features were extracted from various color spaces. The extracted features were used to develop single and multivariable models by regression method. Totally 285 single variable models were obtained and compared to select efficient features among them. The best single variable model had a predictive accuracy of 91%. The features were used for multivariable modeling and the best model was selected with a predictive accuracy of 100% and a mean squared error of 1.44 × 10−23. The results showed the high ability of the developed machine vision system for estimating dust concentration with high speed and accuracy.

Non-Destructive Estimation of Physicochemical Properties and Detection of Ripeness Level of Apples Using Machine Vision

ABSTRACT Nondestructive estimation of physicochemical properties, post-harvest physiology, and level of ripeness of fruits is essential to their automated harvesting, sorting, and handling. Recent research efforts have identified machine vision systems as a promising noninvasive nondestructive tool for exploring the relationship between physicochemical and appearance characteristics of fruits at various ripening levels. In this regard, the purpose of the current study is to provide an intelligent algorithm for estimating two physical properties including firmness, and soluble solid content (SSC), three chemical properties viz. starch, acidity, and titratable acidity (TA), as well as detection of the ripening level of apples (cultivar Red Delicious) using video processing and artificial intelligence. To this end, videos of apples in orchards at four levels of ripeness were recorded and 444 color and texture features were extracted from these samples. Five physicochemical properties including firmness, SSC, starch, acidity, and TA were measured. Using the hybrid artificial neural network-difference evolution (ANN-DE), six most effective features (one texture and five color features) were selected to estimate the physicochemical properties of apples. The physicochemical estimation was then further optimized using a hybrid multilayer perceptron artificial neural network-cultural algorithm (ANN-CA). The results showed that the coefficient of determinations (R2) related to the prediction models for the physicochemical properties were in excess of 0.92. Additionally, the ripeness level of apples was estimated based on physicochemical properties using a hybrid multilayer perceptron artificial neural network-harmonic search algorithm (ANN-HS) classifier. The developed machine vision system examined ripeness levels of 1356 apples in natural orchard environments and achieved a correct classification rate (CCR) of 97.86%.

Development of Machine Vision System for Off-Line Inspection of Fine Defects on Glass Screen Surface

The complicated manufacturing process and poor industrial environment inevitably produces unacceptable scratches on glass screen surface. In this study, a machine vision system is developed to automatically detect fine scratch, which has a width smaller than 30 μm, on glass screen surface for the purpose of appearance quality assessment. The imaging hardware of the proposed machine vision system is made up of a high-resolution camera and a wide field lens so that a high-quality image is achieved in a relatively large image area. The captured glass screen images were firstly classified with YOLOv3 to exactly distinguish fine scratches in two other kinds of defects including dusts and stains. Then the U-Net network was applied on the selected scratches to estimate their dimensions approximately. The proposed algorithms combination has a fast image processing speed and it is able to effectively reduce the false segmentation of other defects. The results of the developed machine vision system show a relatively good agreement with the well-established microscopic method. It demonstrates our machine vision method is able to image and quantitative analyze scratches with dimensions between 25 μm to 100 μm.

Defective egg detection based on deep features and Bidirectional Long-Short-Term-Memory

Eggs are one of the most important nutritional sources worldwide. In addition, defects that may occur in the eggshells endanger food safety and cause adverse effects on production companies. Automatic separation of defective eggs from quality eggs is an important issue due to economic and health reasons. On this motivation, a real-time machine vision system based on deep learning has been developed for the detection of cracked, bloody and dirty eggs. In this study, a continuous rotating system was designed to visualize all surfaces of the egg. Thus, the adverse conditions such as dirt, blood, and cracks that may occur on any surface of the egg have been successfully monitored. In the proposed system for the detection of robust eggs, deep features are extracted using a pre-trained residual network model and then the obtained features are fed into the Bidirectional Long-Short-Term-Memory (BiLSTM). The efficiency of the proposed model was calculated using dirty, bloody, cracked and robust egg images with the developed machine vision system. The experimental works showed that the proposed model achieved a 99.17% accuracy score. The obtained result was also compared with state-of-the-art methods, and the proposed model was observed to exhibit the highest accuracy among the compared methods.

Development and performance evaluation of a machine vision system and an integrated prototype for automated green shoot thinning in vineyards

AbstractGreen shoot thinning in vineyards is an essential, perennial operation for maintaining canopy health and optimizing yield and quality of wine grapes. Use of mechanized thinning system, which is essential to reduce labor dependency and associated cost, causes high variability in shoot removal efficiency due to difficulty in precisely positioning the thinning end‐effector along cordon trajectories. Automated/robotic solution for precise positioning of the thinning end‐effector could significantly improve the performance and efficiency of mechanical green shoot thinning. This study presents: (i) a machine vision‐based cordon detection system that can estimate cordon trajectories at different shoot growth stages in a vineyard; and (ii) evaluation of an integrated green shoot thinning system capable of automatically positioning the thinning end‐effector following vine cordon trajectories. The developed machine vision system uses deep learning‐based techniques that could accurately estimate cordon trajectories with root mean square error (RMSE) of 7.3, 10.3, and 16.1 pixels for canopy images captured in 2–4 weeks of shoot growth, respectively. Then, a control strategy was presented for the integrated system, which receives the computed cordon trajectories from machine vision system to automatically position the thinning end‐effector to cordon trajectories. Field evaluations in a research vineyard showed that the integrated system can achieve an RMSE of 1.47 cm in following the cordon trajectories at 6.6 cm·s−1forward speed. Future work will incorporate additional sensing system to detect individual shoot on cordon and integrating it with an existing system to achieve higher level of precision in green shoot thinning.

Machine vision system for high-temperature wind tunnel: Principle, design and application

In aerospace and related fields, simulation tests in the high-temperature wind tunnel have been playing an important role in evaluating and optimizing the thermal performance of key materials/structures of the aircraft. Based on the characteristics of the test environment, the development of machine vision system for the high-temperature wind tunnel can obtain the wide ranging and comprehensive process information of the tests, which is of great significance to the development of high-temperature ground simulation tests. Thus, this paper reviews the theoretical principles, key technologies and equipment design of the machine vision system for high-temperature wind tunnel. Meanwhile, several application examples are also introduced along with the relevant research results of our research group in this field. Finally, the development direction and application prospects of the machine vision system for the high-temperature wind tunnel are also forecasted with the future-oriented needs of the simulation tests.

A CCD based machine vision system for real-time text detection.

Text detection and recognition is a hot topic in computer vision, which is considered to be the further development of the traditional optical character recognition (OCR) technology. With the rapid development of machine vision system and the wide application of deep learning algorithms, text recognition has achieved excellent performance. In contrast, detecting text block from complex natural scenes is still a challenging task. At present, many advanced natural scene text detection algorithms have been proposed, but most of them run slow due to the complexity of the detection pipeline and cannot be applied to industrial scenes. In this paper, we proposed a CCD based machine vision system for real-time text detection in invoice images. In this system, we applied optimizations from several aspects including the optical system, the hardware architecture, and the deep learning algorithm to improve the speed performance of the machine vision system. The experimental data confirms that the optimization methods can significantly improve the running speed of the machine vision system and make it meeting the real-time text detection requirements in industrial scenarios.

Measurements of Tool Wear Parameters Using Machine Vision System

Monitoring tool wear is very important in machining industry as it may result in loss of dimensional accuracy and quality of finished product. This work includes the development of machine vision system for the direct measurement of flank wear of carbide cutting tool inserts. This system consists of a digital camera to capture the tool wear image, a good light source to illuminate the tool, and a computer for image processing. A new approach of inline automatic calibration of a pixel is proposed in this work. The captured images of carbide inserts are processed, and the segmented tool wear zone has been obtained by image processing. The vision system extracts tool wear parameters such as average tool wear width, tool wear area, and tool wear perimeter. The results of the average tool wear width obtained from the vision system are experimentally validated with those obtained from the digital microscope. An average error of 3% was found for measurements of all 12 carbide inserts. Scanning electron micrographs of the wear zone indicate the severe abrasion marks and damage to the cutting edge for higher machining time. This study indicates that the efficient and reliable vision system can be developed to measure the tool wear parameters.

A machine vision system for the braid angle measurement of tubular braided structures

Two-dimensional braided composite materials consist of an impregnated and cured braided fiber structure. The braid angle, which describes the orientation of fibers on the mandrel, affects the material properties of the composite part. Furthermore, braid production is often inaccurate, requiring quality assurance measurements of the fiber orientation. In this study, a frequency domain machine vision algorithm was applied to tubular braided preforms to measure fiber alignment during production. Several approaches to compensate for the tubular mandrel shape were tested and the fiber distribution in both the spatial and frequency domains was compared. The developed machine vision system allowed braid angle measurements to be made in real-time and in-line during the braiding process. The integration of such a system eliminates the need to make manual quality assurance measurements of the braided fiber structure, it further automates the braiding process, and it decreases production error.

Algorithm for automatic calibration of color vision system in foods

One of the important steps in development of machine vision system is calibration. In this study, a graphic user interface based program was developed and evaluated for calibration of color vision system. Algorithm is capable of processing user defined shades and generates calibration files. System was evaluated for calibration of color vision system to measure CIE L*a*b* color values of flavored milk and skim milk powder. The proposed method takes advantage of individual L*a*b* channel calibration using multiple regression analysis. Effect of image at different resolutions (0.3, 2, 8, 16 mega pixel) and calibration models (linear, quadratic, cubic and quartic) on color measurement was studied. Innovative trend analysis program for L*a*b* channel was useful in identifying and eliminating errors in early calibration steps. Image resolution of 8 mega pixel or above was found to be adequate to capture all graphical details for food colorimetric applications. The algorithm was successful in calibration of color vision system.

Development of Machine Vision System for Automatic Inspection of Vehicle Identification Number

Development of Machine Vision System for Automatic Inspection of Vehicle Identification Number

Development of machine vision system for automated mechanical objects recognition

Object recognition is a type of pattern recognition. Object recognition is widely used in the manufacturing industry for the purpose of inspection. Machine vision techniques are being applied in areas ranging from medical imaging to remote sensing, industrial inspection to document processing, and nanotechnology to multimedia databases. In this paper, mechanical objects recognition used in manufacturing industry is taken into consideration. Mechanically manufactured parts have recognition difficulties due to manufacturing process including machine malfunctioning, tool wear, and variations in raw material. This paper considers the problem of recognising and classifying of such objects. These objects are kept in different orientations for invariant rotation, translation and scaling. The features are extracted using Fourier descriptor technique. Artificial neural network (ANN) with back-propagation learning algorithm is used to train the network and for classification of five different objects. This paper shows the effect of learning rate and momentum on the classification accuracy of objects.

Development of machine vision system for automated mechanical objects recognition

Development of machine vision system for automated mechanical objects recognition

Development of machine-vision system for gap inspection of muskmelon grafted seedlings.

Grafting robots have been developed in the world, but some auxiliary works such as gap-inspecting for grafted seedlings still need to be done by human. An machine-vision system of gap inspection for grafted muskmelon seedlings was developed in this study. The image acquiring system consists of a CCD camera, a lens and a front white lighting source. The image of inspected gap was processed and analyzed by software of HALCON 12.0. The recognition algorithm for the system is based on principle of deformable template matching. A template should be created from an image of qualified grafted seedling gap. Then the gap image of the grafted seedling will be compared with the created template to determine their matching degree. Based on the similarity between the gap image of grafted seedling and the template, the matching degree will be 0 to 1. The less similar for the grafted seedling gap with the template the smaller of matching degree. Thirdly, the gap will be output as qualified or unqualified. If the matching degree of grafted seedling gap and the template is less than 0.58, or there is no match is found, the gap will be judged as unqualified; otherwise the gap will be qualified. Finally, 100 muskmelon seedlings were grafted and inspected to test the gap inspection system. Results showed that the gap inspection machine-vision system could recognize the gap qualification correctly as 98% of human vision. And the inspection speed of this system can reach 15 seedlings·min-1. The gap inspection process in grafting can be fully automated with this developed machine-vision system, and the gap inspection system will be a key step of a fully-automatic grafting robots.

Surface Defects Detection of Red Jujube Based on Near-Infrared Vision System

Quality evaluation of agricultural and food products is important for processing, inventory control, and marketing. Fruit surface defects are important quality factors for the jujube industry, especially for high quality jujubes such as Xinjiang red jujube. This paper presents the development and test results of a machine vision system for automatic jujube surface defects detection. Unlike other near-infrared spectrometric approaches, the developed machine vision system uses reflective near-infrared image to evaluate jujube quality by analyzing two-dimensional images. Near-infrared image, vision algorithms and a variety of operational details of the system, including cameras, optics, illumination, and fruit carrier are presented. The complete machine vision system has been built, and the experimental results show that the designed machine vision system is feasible to detect the defects of jujubes.

In-Process Vision Inspection Systems for Sorting using Image Processing Techniques

Problem statement: Machine vision system is based on digital image processing and is found to be the best sensor detection as its operation is similar to the human eye. The purpose of machine vision is the desire to provide real time machines with visual abilities. Approach: A real time system is developed and is interfaced with the mechanical structure to be used in automobile industry. Dynamic thersholding is used and image pre-processing techniques are considered and implemented namely ellipse and circle algorithm. Results: The developed machine vision system consists of a mechanical structure, DIO card for communication and pneumatic components. Conclusion: The proposed model is tested on real time for segregation and mix up of automobile nut samples.

Real-time hyperspectral processing for automatic nonferrous material sorting

The application of hyperspectral sensors in the develop- ment of machine vision solutions has become increasingly popular as the spectral characteristics of the imaged materials are better mod- eled in the hyperspectral domain than in the standard trichromatic red, green, blue data. While there is no doubt that the availability of detailed spectral information is opportune as it opens the possibility to construct robust image descriptors, it also raises a substantial chal- lenge when this high-dimensional data is used in the development of real-time machine vision systems. To alleviate the computational demand, often decorrelation techniques are commonly applied prior to feature extraction. While this approach has reduced to some extent the size of the spectral descriptor, data decorrelation alone proved insufficient in attaining real-time classification. This fact is particularly apparent when pixel-wise image descriptors are not sufficiently robust to model the spectral characteristics of the imaged materials, a case when the spatial information (or textural properties) also has to be included in the classification process. The integration of spectral and spatial information entails a substantial computational cost, and as a result the prospects of real-time opera- tion for the developed machine vision system are compromised. To answer this requirement, in this paper we have reengineered the approach behind the integration of the spectral and spatial information in the material classification process to allow the real-time sorting of the nonferrous fractions that are contained in the waste of electric and electronic equipment scrap. © 2012 SPIE and IS&T. (DOI: 10.1117/ 1.JEI.21.1.013018)