Automated Vision System Research Articles

Implementación de un sistema automatizado de clasificación de Solanum Tuberosum mediante procesamiento de imágenes

Automated vision systems are used as an effective solution to improve accuracy in objects inspection. However, the transition from manual sorting methods to automated systems has been slow due to factors such as the size and shape of the objects to be inspected. In the food industry, tubers represent a challenge for vision systems due to their high dimensional and morphological variability. This paper presents the design and implementation of an automated system for sorting Solanum tuberosum (potato) based on machine vision and image processing technologies. The system employs a low-cost camera together with an algorithm developed in LabVIEWTM, which allows determining the size of tubers in a production line. Sorting is carried out by means of a servo-driven sorting mechanism, achieving greater accuracy and efficiency compared to traditional manual methods. A controlled lighting system was implemented to optimize the quality of the captured images, which allowed for a significant reduction in sorting errors. The results showed a 260% increase in production capacity and a 75% reduction in the error rate, validating the effectiveness of the proposed solution to improve both productivity and process quality in the food industry. In addition, the system offers a high level of flexibility and safety in operation. By improving potato grading efficiency and reducing the need for manual intervention, a return on investment is anticipated, as well as a positive impact on process responsiveness, which could lead to an increase in product demand.

A relaxor ferroelectric crystal based Two-DOF miniature piezoelectric motor with fish body structure

Multi-dimensional movements and miniaturization are required for robotic joints motion, multi-axis optical alignments and other multiple degree-of-freedom (DOF) actuation scenes. However, it is a challenge for piezoelectric motors to guarantee multi-DOF movements, miniaturization, high load capacity and high resolution simultaneously. Inspired by synergic movement of body and caudal fin of fish in nature and the construction idea of metamaterials, we proposed a two-DOF linear piezoelectric motor (4mmHeight × 3mmLength × 3mmWidth) based on the artificial elongation 31z and shear 35x/y mode by using Pb(In1/2Nb1/2)O3–Pb(Mg1/3Nb2/3)O3–PbTiO3 (PIMNT) relaxor ferroelectric single crystal as piezoelectric elements. The results demonstrate that the motor is capable of achieving bidirectional motion in both X and Y axes, with the PIMNT crystal-based motor reaching a velocity of up to 94.8 mm/s. Its mechanical loading capacity per unit volume reaches as high as 1.39 mN/mm3, which exceeds the load capacity of currently reported two-DOF motors by 10 times. Moreover, it exhibits an exceptional step resolution of 5 nm, outperforming existing two-DOF motors by 1–2 orders of magnitude in terms of precision and accuracy. Furthermore, by modulating the driving signals of this motor, it can traverse any trajectory within the XY plane. The proposed ultrasonic motor has potential for applications in micro-robots, nano-precision motion, automatic visual systems and haptic interfaces.

Vision Inspection Method for the Quality Assessment of Paint Coatings on Glassware.

Image analysis is becoming increasingly popular in many industries. Its use is perfect for, among other things, assessing the quality of products on or off the production line. Highly automated, high-performance systems can be used for this purpose. However, there are situations in which automated vision systems cannot be used on the production line due to the specific nature of the process. One such situation is testing the resistance of paint applied to glass when washing in automatic dishwashers. It is carried out outside the production line, and typical production vision systems are not used here. An attempt was made to develop a cheap and easy-to-implement research method enabling quantitative measurement of paint loss on glass when testing the coating's resistance to automatic washing. For this purpose, analysis of images taken during the study was carried out. The developed method is based on taking a series of photos of the tested object between each stage of the wash resistance test. The obtained photographic material is then analyzed by measuring the size of paint losses expressed in the number of pixels. Then, the percentage of paint loss is calculated. This method is cheap to implement and highly accurate. Statistical analysis of the results confirmed the method's accuracy at 98%.

Sliding mode observer based fault identification in automatic vision system of robot

It is analyzed the possibility to apply sliding mode observers for fault identification and consecutively the improvement of Technical Vision System (TVS) functioning in the tasks of mobile robotics. It is important that in swarm robotics tasks, where the Machine Vision is based on collaborative scanning of frame by several TVSs, the value of each single fault is multiplied. The original approach to system modeling permits us to build the sliding mode observer of reduced order, and apply its estimation for control improvement of scanning aperture DC motor in TVS. It is shown by simulations that estimation of scanner’s motor instability is robust and not sensitive to many practical perturbations.

A Low-Cost Video-Based System for Neurodegenerative Disease Detection by Mobility Test Analysis

The observation of mobility tests can greatly help neurodegenerative disease diagnosis. In particular, among the different mobility protocols, the sit-to-stand (StS) test has been recognized as very significant as its execution, both in terms of duration and postural evaluation, can indicate the presence of neurodegenerative diseases and their advancement level. The assessment of an StS test is usually done by physicians or specialized physiotherapists who observe the test and evaluate the execution. Thus, it mainly depends on the experience and expertise of the medical staff. In this paper, we propose an automatic visual system, based on a low-cost camera, that can be used to support medical staff for neurodegenerative disease diagnosis and also to support mobility evaluation processes in telehealthcare contexts. The visual system observes people while performing an StS test, then the recorded videos are processed to extract relevant features based on skeleton joints. Several machine learning approaches were applied and compared in order to distinguish people with neurodegenerative diseases from healthy subjects. Real experiments were carried out in two nursing homes. In light of these experiments, we propose the use of a quadratic SVM, which outperformed the other methods. The obtained results were promising. The designed system reached an accuracy of 95.2% demonstrating its effectiveness.

The surrogate labor of the eye: Farocki, Papa, and the eeefff collective

ABSTRACT The following essay explores the work of art as a site of encounter with human perceptual labor that plays a role in technical operations. It tackles the way such labor is deemed obsolete, soon to be replaced, and therefore surrogate even if it actually animates and reproduces automated vision systems. It explores how art goes about representing the ways in which such labor is undervalued and unrecognized. The text argues for reading in between the lines and images of Harun Farocki’s films, installations, and writings where the obsolescence of human labor emerges more as an ideological screen than a fact. It focuses on moments in his oeuvre which indicate that human labor, including cognition as automation’s last frontier, is not automated away but persists, changes site, undergoes restructuring, and becomes more hidden. More recent works by the eeefff collective and Elisa Giardina Papa explore the intertwined roles of human affection and vision labor in the necessarily failed attempts to teach machines to see and feel, to “clean” the algorithmic vision and affection from opacity and the queerness of real life. Both artists leave behind Farocki’s self-reflexive, detached spectator to involve the audience in more situated and embodied experiences of perception labor and the particular ways in which such labor has become outsourced and dispersed in semi-peripheries such as Sicily or Belarus. They try to express the price that people pay with their emotions and bodies for such work. Yet, in principle, they follow Farocki’s take on labor’s in/visibility in that they challenge the ruling ideologies that blind human vision to the realities of labor. The essay also pays attention to the ways in which both artistic and technical vision today are pre-determined by the logic of the gig economy.

Inline Quality Monitoring of Reverse Extruded Aluminum Parts with Cathodic Dip-Paint Coating (KTL).

Perfectly coated surfaces are an essential quality feature in the automotive and consumer goods industries. They are the result of an optimized, controlled coating process. Because entire assemblies could be rejected if Out-of-Specification (OOS) parts are installed, this has a severe economic impact. This paper presents a novel, line-integrated multi-camera system with intelligent algorithms for anomaly detection on small KTL-coated aluminum parts. The system also aims to automatize the previously used human inspection to a sophisticated and automated vision system that efficiently detects defects and anomalies on coated parts.

Strong semantic segmentation for Covid-19 detection: Evaluating the use of deep learning models as a performant tool in radiography

IntroductionWith the increasing number of Covid-19 cases as well as care costs, chest diseases have gained increasing interest in several communities, particularly in medical and computer vision. Clinical and analytical exams are widely recognized techniques for diagnosing and handling Covid-19 cases. However, strong detection tools can help avoid damage to chest tissues. The proposed method provides an important way to enhance the semantic segmentation process using combined potential deep learning (DL) modules to increase consistency. Based on Covid-19 CT images, this work hypothesized that a novel model for semantic segmentation might be able to extract definite graphical features of Covid-19 and afford an accurate clinical diagnosis while optimizing the classical test and saving time. MethodsCT images were collected considering different cases (normal chest CT, pneumonia, typical viral causes, and Covid-19 cases). The study presents an advanced DL method to deal with chest semantic segmentation issues. The approach employs a modified version of the U-net to enable and support Covid-19 detection from the studied images. ResultsThe validation tests demonstrated competitive results with important performance rates: Precision (90.96% ± 2.5) with an F-score of (91.08% ± 3.2), an accuracy of (93.37% ± 1.2), a sensitivity of (96.88% ± 2.8) and a specificity of (96.91% ± 2.3). In addition, the visual segmentation results are very close to the Ground truth. ConclusionThe findings of this study reveal the proof-of-principle for using cooperative components to strengthen the semantic segmentation modules for effective and truthful Covid-19 diagnosis. Implications for practiceThis paper has highlighted that DL based approach, with several modules, may be contributing to provide strong support for radiographers and physicians, and that further use of DL is required to design and implement performant automated vision systems to detect chest diseases.

Graphical Image Region Extraction with K-Means Clustering and Watershed.

With a wide range of applications, image segmentation is a complex and difficult preprocessing step that plays an important role in automatic visual systems, which accuracy impacts, not only on segmentation results, but directly affects the effectiveness of the follow-up tasks. Despite the many advances achieved in the last decades, image segmentation remains a challenging problem, particularly, the segmenting of color images due to the diverse inhomogeneities of color, textures and shapes present in the descriptive features of the images. In trademark graphic images segmentation, beyond these difficulties, we must also take into account the high noise and low resolution, which are often present. Trademark graphic images can also be very heterogeneous with regard to the elements that make them up, which can be overlapping and with varying lighting conditions. Due to the immense variation encountered in corporate logos and trademark graphic images, it is often difficult to select a single method for extracting relevant image regions in a way that produces satisfactory results. Many of the hybrid approaches that integrate the Watershed and K-Means algorithms involve processing very high quality and visually similar images, such as medical images, meaning that either approach can be tweaked to work on images that follow a certain pattern. Trademark images are totally different from each other and are usually fully colored. Our system solves this difficulty given it is a generalized implementation designed to work in most scenarios, through the use of customizable parameters and completely unbiased for an image type. In this paper, we propose a hybrid approach to Image Region Extraction that focuses on automated region proposal and segmentation techniques. In particular, we analyze popular techniques such as K-Means Clustering and Watershedding and their effectiveness when deployed in a hybrid environment to be applied to a highly variable dataset. The proposed system consists of a multi-stage algorithm that takes as input an RGB image and produces multiple outputs, corresponding to the extracted regions. After preprocessing steps, a K-Means function with random initial centroids and a user-defined value for k is executed over the RGB image, generating a gray-scale segmented image, to which a threshold method is applied to generate a binary mask, containing the necessary information to generate a distance map. Then, the Watershed function is performed over the distance map, using the markers defined by the Connected Component Analysis function that labels regions on 8-way pixel connectivity, ensuring that all regions are correctly found. Finally, individual objects are labelled for extraction through a contour method, based on border following. The achieved results show adequate region extraction capabilities when processing graphical images from different datasets, where the system correctly distinguishes the most relevant visual elements of images with minimal tweaking.

Developing an Automated Vision System for Maintaing Social Distancing to Cure the Pandemic

The world is currently facing a medical crisis. The epidemic has affected millions of people around the world since its appearance. This situation needs an urgent solution. Most countries have used different solutions to stop the spread of the epidemic. The World Health Organization has imposed some rules that people should adhere. The rules are such, wearing masks, quarantining infected people and social distancing. Social distancing is one of the most important solutions that have given good results to confront the emerging virus. Several systems have been developed that use artificial intelligence and deep learning to track social distancing. In this study, a system based on deep learning has been proposed. The system includes monitoring and detecting people besides measuring the social distance between them. The proposed system consists of two parts: (1) detecting the faces of people using the Viola-Jones algorithm. The Cascade classifiers were trained. The Cascade classifiers used in the algorithm with feature descriptors to detect side faces and wear masks. Hence, training is dominant for detection. (2) measurement of the Euclidean distance between the centers of the rectangles of the people who were revealed in the first part. The distance between individuals' is measured to check how well they adhere to social distancing. The results revealed that the proposed system can perform well in applying images to track the distance between people.

Situational Awareness: Techniques, Challenges, and Prospects

Situational awareness (SA) is defined as the perception of entities in the environment, comprehension of their meaning, and projection of their status in near future. From an Air Force perspective, SA refers to the capability to comprehend and project the current and future disposition of red and blue aircraft and surface threats within an airspace. In this article, we propose a model for SA and dynamic decision-making that incorporates artificial intelligence and dynamic data-driven application systems to adapt measurements and resources in accordance with changing situations. We discuss measurement of SA and the challenges associated with quantification of SA. We then elaborate a plethora of techniques and technologies that help improve SA ranging from different modes of intelligence gathering to artificial intelligence to automated vision systems. We then present different application domains of SA including battlefield, gray zone warfare, military- and air-base, homeland security and defense, and critical infrastructure. Finally, we conclude the article with insights into the future of SA.

Vision-Based System for Automated Estimation of the Frontal Area of Swimmers: Towards the Determination of the Instant Active Drag: A Pilot Study.

Swimmers take great advantage by reducing the drag forces either in passive or active conditions. The purpose of this work is to determine the frontal area of swimmers by means of an automated vision system. The proposed algorithm is automated and also allows to determine lateral pose of the swimmer for training purposes. In this way, a step towards the determination of the instantaneous active drag is reached that could be obtained by correlating the effective frontal area of the swimmer to the velocity. This article shows a novel algorithm for estimating the frontal and lateral area in comparison with other models. The computing time allows to obtain a reasonable online representation of the results. The development of an automated method to obtain the frontal surface area during swimming increases the knowledge of the temporal fluctuation of the frontal surface area in swimming. It would allow the best monitoring of a swimmer in their swimming training sessions. Further works will present the complete device, which allows to track the swimmer while acquiring the images and a more realistic model of conventional active drag ones.

Denoising and Dehazing an Image in a Cascaded Pattern for Continuous Casting

Automatic vision systems have been widely used in the continuous casting of the steel industry, which improve efficiency and reduce labor. At present, high temperatures with evaporating fog cause images to be noisy and hazy, impeding the usage of advanced machine learning algorithms in this task. Instead of considering denoising and dehazing separately like previous papers, we established that by taking advantage of deep learning in a modeling complex formulation, our proposed algorithm, called Cascaded Denoising and Dehazing Net (CDDNet) reduces noise and hazy in a cascading pattern. Experimental results on both synthesized images and a pragmatic video from a continuous casting factory demonstrate our method’s superior performance in various metrics. Compared with existing methods, CDDNet achieved a 50% improvement in terms of peak signal-to-noise ratio on the validation dataset, and a nearly 5% improvement on a dataset that has never seen before. Besides, our model generalizes so well that processing a video from an operating continuous casting factory with CDDNet resulted in high visual quality.

Automated Vision Systems for Condition Assessment of Sewer and Water Pipelines

Sewer networks and water distribution systems are among the most valuable and critical urban assets for a community. These systems have their respective time span to provide continuous service to the residents. Hence, it is paramount to assess the condition of sewer and water pipelines to ensure sustainable, reliable, and cost-effective transportation of sewerage and water supply. The assessment is usually done by the inspection robots, which are equipped with machine vision systems and/or sensors. The inspection robots acquire the inspection data, and the operators then conduct the survey of the captured video and/or sensory data to interpret the results. Nowadays, automated solutions are being adopted by industry to achieve an accurate and efficient assessment. This article surveys the state of the art of the automated vision systems, which are employed for condition assessment of sewer and water pipelines, and identifies the challenges for future research. The following areas are highlighted in this survey: 1) the typical types of the faults and failures, which include the concept and definition of the sewer and water pipelines; 2) the inspection systems, e.g., robotic platforms for sewer and water pipeline inspection; 3) the machine vision systems for fault detection; 4) the computational frameworks for condition assessment; and 5) the challenges and suggestions for future research. This article summarizes the current state of automation in the machine vision technology for condition assessment (both hardware and software perspectives) of sewer and water pipelines and also provides a reference for researchers to further advance the technology in this field. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The motivation behind this article was the current challenges and the open issues of the vision system for the condition assessment of sewer and water pipelines. This article presents detailed reviews of the state-of-the-art of the vision system (both hardware and software) and discusses the existing problems. This survey aims to help engineers and researchers to resolve and improve the problems and extend the field of the existing automated frameworks. This article is organized in the form of a survey so that researchers can benefit and get all the useful information at a glance. This article provides a rigorous overview of typical faults, inspection robots, visual techniques, and automated frameworks for the condition assessment and discusses the challenges and future research directions. The objective of this article is to create a baseline for the readers to acquaint themselves with the state of the art and advance the research in this field.

Multiple Object Recognition Using OpenCV

For automatic vision systems used in agriculture, the project presents object characteristics analysis using image processing techniques. In agriculture science, automatic object characteristics identification is important for monitoring vast areas of crops, and it detects signs of object characteristics as soon as it occurs on plant leaves. Image content characterization and supervised classifier type neural network are used in the proposed deciding method. Pre-processing, image segmentation, and detection are some of the image processing methods used in this form of decision making. An image data will be rearranged and, if necessary, a region of interest will be selected during preparation. For network training and classification, colour and texture features are extracted from an input. Colour characteristics such as mean and variance in the HSV colour space, as well as texture characteristics such as energy, contrast, homogeneity, and correlation. The device will be trained to automatically identify test images in order to assess object characteristics. With some training samples of that type, an automated classifier NN could be used for classification supported learning in this method. The tangent sigmoid function is used as the kernel function in this network. Finally, the simulated results show that the used network classifier has a low error rate during training and higher classification accuracy. In the previous researches Object detection has been made possible, but in our current research we have attempted to do live Object Detection using OpenCV and also the techniques involved in it.

An explainable deep vision system for animal classification and detection in trail-camera images with automatic post-deployment retraining

This paper introduces an automated vision system for animal detection in trail-camera images taken from a field under the administration of the Texas Parks and Wildlife Department. As traditional wildlife counting techniques are intrusive and labor intensive to conduct, trail-camera imaging is a comparatively non-intrusive method for capturing wildlife activity. However, given the large volume of images produced from trail-cameras, manual analysis of the images remains time-consuming and inefficient. We implemented a two-stage deep convolutional neural network pipeline to find animal-containing images in the first stage and then process these images to detect birds in the second stage. The animal classification system classifies animal images with overall 93% sensitivity and 96% specificity. The bird detection system achieves better than 93% sensitivity, 92% specificity, and 68% average Intersection-over-Union rate. The entire pipeline processes an image in less than 0.5 s as opposed to an average 30 s for a human labeler. We also addressed post-deployment issues related to data drift for the animal classification system as image features vary with seasonal changes. This system utilizes an automatic retraining algorithm to detect data drift and update the system. We introduce a novel technique for detecting drifted images and triggering the retraining procedure. Two statistical experiments are also presented to explain the prediction behavior of the animal classification system. These experiments investigate the cues that steers the system towards a particular decision. Statistical hypothesis testing demonstrates that the presence of an animal in the input image significantly contributes to the system’s decisions.

Automated vision system for magnetic particle inspection of crankshafts using convolutional neural networks

This paper proposes a fully automated vision system to inspect the whole surface of crankshafts, based on the magnetic particle testing technique. A stepper motor combined with multiple cameras is needed to ensure the inspection of the whole surface of the crankshaft in real-time. Due to the very textured surface of crankshafts and the variability in defect shapes and types, defect detection methods based on deep learning algorithms, more precisely convolutional neural networks (CNNs), become a more efficient solution than traditional methods. This paper discusses the various approaches of defect detection with CNNs, mainly classification, object detection, and semantic segmentation. The advantages and weaknesses of each approach for real-time defect detection are presented. It is important to note that the proposed visual inspection system only replaces the manual inspection of crankshafts conducted by operators at the end of the magnetic particle testing procedure, allowing for an easy integration in any crankshaft factory.

Non-Destructive Quality Inspection of Potato Tubers Using Automated Vision System

In this investigation, an automated vision system "AVS" for non-destructive quality inspection of potato tubers "PT" was developed. Color, size, mass, firmness, and the texture homogeneity of the "PT" surface, various sensitive features were studied, and extracted from the digital image by using the R program. Otsu threshold method, RGB, Lu*v*, CIE LChuv color models, and texture analysis by using the package Gray-Level Co-Occurrence Matrices (GLCMs) were applied. The results showed a great correlation between the tuber pixel area percentages (DIM=dimension as a percentage of total pixels), and both mass and geometric mean diameter (GMD) of all "PT" varieties. The color results demonstrated that the hue angle (huv) ranged from 68.92 to 96.61°, and the "PT" color was classified into deep and light color intensity. The "AVS" could predict the mass and size, and gave statistical data at the mass production level, in terms of the inspecting samples No., mass, and grades based on size, color, and free from injuries through the texture homogeneity of tuber surface. A predictive model hypothesized based on the tuber's surface texture characteristics for predicting the tubers firmness was statistically significant. This "AVS" can be applied as a non-destructive, precise, and symmetric technique in-line inspection, the quality of "PT", also helping decision-makers in the agricultural field and stakeholders to improve the horticulture sector through the statistical data issued by this system.

Robotic 3D Vision-Guided System for Half-Sheep Cutting Robot

Sheep body segmentation robot can improve production hygiene, product quality, and cutting accuracy, which is a huge change for traditional manual segmentation. With reference to the New Zealand sheep body segmentation specification, a vision system for Cartesian coordinate robot cutting half-sheep was developed and tested. The workflow of the vision system was designed and the image acquisition device with an Azure Kinect sensor was developed. Furthermore, a LabVIEW software with the image processing algorithm was then integrated with the RGBD image acquisition device in order to construct an automatic vision system. Based on Deeplab v3+ networks, an image processing system for locating ribs and spine was employed. Taking advantage of the location characteristics of ribs and spine in the split half-sheep, a calculation method of cutting line based on the key points is designed to determine five cutting curves. The seven key points are located by convex points of ribs and spine and the root of hind leg. Using the conversion relation between depth image and the space coordinates, the 3D coordinates of the curves were computed. Finally, the kinematics equation of the rectangular coordinate robot arm is established, and the 3D coordinates of the curves are converted into the corresponding motion parameters of the robot arm. The experimental results indicated that the automatic vision system had a success rate of 98.4% in the cutting curves location, 4.2 s time consumption per half-sheep, and approximately 1.3 mm location error. The positioning accuracy and speed of the vision system can meet the requirements of the sheep cutting production line. The vision system shows that there is potential to automate even the most challenging processing operations currently carried out manually by human operators.

Vision guided robotic inspection for parts in manufacturing and remanufacturing industry

Environmental and commercial drivers are leading to a circular economy, where systems and components are routinely recycled or remanufactured. Unlike traditional manufacturing, where components typically have a high degree of tolerance, components in the remanufacturing process may have seen decades of wear, resulting in a wider variation of geometries. This makes it difficult to translate existing automation techniques to perform Non-Destructive Testing (NDT) for such components autonomously. The challenge of performing automated inspections, with off-line tool-paths developed from Computer Aided Design (CAD) models, typically arises from the fact that those paths do not have the required level of accuracy. Beside the fact that CAD models are less available for old parts, these parts often differ from their respective virtual models. This paper considers flexible automation by combining part geometry reconstruction with ultrasonic tool-path generation, to perform Ultrasonic NDT. This paper presents an approach to perform custom vision-guided ultrasonic inspection of components, which is achieved through integrating an automated vision system and a purposely developed graphic user interface with a robotic work-cell. The vision system, based on structure from motion, allows creating 3D models of the parts. Also, this work compares four different tool-paths for optimum image capture. The resulting optimum 3D models are used in a virtual twin environment of the robotic inspection cell, to enable the user to select any points of interest for ultrasonic inspection. This removes the need of offline robot path-planning and part orientation for assessing specific locations on a part, which is typically a very time-consuming phase.