Machine Vision Camera Research Articles

Quality Control of PET Bottles Caps with Dedicated Image Calibration and Deep Neural Networks.

Product quality control is currently the leading trend in industrial production. It is heading towards the exact analysis of each product before reaching the end customer. Every stage of production control is of particular importance in the food and pharmaceutical industries, where, apart from visual issues, additional safety regulations are demanded. Many production processes can be controlled completely contactless through the use of machine vision cameras and advanced image processing techniques. The most dynamically growing sector of image analysis methods are solutions based on deep neural networks. Their major advantages are fast performance, robustness, and the fact that they can be exploited even in complicated classification problems. However, the use of machine learning methods on high-performance production lines may be limited by inference time or, in the case of multiformated production lines, training time. The article presents a novel data preprocessing (or calibration) method. It uses prior knowledge about the optical system, which enables the use of the lightweight Convolutional Neural Network (CNN) model for product quality control of polyethylene terephthalate (PET) bottle caps. The combination of preprocessing with the lightweight CNN model resulted in at least a five-fold reduction in prediction and training time compared to the lighter standard models tested on ImageNet, without loss of accuracy.

Low cost machine vision camera using raspberry pi

Low Cost machine Vision Camera is nothing but designing a machine vision camera with raspberry Pi High Quality Camera and also with Raspberry Pi 4 which is a Low Cost product with high efficiency. This project is focused at developing a Machine Vision camera using a combination of low-cost High Quality Camera hardware and computer algorithms to process the image taken from camera and to show the result whether the product is good or not good. The Keras.h5 model algorithm consists of positive and negative images and with the help of developed program we shall be able to calculate whether it is a good or not good product. The algorithm is based on colour difference, shape difference and coordinate differences.

Calibration of In-Plane Center Alignment Errors in the Installation of a Circular Slide with Machine-Vision Sensor and a Reflective Marker

This paper describes a method for calibrating in-plane center alignment error (IPCA) that occurs when installing the circular motion slide (CMS). In this study, by combini ng the moving carriage of the CMS and the planar PKM (parallel kinematic mechanism) with the machine tool, the small workspace of the PKM is expanded, and the workpiece is placed on the table with the CMS installed is processed through the machine tool. However, to rigidly mount the CMS on the table, the preload between the guide and the support bearings must be adjusted with the eccentric bearing, and in this process, the IPCA occurs. After installing a reflective marker on the PKM, the PKM is slowly rotated along with the ring guide in the way of stop-and-go without the PKM’s own motion. Then, using a machine vision camera installed at the top of the CMS, the IPCA, which is the difference between the actual center position and the nominal center position of the CMS with respect to the camera, can be successfully calibrated through the circular fitting process. Consequently, it was confirmed that the IPCA of 0.37 mm can be successfully identified with the proposed method.

Autonomous Crop Row Guidance Using Adaptive Multi-ROI in Strawberry Fields

Automated robotic platforms are an important part of precision agriculture solutions for sustainable food production. Agri-robots require robust and accurate guidance systems in order to navigate between crops and to and from their base station. Onboard sensors such as machine vision cameras offer a flexible guidance alternative to more expensive solutions for structured environments such as scanning lidar or RTK-GNSS. The main challenges for visual crop row guidance are the dramatic differences in appearance of crops between farms and throughout the season and the variations in crop spacing and contours of the crop rows. Here we present a visual guidance pipeline for an agri-robot operating in strawberry fields in Norway that is based on semantic segmentation with a convolution neural network (CNN) to segment input RGB images into crop and not-crop (i.e., drivable terrain) regions. To handle the uneven contours of crop rows in Norway’s hilly agricultural regions, we develop a new adaptive multi-ROI method for fitting trajectories to the drivable regions. We test our approach in open-loop trials with a real agri-robot operating in the field and show that our approach compares favourably to other traditional guidance approaches.

Automated markerless pose estimation in freely moving macaques with OpenMonkeyStudio

The rhesus macaque is an important model species in several branches of science, including neuroscience, psychology, ethology, and medicine. The utility of the macaque model would be greatly enhanced by the ability to precisely measure behavior in freely moving conditions. Existing approaches do not provide sufficient tracking. Here, we describe OpenMonkeyStudio, a deep learning-based markerless motion capture system for estimating 3D pose in freely moving macaques in large unconstrained environments. Our system makes use of 62 machine vision cameras that encircle an open 2.45 m × 2.45 m × 2.75 m enclosure. The resulting multiview image streams allow for data augmentation via 3D-reconstruction of annotated images to train a robust view-invariant deep neural network. This view invariance represents an important advance over previous markerless 2D tracking approaches, and allows fully automatic pose inference on unconstrained natural motion. We show that OpenMonkeyStudio can be used to accurately recognize actions and track social interactions.

REAL-TIME SLAM FOR THE OFF-ROAD AUTONOMOUS DRIVING

Abstract. In this paper we propose a new SLAM algorithm that is robust to the changing environment of the countryside. The hardware part consists of two separate machine vision cameras, joined in stereo, and can be supplemented with LiDAR, IMU and GPS. We introduce a method that can be used to reliably calculate the position of a vehicle in natural environments. To estimate the pose and produce a three-dimensional reconstruction we use a stereo camera rig, inertial measurement unit and the global positioning system. While solving the problem of visual odometry in outdoor scenes we faced a number of difficulties, arising from high dynamic range, as well as the presence of a large number of "similar elements", such as leaves, grass, trees and etc. Under these conditions, it becomes difficult to match feature points in image sequences. HOG-based methods, such as SIFT, SURF and others often do not obtain good matching due to noise, lack of a sufficient number of gradients, and the presence of identical domains. Using neighborhood-based detectors such as DAISY often allows to identify the correct matches, but using them is worth it too expensive. These methods are very demanding on the computational resources and prone to drift. We needed a method that is less expensive, but at the same time provides sufficient accuracy in the trajectory estimation. Direct methods, such as optical flow calculating or direct image matching allow us to map point-to-point in these conditions with high reliability. They also have disadvantages that can be eliminated by using an IMU and modern algorithms. To improve the quality of the algorithms, we solve the reconstruction problem for several frames using the Levenberg-Marquardt optimization method for bundle adjustment. Each pass optimizes frames that are directly related to the last one, we use two threads that perform partial and full optimization of the entire trajectory using graphs to significantly increase the performance of the method.

DEVELOPING AN IMAGE BASED LOW-COST MOBILE MAPPING SYSTEM FOR GIS DATA ACQUISITION

Abstract. The aim of this study is to develop a low-cost mobile mapping system (MMS) with the integration of vehicle-based navigation data and stereo images acquired along vehicle paths. The system consists of a dual frequency GNSS board combined with a low-cost INS unit and two machine vision cameras that collect colour image data for road and roadside objects. The navigation data and the image acquisition are properly synchronized to associate position and attitude to each digital frame captured. In this way, upon pixel location of objects appearing on the video frames, their absolute geographical coordinates can be extracted by employing standard photogrammetric methods. Several calibration steps are implemented before survey operation: camera calibration, relative orientation between cameras and determination of rotation angles and offsets between vehicle and cameras reference frames. A software tool has been developed to facilitate and speed up the calibration procedures. Furthermore, easy object coordinate extraction is supported, either in auto mode, where the conjugate image coordinates are obtained in real time using image correlation techniques. Several surveying experiments were executed to certify and check the accuracy and efficiency of the system. From the achieved results, the developed system is efficient for collecting and positioning road spatial objects such as such as road boundaries, traffic lights, road signs, power poles, etc, more rapidly and less expensively. The obtained absolute positional accuracy is less than 1 meter, depending on the availability and quality of the GPS signal.

РАЗРАБОТКА АВТОМАТИЗИРОВАННОГО РАБОЧЕГО МЕСТА МЕТРОЛОГА ДЛЯ ПАО «КОРПОРАЦИЯ «ИРКУТ»

An approach to automate the process of verification of electrical measuring instruments at PJSC «Irkut» Corporation» using machine vision cameras is proposed

Photoplethysmography imaging:camera performance evaluation by means of an optoelectronic skin perfusion phantom

Objective: Photoplethysmography imaging (PPGI) is a promising contactless camera-based method of non-invasive cardiovascular diagnostics. To achieve the best results, it is important to choose the most suitable camera for a specific application. The settings of the camera influence the quality of the detected signal. Approach: The standard (European Machine Vision Association 2016 EMVA Standard 1288—Standard for Characterization of Image Sensors and Cameras pp 1–39 (available at: https://www.emva.org/wp-content/uploads/EMVA1288-3.1a.pdf)) for evaluating the imaging performance of machine vision cameras (MVC) helps at the initial decision of the sensor, but the camera should always be tested in terms of usability for a specific application. So far, PPGI lacks a standardized measurement scenario for evaluating the performance of individual cameras. For this, we realized a controllable optoelectronic phantom with artificial silicone skin allowing reproducible tests of cameras to verify their suitability for PPGI. The entire system is housed in a light-tight box. We tested an MVC, a digital single-lens reflex camera (DSLR) camera and a webcam. Each camera varies in used technology and price. Main results: We simulated real PPGI measurement conditions simulating the ratio of pulse (AC) and non-pulse (DC) component of the photoplethysmographic signal and achieved AC/DC ratios of 0.5 % on average. An additional OLED panel ensures proper DC providing reproducible measurement conditions. We evaluated the signal morphological features, amplitude spectrum, signal-to-noise ratio (SNR) and spatially dependent changes of simulated subcutaneous perfusion. Here, the MVC proved to be the most suitable device. A DSLR is also suitable for PPGI, but a larger smoothing kernel is required to obtain a perfusion map. The webcam, as the weakest contender, proved to be very susceptible to any inhomogeneous illumination of the examined artificial skin surface. However, it is still able to detect cardiac rhythm. Significance: The result of our work is an optoelectronic phantom for reproducible testing of PPGI camera performance in terms of signal quality and measurement equipment costs.

Rodent Arena Tracker (RAT): A Machine Vision Rodent Tracking Camera and Closed Loop Control System.

Video tracking is an essential tool in rodent research. Here, we demonstrate a machine vision rodent tracking camera based on a low-cost, open-source, machine vision camera, the OpenMV Cam M7. We call our device the rodent arena tracker (RAT), and it is a pocket-sized machine vision-based position tracker. The RAT does not require a tethered computer to operate and costs about $120 per device to build. These features make the RAT scalable to large installations and accessible to research institutions and educational settings where budgets may be limited. The RAT processes incoming video in real-time at 15 Hz and saves x and y positional information to an onboard microSD card. The RAT also provides a programmable multi-function input/output pin that can be used for controlling other equipment, transmitting tracking information in real time, or receiving data from other devices. Finally, the RAT includes a real-time clock (RTC) for accurate time stamping of data files. Real-time image processing averts the need to save video, greatly reducing storage, data handling, and communication requirements. To demonstrate the capabilities of the RAT, we performed three validation studies: (1) a 4-d experiment measuring circadian activity patterns; (2) logging of mouse positional information alongside status information from a pellet dispensing device; and (3) control of an optogenetic stimulation system for a real-time place preference (RTPP) brain stimulation reinforcement study. Our design files, build instructions, and code for the RAT implementation are open source and freely available online to facilitate dissemination and further development of the RAT.

Improving Visual Feature Extraction in Glacial Environments

Glacial science could benefit tremendously from autonomous robots, but previous glacial robots have had perception issues in these colorless and featureless environments, specifically with visual feature extraction. This translates to failures in visual odometry and visual navigation. Glaciologists use near-infrared imagery to reveal the underlying heterogeneous spatial structure of snow and ice, and we theorize that this hidden near-infrared structure could produce more and higher quality features than available in visible light. We took a custom camera rig to Igloo Cave at Mt. St. Helens to test our theory. The camera rig contains two identical machine vision cameras, one which was outfitted with multiple filters to see only near-infrared light. We extracted features from short video clips taken inside Igloo Cave at Mt. St. Helens, using three popular feature extractors (FAST, SIFT, and SURF). We quantified the number of features and their quality for visual navigation by comparing the resulting orientation estimates to ground truth. Our main contribution is the use of NIR longpass filters to improve the quantity and quality of visual features in icy terrain, irrespective of the feature extractor used.

A Complete Process For Shipborne Sea-Ice Field Analysis Using Machine Vision

A sensor instrumentation and an automated process are proposed for sea-ice field analysis using ship mounted machine vision cameras with the help of inertial and satellite positioning sensors. The proposed process enables automated acquisition of sea-ice concentration, floes size and distribution. The process contains pre-processing steps such as sensor calibration, distortion removal, orthorectification of image data, and data extraction steps such as sea-ice floe clustering, detection, and analysis. In addition, we improve the state of the art of floe clustering and detection, by using an enhanced version of the k-means algorithm and the blue colour channel for increased contrast in ice detection. Comparing to manual visual observations, the proposed method gives significantly more detailed and frequent data about the size and distribution of individual floes. Through our initial experiments in pack ice conditions, the proposed system has proved to be able to segment most of the individual floes and estimate their size and area.

RhizoVision Crown: An Integrated Hardware and Software Platform for Root Crown Phenotyping.

Root crown phenotyping measures the top portion of crop root systems and can be used for marker-assisted breeding, genetic mapping, and understanding how roots influence soil resource acquisition. Several imaging protocols and image analysis programs exist, but they are not optimized for high-throughput, repeatable, and robust root crown phenotyping. The RhizoVision Crown platform integrates an imaging unit, image capture software, and image analysis software that are optimized for reliable extraction of measurements from large numbers of root crowns. The hardware platform utilizes a backlight and a monochrome machine vision camera to capture root crown silhouettes. The RhizoVision Imager and RhizoVision Analyzer are free, open-source software that streamline image capture and image analysis with intuitive graphical user interfaces. The RhizoVision Analyzer was physically validated using copper wire, and features were extensively validated using 10,464 ground-truth simulated images of dicot and monocot root systems. This platform was then used to phenotype soybean and wheat root crowns. A total of 2,799 soybean (Glycine max) root crowns of 187 lines and 1,753 wheat (Triticum aestivum) root crowns of 186 lines were phenotyped. Principal component analysis indicated similar correlations among features in both species. The maximum heritability was 0.74 in soybean and 0.22 in wheat, indicating that differences in species and populations need to be considered. The integrated RhizoVision Crown platform facilitates high-throughput phenotyping of crop root crowns and sets a standard by which open plant phenotyping platforms can be benchmarked.

Adaptive exposure control for image-based visual-servo systems using local gradient information

We present a new method for automatic adjustment of camera exposure time for visual-servo systems. The proposed method can improve the robustness of image processing in a high-dynamic-range environment. In this paper, we evaluate an appropriate exposure time by computing the local gradient information of a target area, allowing a camera to capture images without losing target features under artificially adjusted illumination conditions. To validate the advantage of the proposed method, an evaluation is made using an off-the-shelf visual-servo system with a machine vision camera. The experimental results demonstrate the effectiveness of our method, which improves the performance of the target recognition algorithm.

An Experimental Investigation on Uncertainty in Measuring Vibration Deflection Shapes With Digital Image Correlation

3-D digital image correlation (DIC) is a widespread full-field displacement measurement technique based on stereo vision. It was formulated to deal with static problems in experimental mechanics in the early seventies, and since then, the applicability of the technique increased due to increasing frame rates and dropping prices of machine vision cameras. As a consequence, DIC became a feasible solution for general-purpose vibration testing. This paper, however, does not offer a comprehensive analysis of DIC accuracy when applied to vibration testing, and therefore, this paper offers a “Type A” evaluation of uncertainty when measuring vibration deflection shapes with DIC. Uncertainty is evaluated for different experimental conditions on a stepped sine test. Data show that the average value of uncertainty normally lies below 0.02 mm, but in resonant conditions, it can increase up to 0.05 mm (considering a field of view of about half a meter). This demonstrates a strong correlation between the deflection amplitude and the random uncertainty due to motion blur.

Lever-arm and boresight correction, and field of view determination of a spectroradiometer mounted on an unmanned aircraft system

This study addresses the correction of lever-arm offset and boresight angle, and field of view (FOV) determination to enable accurate footprint determination of a spectroradiometer mounted on an unmanned aircraft system (UAS). To characterise the footprint, an accurate determination of the spectroradiometer position and orientation (pose) must be acquired with a global navigation satellite system (GNSS) and an inertial measurement unit (IMU). Accurate pose estimation requires an accurate lever-arm and boresight correction between the pose measuring sensors and the spectroradiometer. Similarly, the spectroradiometer FOV is required to determine the footprint size as a function of above ground level (AGL) flying height. The system used in this study consists of an IMU with dual-frequency and dual-antenna GNSS receiver, a machine vision camera, and a point-measuring spectroradiometer (Ocean Optics QE Pro). The lever-arm offset was determined from a scaled 3D point cloud of the system, created using photos of the airframe and processed with the structure-from-motion (SfM) algorithm. The boresight angles were estimated with stationary experiments by computing the difference between the orientations of the IMU, the spectroradiometer, and the camera. The orientation of the spectroradiometer was determined by moving a spectrally distinct target into the FOV. The orientation of IMU was measured by averaging its readings during the stationary epoch, while SfM was employed as an independent technique to estimate the orientation of the camera. The footprint of the spectroradiometer for a combination of AGL height and Gershun tube aperture ring was determined experimentally, enabling computation of the effective FOV. In-flight validation of the lever-arm and boresight correction was performed by comparing the corrected pose of the co-mounted camera with the pose derived from SfM as the reference. Our experimental results demonstrate that controlled determination and correction of lever-arm and boresight increases the pose estimation accuracy and thereby supports the direct georeferencing of a UAS-mounted spectroradiometer point observation.

Reading Aviation Wire Text in Natural Images under Assembly Workshop via Deeplearning

Reading aviation wire text is an important step for aircraft wire assembly and also a challenging problem. Aviation wire text reading means to detect and recognize the serial number printed on the surface of aviation wires or aircraft cables. The main challenges of this task lie on small sizes, low contrast and background noise in images. In this paper, we propose a new method for aviation wire text detection and recognition via deep neural networks. The detection network utilizes text border features to roughly locate text regions in images. Then the coordinates of bounding boxes of texts are regressed by pixel-level predicted offsets in adjacent regions. The recognition network is capable to recognize the text regions with diverse lengths after padding strategies. In addition, a dataset of aviation wires captured by machine vision camera in the assembly workshop is developed. A flexible method to generate synthetic images of text whose characters are arranged in styles of both head-to-tail and left-to-right is presented. On aviation wire datasets, the proposed method for text detection and recognition achieves comparable performance and significantly outperforms previous approaches.

Portable, low-cost multispectral imaging system: design, development, validation, and utilization.

Optical spectral images can be used to estimate the amount of bulk absorbers in tissues, specifically oxy- and deoxyhemoglobin, as well as scattering parameters. Most systems that capture spectral image data are large, heavy, and expensive. This paper presents a full end-to-end analysis of a low-cost reflectance-mode multispectral imaging system operating in the visible and near-infrared spectra. The system consists of 13 LEDs mounted on a printed circuit board, a monochrome machine vision camera, and a tablet computer to control the hardware. The bill of materials for the system is less than $1000. Hardware design and implementation are detailed. Calibration, image capture, and preprocessing are also discussed. In validation experiments, excellent agreement is observed in diffuse reflectance measurements between the spectral camera setup and a spectrometer. To demonstrate that such spectral image data can yield meaningful optical measurements in vivo, the forearms of eight volunteers are imaged in the system. Their data are then analyzed to estimate the tissue optical properties of different skin layers using a Monte Carlo lookup table. In three volunteers, spectral images are captured before and after inducing erythema using a warm wet towel. Across the three subjects, a clear increase in the blood content of the superficial plexus layer was observed as a result of the erythema. Collectively, these findings suggest that a low-cost system can capture accurate spectral data and that clinically meaningful information can be derived from it.

Application of photogrammetry for spatial free surface elevation and velocity measurement in wave flumes

This article presents a method for obtaining the spatial free surface elevation and velocity field for the water surface in a wave flume over a relatively large measurement area for this type of application (approximately 1.5 m × 1.5 m). The technique employs proprietary videogrammetry software to post-process stereo images captured by multiple synchronised machine vision cameras. Dimensional resolution and other limitations are similar to that experienced for particle imaging velocimetry systems ( x, y resolution of 2 mm). Imaging of the free surface was enabled by the use of millions of bespoke slightly positively buoyant fluorescent flakes. Ultraviolet light was used as the primary light source to excite the fluorescent flakes. Reflected ultraviolet light was attenuated by a high-pass filter fitted to the cameras so that only the emitted light from the fluorescent flakes was visible. The software was validated using a simple linear translation experiment. An application is demonstrated for the radiated wave field generated from a submerged sinusoidal heaving sphere for two cases: one single and five consecutive oscillations. Results agree with linear wave theory which indicates that the floating flakes had minimal impact on the water surface particle motion at the scale tested. It is, therefore, concluded that spatial measurement of the free surface elevation and velocity using the method presented has good resolution over a large measurement field. The flakes were found to follow the free surface well, but the measurement area is constrained to where the pattern of flakes exists in the image. Hence, application of floating markers is not suitable for experiments with significant outflow/upwelling which would wash away the floating markers from the intended measurement area.

Gradient-Based Camera Exposure Control for Outdoor Mobile Platforms

We introduce a novel method to automatically adjust camera exposure for image processing and computer vision applications on mobile robot platforms. Because most image processing algorithms rely heavily on low-level image features that are based mainly on local gradient information, we consider that gradient quantity can determine the proper exposure level, allowing a camera to capture the important image features in a manner robust to illumination conditions. We then extend this concept to a multi-camera system and present a new control algorithm to achieve both brightness consistency between adjacent cameras and a proper exposure level for each camera. We implement our prototype system with off-the-shelf machine-vision cameras and demonstrate the effectiveness of the proposed algorithms on practical applications, including pedestrian detection, visual odometry, surround-view imaging, panoramic imaging and stereo matching.