Vision Measurement System Research Articles

Micro vision-based measurement of concentricity for both TO base and active area of a APD chip in optical component packaging.

The avalanche photodiode (APD) chip is the core component of the transistor outline (TO). The concentricity between the inner circle (IC) of the APD active area and the outer circle (OC) of the TO base will directly affect a component's key performance indicators, such as external quantum efficiency, receiving sensitivity and responsivity, thereby impacting quality assurance, performance improvement, and stable operation. Nevertheless, as the surge in demand for components increases, the traditional visual inspection relying on manual and microscope has been unable to meet the requirements of mass manufacturing for real-time quality and efficiency. Thus, a Concentricity Microscopic Vision Measurement System (CMVMS) mainly composed of a microscopic vision acquisition unit and an intelligent concentricity measurement unit has been proposed, designed, and implemented. On the basis of analyzing the 3D complex environment of TO components, a coaxial illumination image acquisition scheme that can take into account the characteristics of the OC and IC has been proposed. Additionally, a concentricity image measurement method based on dynamic threshold segmentation has been designed to reduce the interference of complex industrial environment changes on measurement accuracy. The experiment results show that the measurement accuracy of the CMVMS system is over 97%, and with a single measurement time of less than 0.2s, it can better meet the real-time and accuracy requirements. To the best of our knowledge, this is the first report on the realization of real-time concentricity measurement in optical component packaging, and this technology can be extended to other fields of concentricity measurement.

Three-dimensional reconstruction from a fringe projection system through a planar transparent medium.

A vision measurement system is placed in a protective cover made of a transparent medium to avoid environmental influences. Due to the deflection of light rays on the front and rear surfaces of the transparent medium, the imaging position of an object on the camera target plane is deviated, which makes the traditional vision detection methods based on the triangulation principle produce large measurement errors. This work introduces a three-dimensional (3D) reconstruction method by fringe projection system through a planar transparent medium. We derive the coordinate transformation relationship between a real-object point and a pseudo-object point caused by light refraction based on Snell's law of flat refraction. Based on the relationship, a modified fringe projection method is proposed for unbiased 3D reconstruction. Two experiments, including 3D shape measurement of a white plate with ring markers and 3D shape measurement of a regular spherical object are conducted. The results demonstrate the effectiveness of the proposed method in such measurement environment.

Visual encoder-based angle measurement method in low-frequency angular vibration calibration.

Angular vibration calibration is required to determine the sensitivity of sensors such as dynamic inclinometers, gyroscopes, and angular accelerometers, which are used for angular motion measurement in engineering applications. Additionally, the calibration performance depends on the accuracy of the angle measurement by laser interferometry or a circular grating (CG) method that is commonly used in vibration calibration. However, these methods usually own a complex and high-cost system or limited frequency and amplitude ranges. In this study, a novel, to the best of our knowledge, angle measurement method that combines a special visual encoder and an accurate angular position detection method is investigated; the method requires only a simple and flexible telecentric vision measurement system. Comparison experiments with the CG method demonstrate that the investigated method has the maximum measurement deviation of 0.0014° and 0.0138° for static angle measurement in the small-angle range and continuous full circle, respectively. The relative deviation of the angular vibration measurement in the range of 0.1-8 Hz with amplitudes 0-100° is less than 0.173%. Additionally, the relative deviation of calibrated sensitivity of a gyroscope by the investigated and CG methods is less than 0.096%.

The visual measurement system in the parachute used on the Tianwen-1 Mars exploration mission

ABSTRACT The parachute of the probe plays an important role in the deceleration of the Mars landing mission. This paper provides an overview of the visual measurement system of the parachute in Tianwen-1 exploration mission. Using two monitoring cameras, the system has captured stereo images during the parachute deployment. The morphological and motion parameters of the parachute are reconstructed by image-based feature tracking and 3D reconstruction methods. To improve the image quality, a super-resolution (SR) reconstruction method based on sparse dictionary coding is proposed. In addition, we use a multi-scale feature tracking method to track the feature points and update the motion parameters. The system has been verified by the test on the Earth and successfully applied on the onboard images. The reconstructed parachute parameters are important in the post-flight performance assessment, and the research results can be used as a regression analysis means for the structural design.

Automatic Detection of Chip Pin Defect in Semiconductor Assembly Using Vision Measurement

Abstract With the development of semiconductor assembly technology, the continuous requirement for the improvement of chip quality caused an increasing pressure on the assembly manufacturing process. The defects of chip pin had been mostly verified by manual inspection, which has low efficiency, high cost, and low reliability. In this paper, we propose a vision measurement method to detect the chip pin defects, such as the pin warping and collapse that heavily influence the quality of chip assembly. This task is performed by extracting the corner feature of the chip pins, computing the corresponding point pairs in the binocular sequence images, and reconstructing the target features of the chip. In the corner feature step, the corner detection of the pins using the gradient correlation matrices (GCM), and the feature point extraction of the chip package body surface using the crossing points of the fitting lines are introduced, respectively. After obtaining the corresponding point pairs, the feature points are utilized to reconstruct the three dimensional (3D) coordinate information in the binocular vision measurement system, and the key geometry dimension of the pins is computed, which reflects whether the quality of the chip pins is up to the standard. The proposed method is evaluated on the chip data, and the effectiveness is also verified by the comparison experiments.

Research on global calibration method of large-scene multi-vision sensors in wheel alignment

Wheel measurement and positioning technology is an important technology in vehicle production, and is of great importance for the safety of vehicles. At present, visual measurement and other methods are widely used in automotive production and inspection, but these methods are limited to the measurement of regular-sized vehicles. There is no effective solution to the problem of wheel alignment for large special-purpose vehicles with multiple tires. In order to solve the wheel positioning problem of large-size special-purpose vehicles, this paper designs a vision measurement system for wheel parameters in large scenes by combining vision sensors with linear motion guides to extend the vision measurement range and complete the global calibration of multiple-vision sensors with the help of laser trackers and self-researched stereo targets. This paper proposes an Elementary–Advance global calibration algorithm to improve the accuracy of coordinate system conversion during global calibration. This paper establishes a correction model for the measurement errors that exist in the initial global calibration process, and iterates the objective function to optimize the coordinate system conversion parameters between multiple vision sensors to improve the measurement accuracy of the system. The experimental results show that the goodness of fit and the accuracy of fit for the feature cylinder are 98.31% and 99.03% based on the global calibration method of this paper. The standard deviation of measurements for the standard ruler of the Inva alloy is less than 0.391 mm in the large dimensional range of 6050 mm × 3500 mm × 800 mm. Finally, by comparing the measurement results with the TrukCam four-wheel alignment instrument, the feasibility of the visual measurement system designed in this paper for large scenarios of wheel parameters is verified, and the measurement accuracy requirements for four-wheel alignment of special-purpose vehicles are met.

Least-Square-Method-Based Optimal Laser Spots Acquisition and Position in Cooperative Target Measurement.

The relative positioning precisions of coordinate points is an important indicator that affects the final accuracy in the visual measurement system of space cooperative targets. Many factors, such as measurement methods, environmental conditions, data processing principles and equipment parameters, are supposed to influence the cooperative target’s acquisition and determine the precision of the cooperative target’s position in a ground simulation experiment with laser projected spots on parallel screens. To overcome the precision insufficiencies of cooperative target measurement, the factors of the laser diode supply current and charge couple device (CCD) camera exposure time are studied in this article. On the hypothesis of the optimal experimental conditions, the state equations under the image coordinates’ system that describe the laser spot position’s variation are established. The novel optimizing method is proposed by taking laser spot position as state variables, diode supply current and exposure time as controllable variables, calculating the optimal controllable variables through intersecting the focal spot centroid line and the 3-D surface, and avoiding the inconvenience of solving nonlinear equations. The experiment based on the new algorithm shows that the optimal solution can guarantee the focal spot’s variation range in 5–10 pixels under image coordinates’ system equivalent to the space with a 3 m distance and 0.6–1.2 mm positioning accuracy.

Noncontact cable tension force estimation using an integrated vision and inertial measurement system

In this study, a noncontact cable tension force estimation technique was developed using an integrated vision and inertial measurement system (VIS) installed at a reference point, and the movement of the VIS was explicitly considered. Cable displacement was first estimated by applying a proposed contour-based algorithm to the vision measurements. Thereafter, the movement of the VIS at the reference point was estimated from the inertial measurement system and was used to compensate the error in the previously estimated cable displacement. Finally, the cable tension force was estimated using the compensated cable displacement. The feasibility of the proposed technique was validated through a laboratory test on a full-scale pedestrian bridge cable and a field test on a single-pylon cable-stayed pedestrian bridge. Overall, the proposed technique estimated the cable tension forces reliably with less than 1.3% discrepancy compared to those estimated by accelerometers.

Research on Vision Measurement System of Mechanical Workpiece Based on Machine Vision

With the rapid development of modern industrial industry, the workpiece measurement system of many industrial products gradually exceeds the operating load, and the operating efficiency is not high, the cycle is too long, and the accuracy rate is low. In order to solve the problem of automatic and intelligent measurement of mechanical workpieces in industrial production, an automatic mechanical workpiece measurement system based on machine vision is designed in this paper. The digital division of the workpiece is carried out automatically, and fast visual measurement is performed to judge whether the workpiece is qualified or not, which greatly improves the accuracy of workpiece detection. On this basis, this paper optimizes the machine vision measurement system of the workpiece based on the picture and label recognition system to make its operation efficiency higher and operation accuracy better.

Estimating Muscle Activity from the Deformation of a Sequential 3D Point Cloud.

Estimation of muscle activity is very important as it can be a cue to assess a person’s movements and intentions. If muscle activity states can be obtained through non-contact measurement, through visual measurement systems, for example, muscle activity will provide data support and help for various study fields. In the present paper, we propose a method to predict human muscle activity from skin surface strain. This requires us to obtain a 3D reconstruction model with a high relative accuracy. The problem is that reconstruction errors due to noise on raw data generated in a visual measurement system are inevitable. In particular, the independent noise between each frame on the time series makes it difficult to accurately track the motion. In order to obtain more precise information about the human skin surface, we propose a method that introduces a temporal constraint in the non-rigid registration process. We can achieve more accurate tracking of shape and motion by constraining the point cloud motion over the time series. Using surface strain as input, we build a multilayer perceptron artificial neural network for inferring muscle activity. In the present paper, we investigate simple lower limb movements to train the network. As a result, we successfully achieve the estimation of muscle activity via surface strain.

A flexible calibration method based on small planar target for defocused cameras

Complex engineering scenes increase the difficulty of camera calibration. High precision and easy operation are the necessary conditions to ensure successful calibration. This paper proposes a flexible calibration method for defocused cameras. Firstly, the initial calibration is completed by traditional Zhang's method. Then, the linear camera calibration based on the image pair is carried out. The image pair is obtained when the lens focuses on the calibration and measuring positions in sequence on the premise of keeping the calibration target stationary. The distortion coefficient is solved independently, the image distance conversion model is established to decouple the intermediate parameters, and finally, the linear solution of the camera parameters is obtained. The proposed method can complete the entire calibration process for a large-range vision measurement system using a small planar target. The number of calibration images required is close to Zhang's method. There is no need to fix the calibration target under multiple attitudes separately, which has strong operability. Experimental results show that the method is effective. The maximum calibration error of intrinsic parameters is 1.29%, and the maximum mean reprojection error is only 0.127 pixels. In addition, this method has good anti-noise performance and has satisfactory calibration accuracy under the action of different influencing factors, which indicates its broad applicability. The method can be applied to calibrating defocused cameras in special scenes such as engineering sites.

Analysis of the Reliability and Repeatability of Distance Visual Acuity Measurement with EyeSpy 20/20.

BackgroundVisual acuity is a critical component of visual function assessment for all ages. Standardized vision testing protocols may help prevent testing inconsistencies resulting from variations in test administration and interpretation of different examiners. However, most vision assessments outside of research settings, including in doctor’s offices, rarely employ standardized protocols. Validated protocols such as the Early Treatment for Diabetic Retinopathy Study (ETDRS) are frequently employed by vision researchers to ensure accurate and repeatable visual acuity measurements.MethodsThis study evaluates a desktop-based standardized vision testing algorithm (EyeSpy 20/20) specifically designed for use on mobile electronic platforms. Subjects were tested on a desktop version of the EyeSpy software for both the accuracy and duration of measurement of visual acuity and compared to a e-ETDRS chart in a randomized sequence. Children were recruited for this study and tested between two different visual acuity measurement systems. Bland–Altman analysis and correlation tests were done.ResultsHundred and ten children were recruited for the study. The EyeSpy 20/20 visual acuity testing algorithm as tested with the desktop version was non-inferior to the gold standard e-ETDRS testing algorithm on a desktop platform, but statistically faster to implement when administered on the same electronic testing platform.ConclusionEyeSpy 20/20 is a promising tool for vision screening and visual acuity evaluation in children.

Design of Automatic Motion Capture Algorithm for Yao’s Long Drum Dance Based on Multieye Machine Vision

Aiming at the problems of low accuracy and long time consumption in the existing dance motion automatic capture methods, this study designs an automatic capture algorithm of Yao’s long drum dance based on multieye machine vision. The images of Yao’s long drum dance are collected by the multieye machine vision measurement system, and the image signals under different lighting conditions are processed; on this basis, by determining the dance action image threshold and analyzing the change law of action image signal, the dance action image extraction of Yao’s long drum is completed. By converting the dance action images from different angles, the Euler angle of the images from different angles is determined and fused. On this basis, the observation items of the observation part of the Yao long drum dance action image are set, and the a priori conditions and a posteriori probability of the Yao long drum dance action image preprocessing are determined, so as to obtain the best result of the dance action image, and dance action image preprocessing is completed. According to the preprocessed image, the capture algorithm of area range is designed, the remaining areas to be captured are determined by means of classification, and the automatic capture algorithm design of Yao’s long drum dance is completed. The experimental results show that the motion image captured by the automatic motion capture algorithm of Yao’s long drum dance based on multieye machine vision has high accuracy, and the capture time is long, which is feasible.

A rotary vision system for high-precision measurement over large size

This paper introduced a rotary binocular vision measurement system (RBVMS) consisting of two cameras and two theodolites, with the latter employed as a two-dimensional rotation platform. The camera was driven by mounting it on the theodolite to cover different areas, effectively increasing the measuring range. In the camera calibration stage, virtual 3D points were generated by accurately moving the planar calibration board. A two-order radial and two-order tangential distortion model was considered. The experimental results showed that the intrinsic parameters of the camera had high stability, and the mean reprojection error was about 0.035 pixels. A unit quaternion method was applied to calibrate the transformation matrix between the camera and the theodolite. By solving the transformation matrix between the two theodolites, the measurement data of each position was unified under a fixed coordinate system. At 500 mm in front of the camera, the camera's field of view was about 200 mm. Two targets of different sizes were measured to verify the effectiveness and accuracy of the method. The measurement accuracy of 300 mm and 600 mm targets was about 0.014 mm and 0.027 mm, respectively.

Seismic performance evaluation of recycled aggregate concrete-filled steel tubular columns with field strain detected via a novel mark-free vision method

In this study, low cycle reciprocating load tests were conducted on nine recycled aggregate concrete-filled steel tube columns (RACSTCs) and a solitary ordinary concrete-filled steel tube contrast column (OCSTC). The axial compression ratio and steel-tube wall thickness were considered as the parameters of interest. Accordingly, the failure mechanism, hysteresis performance, skeleton curve, ductility coefficient, energy consumption performance, and stiffness degradation of the two types of columns were compared. A vision measurement system was developed to track the full field deformations of specimens, and a quadocular vision system was constructed to determine the concrete column deformations. The deformations were compared with those obtained from laser measurements using an image and point cloud stitching algorithm. The results show that the hysteresis curves of RACSTCs are non-pinched with large ductility coefficients for all the specimens and the equivalent viscous damping coefficients are in the range of 0.462–0.975. RACSTC specimens exhibit good seismic and deformation performance. A three-fold restoring force model was established for RACSTC specimens based on skeleton curve test data. The theoretical hysteresis curve obtained using the skeleton curve model and hysteresis rule shows good agreement with the hysteresis curve obtained from experiment. This can be used as a reference for research on the restoring force characteristics of RACSTCs. The results obtained using the vision measurement system show that the measured mean relative error values are less than 0.504%. This confirms that the multiple vision measurement method can be used to accurately measure curved surface deformations as well as the full field strain values of RACSTCs. Therefore, the proposed method can be used to evaluate the asymmetric deformation of structures subjected to vibrations, impact, and other load types, thereby providing early warning decision data.

Pose Determination of the Disc Cutter Holder of Shield Machine Based on Monocular Vision.

The visual measurement system plays a vital role in the disc cutter changing robot of the shield machine, and its accuracy directly determines the success rate of the disc cutter grasping. However, the actual industrial environment with strong noise brings a great challenge to the pose measurement methods. The existing methods are difficult to meet the required accuracy of pose measurement based on machine vision under the disc cutter changing conditions. To solve this problem, we propose a monocular visual pose measurement method consisting of the high precision optimal solution to the PnP problem (OPnP) method and the highly robust distance matching (DM) method. First, the OPnP method is used to calculate the rough pose of the shield machine’s cutter holder, and then the DM method is used to measure its pose accurately. Simulation results show that the proposed monocular measurement method has better accuracy and robustness than the several mainstream PnP methods. The experimental results also show that the maximum error of the proposed method is 0.28° in the direction of rotation and 0.32 mm in the direction of translation, which can meet the measurement accuracy requirement of the vision system of the disc cutter changing robot in practical engineering application.

Two-step calibration for vision measurement system with large field of view and high depth

Two-step calibration for vision measurement system with large field of view and high depth

Research on Adjustable Baseline Binocular Vision Measurement System

Binocular stereo vision belongs to machine vision. It uses two cameras to imitate human eyes to obtain two images of the object to be measured, and obtains the three-dimensional information of the object by calculating the positional deviation between the corresponding points of the images. The baseline length of the binocular camera determines the size of the observable distance and range. In today's increasingly complex work environment, the baseline of traditional binocular measurement equipment is fixed, and the usage scenarios are too limited. Therefore, this paper proposes a binocular vision measurement system with dynamically adjustable baselines. The system solves the problem of object localization at different depths by changing the baseline length.

Calibration Method of Distance Between Optical Center of Camera and Rotation Axis in Rotating Vision Measurement System

Calibration Method of Distance Between Optical Center of Camera and Rotation Axis in Rotating Vision Measurement System

Research on dynamic measurement of hot ring rolling dimension based on machine vision

In the process of ring rolling, the rolling process parameters of the equipment have an important influence on the forming process and forming quality of the ring. The ring rolling process has the characteristics of non-linearity, easy interference, rapid growth and asymmetric shape. These factors have resulted in the inability to measure the dimensions of the ring directly. In this work, we established a vision measurement system, and performed a series of operations on the image, such as threshold segmentation, hot ring contour extraction, oxide scale filtering, inner and outer ring extraction, inner and outer ring sampling, data visualization, etc. Moreover, we measured the dynamic dimension characteristics of small and medium size hot ring in the rolling process. The circularity, dimensional difference, center swing and vibration characteristics of the ring in the rolling process are obtained through the algorithm design, which characterizing the size change of the hot ring. The study provides practical methods and theoretical guidance for the subsequent measurement of large-size rings and feedback control of the rolling ring.