Binocular Imaging Research Articles

Image-to-image machine translation enables computational defogging in real-world images.

Computational defogging using machine learning presents significant potential; however, its progress is hindered by the scarcity of large-scale datasets comprising real-world paired images with sufficiently dense fog. To address this limitation, we developed a binocular imaging system and introduced Stereofog-an open-source dataset comprising 10,067 paired clear and foggy images, with a majority captured under dense fog conditions. Utilizing this dataset, we trained a pix2pix image-to-image (I2I) translation model and achieved a complex wavelet structural similarity index (CW-SSIM) exceeding 0.7 and a peak signal-to-noise ratio (PSNR) above 17, specifically under dense fog conditions (characterized by a Laplacian variance, vL < 10). We note that Stereofog contains over 70% of dense-fog images. In contrast, models trained on synthetic data, or real-world images augmented with synthetic fog, exhibited suboptimal performance. Our comprehensive performance analysis highlights the model's limitations, such as issues related to dataset diversity and hallucinations-challenges that are pervasive in machine-learning-based approaches. We also propose several strategies for future improvements. Our findings emphasize the promise of machine-learning techniques in computational defogging across diverse fog conditions. This work contributes to the field by offering a robust, open-source dataset that we anticipate will catalyze advancements in both algorithm development and data acquisition methodologies.

Binocular reconstruction of breaking ship bow waves in circulating water channel

This study conducts an experiment on the bow waves generated by a 1:80 KRISO container ship (KCS) model in a circulating water channel. The Froude numbers (characterized by the length of perpendiculars) reach high conditions of 0.377, 0.424, and 0.471. A binocular imaging system is employed to reconstruct three-dimensional configurations of the bow wave surface. The dynamic characteristics of bow wave fields are investigated by observing the standard deviation and employing the Proper Orthogonal Decomposition (POD) method. The power spectral density function presents a power-law relationship with frequency. A novel technique referred as Stereoscopic Foam Image Velocimetry (SFIV) is developed to extract the instantaneous three-dimensional velocity of the foam in breaking area. The velocity field obtained through SFIV defines the nominal time-averaged velocity and nominal turbulent energy. A correlation mapping from pixel coordinates to world coordinates is established to evaluate the actual physical uncertainty. The results demonstrate the superior spatiotemporal accuracy and resolution of binocular imaging technology in measuring the characteristics of bow waves. This research provides significant experimental data of surface three-dimensional configurations for the bow waves breaking issues at high Froude numbers.

Monitoring SF6 Gas Leakage Based on a Customized Binocular System

Sulfur hexafluoride (SF6) gas is extensively utilized as an insulating and arc-quenching medium in the circuit breakers and isolating switches of electrical equipment. It effectively isolates the circuits from the atmosphere and promptly extinguishes arcs. Therefore, the issue of SF6 gas leakage poses a significant threat to the related application fields, and the detection of SF6 gas leakage becomes extremely important. Infrared imaging detection offers advantages including non-contact, high precision, and visualization. However, most existing infrared detection systems are equipped with only one filter to detect SF6 gas. The images captured contain background noise and system noise, making these systems vulnerable to interference from such noises. To address these issues, we propose a method for monitoring SF6 gas leakage based on a customized binocular imaging (CBI) system. The CBI system has two filters, greatly reducing the interference of system noise and background noise. The first filter features the absorption resonant peak of SF6 gas. The second filter is used to record background noise and system noise. One aspect to note is that, in order to avoid the interference of other gases, the central wavelength of this second filter should keep away from the absorption resonant peaks of those gases. Accordingly, the central wavelengths of our customized filters were determined as 10,630 nm and 8370 nm, respectively. Then, two cameras of the same type were separately assembled with a customized filter, and the CBI prototype was accomplished. Finally, we utilized the difference method using two infrared images captured by the CBI system, to monitor the SF6 gas leakage. The results demonstrate that our developed system achieves a high accuracy of over 99.8% in detecting SF6 gas. Furthermore, the CBI system supports a plug-and-play customization to detect various gases for different scenarios.

Spatial Design Research on Digital Media Technology Intervention in Immersive Art Theory Exhibition Venues

Abstract The article takes the application of digital media technology in the spatial layout design of art exhibition halls as the entry point, and utilizes Web technology and the integrated simulation platform of urban VRGIS to establish the virtual space of immersive art exhibition halls. The article obtains binocular imaging data through binocular visual imaging model and solves the three-dimensional coordinate system by using the method of polar geometric constraints, and then reconstructs the three-dimensional space through the point cloud data to realize the design of the virtual space of the immersive art exhibition hall. The article evaluates the effectiveness of the immersive art gallery virtual space from three aspects: interactivity, DPI recovery value and eye movement data visualization. It is found that users are most effective with direct interaction within an interaction depth of 1.1 m to 1.7 m. In comparison, ray interaction is more appropriate when the interaction depth exceeds 2.3 m. Applying the method of this paper, the global DPI recovery value of the immersive art gallery can reach 89.67%, and the pupil diameter of the subjects can reach about 5.79 mm in the virtual space test over 160 seconds. In this paper, digital media technology can more effectively realize the immersive presentation of art theories, to better satisfy people’s growing spiritual needs and provide more diverse cultural experiences.

Weld Geometry Prediction Based on Binocular Vision and Deep Learning

To improve the level of welding automation in the industry, there are increasing requirements for highly intelligent and accurate inspections of the welding process in real time. This paper proposed a new method for predicting weld dimensions based on binocular imaging information and a deep learning system. The binocular imaging information was acquired by binocular vision equipment and an image processing algorithm. A convolutional neural network structure was developed by adding a fully connected block and loss function judgment. A new calculating procedure was proposed to extract and link the information of the processed weld pool image and the weld parameters effectively. With the help of 7394 training samples, the results of 1849 testing samples showed that the overall accuracy of the proposed model was higher than 93% for the prediction of weld dimensions, which could meet the requirements in practical applications.

Autonomous perception method of multi‐degree‐of‐freedom industrial robot arm trajectory

AbstractIn this study, a novel autonomous sensing method of multi‐degree‐of‐freedom industrial robot arm trajectory is proposed. The research takes the distance sensor to collect environmental data, and takes the point cloud data scanned by 3D laser as the basis. The environment model of multi‐degree‐of‐freedom industrial robotic arm is established by Iterative Closest Point (ICP). Then the target object is calibrated by binocular imaging technology. Subsequently, angle of each joint of multi‐degree‐of‐freedom industrial robotic arm is calculated to determine the spatial attitude of the robotic arm. In addition, 3D LiDAR is installed at the end of the robotic arm, and the end trajectory of multi‐degree‐of‐freedom industrial robotic arm is sensed autonomously by using the optimal function. The proposed method has advantages of high accuracy and short sensing time in autonomous sensing of multi‐degree‐of‐freedom industrial robot arm trajectory.

Binocular Imaging with the Conformal Eyes

Binocular Imaging with the Conformal Eyes

Binocular stereo matching of real scenes based on a convolutional neural network and computer graphics.

The binocular stereo matching method based on deep learning has limited cross-domain generalization ability, and it is a tricky problem to obtain a large amount of data from real scenes for training. The most advanced stereo matching network is difficult to apply to new real scenes. In this paper, we propose a real-scene stereo matching method based on a convolutional neural network and computer graphics. A virtual binocular imaging system is constructed by introducing graphics software, and a high-quality semi-synthetic dataset close to the texture characteristics of the real scene is constructed for training the network. A feature standardization layer is embedded in the feature extraction module of the proposed network to further reduce the feature space difference between semi-synthetic data and real scene data. Three small 4D cost volumes are constructed to replace one large 4D cost volume, which reduces GPU memory consumption and improves the matching performance of the network. The experimental results show that compared with the traditional stereo matching method, the matching accuracy of the proposed method is significantly improved by about 60%. Compared with other learning-based methods, the matching accuracy is increased by about 30%, the matching speed is increased by 38%, and it has good robustness to the interference of defocus blur and Gaussian noise.

Fruit maturity and location identification of beef tomato using R-CNN and binocular imaging technology

The objective of this study was to identify the maturity and position of tomatoes in greenhouse. Three parts have been included in this study: building the model of image capturing and object detection, position identification of mature fruits and prediction of the size of the mature fruits. For the first part, image capturing in different time and object detection will be conducted in the greenhouse for identification of mature fruits. For the second part, the relative 3D position of the mature fruits calculated by the binocular vision was compared with the actual measured position. For the third part, the size of the bounding box from the object detection was compared with the actual size of the mature fruit, and the correlation was calculated in order to pre-adjust the width of the gripper for plucking operation in the future. The precision and the recall of the mature fruits of this study are over 95%. The average error of the 3D position is 0.5 cm. The actual size of the fruits and the R-squared of the size of the bounding box are over 0.9.

IMPLEMENTATION STRATEGY OF VISIBLE AND NEAR-INFRARED IMAGING SPECTROMETER ON YUTU-2 ROVER BASED ON VISION MEASUREMENT TECHNOLOGY

Abstract. The Chang'e-4 successfully landed on the far side of the moon in January 2019. By the 12th lunar day, its Yutu-2 rover had achieved a breakthrough travel distance of greater than 300 m. A visible and near-infrared imaging spectrometer (VNIS), consisting of a visible and near-infrared (VNIR) imaging spectrometer and a shortwave infrared (SWIR) spectrometer was used for detecting mineralogical compositions of lunar-surface materials. Because VNIS is fixed on the front of the rover, and the field-of-view (FOV) of VNIR and SWIR are small (8.5° and 3.6° respectively), approaching and accurately pointing at the specific science target depend completely on the precise control of the moving rover.In this paper, a successful method of VNIS target detection based on vision measurement is proposed. First, the accurate position of the target is calculated via navigation camera imaging. Then, the moving path is planned by considering the terrain environment, illumination, communication condition, and other constraints. After the rover moves to the designed position, the binocular imaging of the hazard-avoidance cameras are activated, the detection direction and forward distance are calculated according to the images, and the FOV trajectory of the VINS is predicted while moving. Finally, by choosing the required moving control parameters, the imaging field of the VINS accurately cover the detected targets visually.These methods have been verified many times, and the results show that they are effective and feasible. The research results based on the VNIS data have successfully revealed the material composition on the far side of the moon and have deepened human understanding of its formation and evolution.

A Deep-Learning-based 3D Defect Quantitative Inspection System in CC Products Surface.

To create an intelligent surface region of interests (ROI) 3D quantitative inspection strategy a reality in the continuous casting (CC) production line, an improved 3D laser image scanning system (3D-LDS) was established based on binocular imaging and deep-learning techniques. In 3D-LDS, firstly, to meet the requirements of the industrial application, the CCD laser image scanning method was optimized in high-temperature experiments and secondly, we proposed a novel region proposal method based on 3D ROI initial depth location for effectively suppressing redundant candidate bounding boxes generated by pseudo-defects in a real-time inspection process. Thirdly, a novel two-step defects inspection strategy was presented by devising a fusion deep CNN model which combined fully connected networks (for defects classification/recognition) and fully convolutional networks (for defects delineation). The 3D-LDS’ dichotomous inspection method of defects classification and delineation processes are helpful in understanding and addressing challenges for defects inspection in CC product surfaces. The applicability of the presented methods is mainly tied to the surface quality inspection for slab, strip and billet products.

基于双目变焦超分辨率成像的测距方法

基于双目变焦超分辨率成像的测距方法

Effect of Catadioptric Component Postposition on Lens Focal Length and Imaging Surface in a Mirror Binocular System.

The binocular vision system is widely used in three-dimensional measurement, drone navigation, and many other fields. However, due to the high cost, large volume, and inconvenient operation of the two-camera system, it is difficult to meet the weight and load requirements of the UAV system. Therefore, the study of mirror binocular with single camera was carried out. Existing mirror binocular systems place the catadioptric components in front of the lens, which makes the volume of measurement system still large. In this paper, a catadioptric postposition system is designed, which places the prism behind the lens to achieve mirror binocular imaging. The influence of the post prism on the focal length and imaging surface of the optical system is analyzed. The feasibility of post-mirror binocular imaging are verified by experiments, and it is reasonable to compensate the focal length change by changing the back focal plane position. This research laid the foundation for the subsequent research on the 3D reconstruction of the novel mirror binocular system.

3-D Computational Ghost Imaging With Extended Depth of Field for Measurement

The accuracy of traditional 3-D measurements is limited by the depth of field (DOF), such as in binocular imaging and holography. Ghost imaging (GI) can achieve lensless imaging with a DOF deeper than the traditional imaging. To realize this, a computational GI method is proposed and a system built, in which the light field information and modulation information are used to reconstruct the 3-D image. This system uses two projection devices for light modulation and pattern projecting, and utilizes a detector to receive the reflected signals of the two projectors on the object synchronously. The experiment shows that for an original DOF of 0.4 m, our system can extend the depth to 1.5 m. The method should have the great application prospects in the fields of remote sensing, imaging guidance, and medical imaging.

A Pose Measurement Method of a Space Noncooperative Target Based on Maximum Outer Contour Recognition

The relative pose (position and attitude) measurement of space noncooperative targets is very important for on-orbit servicing activities, such as target tracking, approaching, and capturing. The traditional methods rarely consider the instability of feature extraction and image blurring caused by target tumbling. In this paper, a method based on the maximum outer contour (MOC) recognition is proposed to measure the pose of the target. Different feature extraction algorithms can simultaneously achieve close- and long-range measurement tasks. First, the trailing image is restored by the image enhancement method. Second, the “rough + fine” combination recognition method is used for contour extraction and connected component labeling of the restored image, and the target feature extraction time is reduced to one-third of traditional methods. Furthermore, the elliptical surface on the MOC is fitted by the least squares method (LSM), and the ellipse parameters (i.e., the center position, the long- and short-axes, and the deflection angle) are extracted. The accuracy of the target recognition is improved. Third, for the close-range measurement, based on the detected ellipse parameter, the pose of the noncooperative target is solved by the binocular imaging algorithm of the space circle; for the long-range measurement, the contour centroid of the target is calculated by the detected MOC, and the position of the target is solved by the LSM. Moreover, the effectiveness of the method is verified by the OpenSceneGraph numerical simulation system. Finally, an experimental system consisting of a binocular camera, a UR5 manipulator, and a satellite mockup was built. The experimental results verified the proposed method.

Foreign object recognition technology for port transportation channel based on automatic image recognition

In the port transportation, the transportation vehicle is extremely easy to cause a bad condition in the transportation process due to the influence of foreign matter, and the current foreign matter detection is still carried out by manual methods. Based on this, this study is based on the automatic image recognition technology, using the binocular imaging system for video collection and analyzing the video images for foreign object recognition. We use extreme median filtering to perform image noise processing and improve the traditional Canny edge detection algorithm to obtain an improved edge detection method for Canny operator. The algorithm is used to perform image edge detection, and the image gray histogram is used to enhance the foreign object image processing. Combined with the experimental analysis, the detection method of this study has certain effects, which can be used for foreign object identification in port transportation channels and can provide theoretical reference for subsequent related research.

Gender differences of basketball players aged 12-13 years according to the response to a moving object

Purpose: The study aims at the determining gender differences of basketball players aged 12-13 years in response to an object in motion that determines the predominance of excitation and inhibition processes. Material: The participants of the study are 58 basketball players aged 12-13 years, 35 are male basketball players and 23 are female. Male basketball players had 4-5-year training experience, female had 1,5-2-year experience. All athletes had normal visual acuity and were healthy at the time of the study. Young basketball players were informed about the purpose of the study. The computer program containing visual stimuli, moving with acceleration from different points of the monitor was used for testing. Results: The deviations from the target data as well the pole were taken into consideration. It demonstrated the predominance of excitation and inhibition processes of basketball players of different sexes. The response accuracy to visual stimuli without taking into account the pole during perception with the left eye and the right eye is shown. Conclusions: It was found out that the females’ response accuracy during monocular perception is more closely related to the response accuracy during binocular imaging, whereas the role of the males’ leading eye is increased during perception of objects from a convenient side. This may indicate the peculiar reactions under the conditions of double perception of objects in solving complex spatial problems.

Binocular Vision Three-dimensional Imaging Technology by Using Structural Light Projection

SIFT matching algorithm is used to carry out the binocular three-dimensional imaging. Active projection is introduced to solve the problem of low feature quantity and poor matching results in the matching process. By means of projection random speckle, the matching feature is increased, and the matching quality is greatly improved. According to the train running part of the three-dimensional imaging experiment, achieved a good imaging result. Compared with the Fourier profilometry in the active three-dimensional imaging technology. The experimental results show that the structured light projection binocular three-dimensional imaging has a better effect.

Binary descriptor-based dense line-scan stereo matching

We present a line-scan stereo system and descriptor-based dense stereo matching for high-performance vision applications. The stochastic binary local descriptor (STABLE) descriptor is a local binary descriptor that builds upon the principles of compressed sensing theory. The most important properties of STABLE are the independence of the descriptor length from the matching window size and the possibility that more than one pair of pixels contributes to a single-descriptor bit. Individual descriptor bits are computed by comparing image intensities over pairs of balanced random subsets of pixels chosen from the whole described area. On a synthetic as well as real-world examples, we demonstrate that STABLE provides competitive or superior performance than other state-of-the-art local binary descriptors in the task of dense stereo matching. The real-world example is derived from line-scan binocular stereo imaging, i.e., two line-scan cameras are observing the same object line and 2-D images are generated due to relative motion. We show that STABLE performs significantly better than the census transform and local binary patterns (LBP) in all considered geometric and radiometric distortion categories to be expected in practical applications of stereo vision. Moreover, we show as well that STABLE provides comparable or better matching quality than the binary robust-independent elementary features descriptor. The low computational complexity and flexible memory footprint make STABLE well suited for most hardware architectures. We present quantitative results based on the Middlebury stereo dataset as well as illustrative results for road surface reconstruction.

Optical Mapping of Membrane Potential and Epicardial Deformation in Beating Hearts

Cardiac optical mapping uses potentiometric fluorescent dyes to image membrane potential (Vm). An important limitation of conventional optical mapping is that contraction is usually arrested pharmacologically to prevent motion artifacts from obscuring Vm signals. However, these agents may alter electrophysiology, and by abolishing contraction, also prevent optical mapping from being used to study coupling between electrical and mechanical function. Here, we present a method to simultaneously map Vm and epicardial contraction in the beating heart. Isolated perfused swine hearts were stained with di-4-ANEPPS and fiducial markers were glued to the epicardium for motion tracking. The heart was imaged at 750 Hz with a video camera. Fluorescence was excited with cyan or blue LEDs on alternating camera frames, thus providing a 375-Hz effective sampling rate. Marker tracking enabled the pixel(s) imaging any epicardial site within the marked region to be identified in each camera frame. Cyan- and blue-elicited fluorescence have different sensitivities to Vm, but other signal features, primarily motion artifacts, are common. Thus, taking the ratio of fluorescence emitted by a motion-tracked epicardial site in adjacent frames removes artifacts, leaving Vm (excitation ratiometry). Reconstructed Vm signals were validated by comparison to monophasic action potentials and to conventional optical mapping signals. Binocular imaging with additional video cameras enabled marker motion to be tracked in three dimensions. From these data, epicardial deformation during the cardiac cycle was quantified by computing finite strain fields. We show that the method can simultaneously map Vm and strain in a left-sided working heart preparation and can image changes in both electrical and mechanical function 5 min after the induction of regional ischemia. By allowing high-resolution optical mapping in the absence of electromechanical uncoupling agents, the method relieves a long-standing limitation of optical mapping and has potential to enhance new studies in coupled cardiac electromechanics.