Binocular Vision System Research Articles

Vision-based relative pose determination of cooperative spacecraft in neutral buoyancy environment

Neutral buoyancy systems simulate the microgravity environment by taking advantage of buoyancy forces of water to offset the gravity of test bodies. Functional verification of space robots in neutral buoyancy system is of great importance for ground tests. The relative pose determination of a spacecraft plays an essential role in the on-orbit operation of space robots. In order to meet the requirement of on-orbit operation ground verification for space robots, this paper develops a vision-based system for determining the relative pose of cooperative spacecraft in neutral buoyancy environment. Cooperative markers and underwater binocular vision system are designed for the pose determination, and a cooperative spacecraft model is built. A detection and recognition method based on the topological characteristic is proposed for the cooperative marker. An underwater imaging model of binocular camera is established, and its refraction parameters are calibrated. The marker points are measured with an underwater binocular 3D measurement algorithm. Furthermore, the pose of cooperative spacecraft is determined using axisymmetric plane feature points. Additionally, the stable and reliable pose and velocity are obtained after the data are further processed with a Kalman filter. Finally, the experiments are carried out and the experimental results show that the proposed system can achieve a stable and reliable high-precision relative pose determination for cooperative spacecraft in neutral buoyancy environment.

Circle detection and location based on binocular stereo vision

In this paper, a binocular stereo vision system is developed for recognition and localization of a spherical object. The system uses two cameras to acquire images of object, shows and processes the images from two cameras simultaneously. It then applies point Hough transform to recognize the object and get real-time coordinates through the two images. Experiments with the proposed method yielded good results, as the system can detect and position a spherical object successfully.

The Development of Binocular Suppression in Infants.

Little is known about the time of development of binocular suppression. In the present study, we evaluated the emergence of binocular suppression in infants by using continuous flash suppression (CFS, Tsuchiya and Koch, 2005). In our experiment, one eye of infants was presented with a static face image at one side of the screen, while another eye was presented with dynamic Mondrian patterns in full screen. Adult observers confirmed that the static face image was consciously repressed by the changing Mondrian patterns. If binocular suppression was functional, the infants would not perceive the face and thus would not show any preference in the experiment. However, if binocular suppression in the infants was not yet acquired, they would perceive the face and the Mondrian patterns at the same time and would thus show preference for the side where the face was presented. The results showed that infants aged 2–3 months, but not those aged 4–5 months, detected the position of the face. Furthermore, this detection was not due to weak contrast sensitivity to the dynamic Mondrian mask. These results indicated that the immature binocular visual system may perceive different images from different eyes simultaneously and that infants may lose this ability after establishing binocular suppression at 4–5 months of age.

Research on Binocular Visual System of Robotic Arm Based on Improved SURF Algorithm

With the increasingly extensive application of robots in industrial production, visual of the robotic arm has become an important research direction in the field of robots, which is also recognized as believed to widely considered to the most vital directions of modern high-tech development. In the robot vision system, the binocular visual system is most widely applied. When the robotic arm based on binocular vision captures the object, SURF algorithm is adopted to realize feature point extraction and matching of images. Because of the influence of the objective factors, there is false match in the process of SURF matching of feature points. In this paper, SURF algorithm is improved and reverse feature vectors matching strategy is used to initialize the matching points set. Then the RANSAC algorithm is adopted to select the initial matching points set in accordance of the polar geometric model to get the final good match set. The experiment results show that the improved SURF algorithm has higher registration accuracy and faster speed than SURF.

Multifield Coupled Dynamics of Power Supply Arc Based on Image 3D Reconstruction Mathematical Model

During the conversion process of the short-circuit arc to the submerged power supply arc, because the short-circuit arc diameter is much larger than the submerged power supply arc, the specific position of the submerged power supply arc cannot be determined, and the initial position of the submerged power supply arc has a certain randomness. The purpose of this paper is to try to use the binocular stereo vision method to reconstruct the submerged arc image in three dimensions, obtain the moving image of the arc, obtain more comprehensive information than the two-dimensional image, and multi-field coupling dynamics of the power supply arc. In this paper, the multi-field coupling effects of electromagnetic force, thermal buoyancy, wind load and air resistance of the submerged arc of the half-wavelength transmission line are considered comprehensively, and the chain arc model is incorporated into the force equation and motion model of the submerged arc. By deeply analyzing the formation and motion mechanism of the arc root, the arc root model of the submerged power arc is established, and the optimal selection method of the current element length is proposed. The physical simulation experiment of the half-wavelength submerged arc is carried out, and the simulation results are compared with the experimental results to verify the effectiveness of the simulation model. By comparing the trajectory and self-extinguishing time of different submerged currents, wind speeds and winds to the submerged power supply arc, the effects of wind load, thermal buoyancy and electromagnetic force on the arc are analyzed, which indicates that the built arc model can effectively account for the wind load. The influence of submarine power arc motion and self-extinguishing characteristics. The results show that the submarine power arc image is acquired by constructing a binocular stereo vision system, and the obtained submersible arc image is reconstructed three-dimensionally. The three-dimensional image of the arc is obtained and compared with the actual image. The result shows that the three-dimensional reconstruction The result is effective, and the exploration study provides a new method for further obtaining the multiple variations of the true arc length of the arc with the arcing time.

Local Stereo Matching: An Adaptive Weighted Guided Image Filtering-Based Approach

Innovative applications in rapidly evolving domains such as robotic navigation and autonomous (driverless) vehicles rely on binocular computer vision systems that meet stringent response time and accuracy requirements. A key problem in these vision systems is stereo matching, which involves matching pixels from two input images in order to construct the output, a 3D map. Building upon the existing local stereo matching algorithms, this paper proposes a novel stereo matching algorithm that is based on a weighted guided filtering foundation. The proposed algorithm consists of three main steps; each step is designed with the goal of improving accuracy. First, the matching costs are computed using a unique combination of complementary methods (absolute difference, Census, and gradient algorithms) to reduce errors. Second, the costs are aggregated using an adaptive weighted guided image filtering method. Here, the regularization parameters are adjusted adaptively using the Canny method, further reducing errors. Third, a disparity map is generated using the winner-take-all strategy; this map is subsequently refined using a densification method to reduce errors. Our experimental results indicate that the proposed algorithm provides a higher level of accuracy in comparison to a collection of the existing state-of-the-art local algorithms.

Kinematics & compliance analysis of double wishbone air suspension with frictions and joint clearances

Currently available suspension kinematics and compliance (K&C) model is mainly used for qualitative analysis rather than quantitative analysis. In order to reproduce actual suspension K&C characteristic, all the suspension components need to take into account as much details as possible. Suspension stiffness non-linearity, suspension components friction and kinematic joint clearances are examples of how the component models can be made more specific. This paper proposes a modeling technique that can not only reliably represent the hysteretic characteristic but also analyze the effect of joint clearances. The suspension air spring model and bushing model are composed of nonlinear elastic force and Berg's friction force. The suspension supporting components are modeled as flexible bodies. The suspension parameter identification and model validation are also proposed. The test procedure of a new K&C test rig based on non-contact binocular vision system is introduced and the simulation results are compared with the experimental results. Results from tests show good agreement between the proposed suspension model and experimental data. Effects of the suspension components friction and clearance joints are investigated. This type of study can provide useful information on the vehicle design, analysis and optimization.

Pose measurement approach based on two-stage binocular vision for docking large components

Large components docking is an important step in industrial manufacturing. During the docking process, the relative pose of joining components needs real-time and accurate acquisition. Existing methods rely on expensive and complex optical instruments. Photogrammetry has the advantages of low cost and fast measurement speed, but its measurement accuracy decreases sharply along with the increase of the measurement range. This paper proposes a two-stage binocular vision which consists of two sets of binocular vision systems with different accuracy levels in the same coordinate system. Binocular system with corresponding structural parameters is designed for pose measurement at corresponding stages to solve the contradiction between the range and accuracy in measurement. A triangulation and spatial plane fitting method is proposed to calculate relative poses without introducing coordinate transformation errors. We adopt a novel 3D optimization method to further improve the accuracy. Experiment results show that this method can meet the measurement range and accuracy requirements for large components docking. Compared with traditional combined vision measurement station based on multi-vision sensor, the proposed method reduces coordinate transformation errors and overcomes the contradiction between the measurement range and accuracy, which can improve the accuracy and meet the requirements of engineering application.

3D reconstruction of specular surface by combined binocular vision and zonal wavefront reconstruction.

This paper proposes a framework for the three-dimensional (3D) reconstruction of specular surfaces. The framework begins with a modified reconstruction method that uses gradient descent to accurately search for the correspondence matches in the binocular vision system. An analysis of the binocular vision system shows that the reconstruction result is not necessarily unique and the reconstruction accuracy is significantly compromised by a bilinear interpolation error. An error-based selection algorithm is proposed to select the correct result. To improve the reconstruction accuracy, binocular vision and zonal wavefront reconstruction are incorporated. The reconstruction result from the binocular vision system is taken as a reference surface for slope estimation. The measured slope is then integrated into a height map by the zonal wavefront reconstruction. The overall reconstruction process provides an accurate reconstruction result without the need for prior surface information. Simulations and experiments were conducted to verify the feasibility of the proposed algorithms. A relative error of 0.59% or less is experimentally recorded in the step-size measurement of a stair-like object.

Eye Gaze Based 3D Triangulation for Robotic Bionic Eyes.

Three-dimensional (3D) triangulation based on active binocular vision has increasing amounts of applications in computer vision and robotics. An active binocular vision system with non-fixed cameras needs to calibrate the stereo extrinsic parameters online to perform 3D triangulation. However, the accuracy of stereo extrinsic parameters and disparity have a significant impact on 3D triangulation precision. We propose a novel eye gaze based 3D triangulation method that does not use stereo extrinsic parameters directly in order to reduce the impact. Instead, we drive both cameras to gaze at a 3D spatial point P at the optical center through visual servoing. Subsequently, we can obtain the 3D coordinates of P through the intersection of the two optical axes of both cameras. We have performed experiments to compare with previous disparity based work, named the integrated two-pose calibration (ITPC) method, using our robotic bionic eyes. The experiments show that our method achieves comparable results with ITPC.

A standard expression of underwater binocular vision for stereo matching

A camera in an underwater stereo vision system conforms to a multi-layer refraction model, and the perspective projection camera model is invalid. Because the imaging light is refracted, the corresponding pixel does not conform to the traditional polar constraint, and the stereo matching methods must conduct a two-dimensional search. In this paper, a standard expression is proposed to reduce the search space to one dimension. Based on a light field, any specific underwater binocular stereo vision is converted to a standard setup in which the axes of two cameras are parallel and the image is expressed as the light field, the direction information image and the position image. We found and proved that the direction information image conforms to the rule of corresponding rows. With this standard expression, the stereo matching is efficiently conducted and the 3D information can be computed from the disparity map and the position image. In the experiment, an underwater binocular stereo vision system is developed and the proposed method is compared with the traditional method. The results confirm that the proposed method has several advantages of conciseness, correctness, efficiency, and generality.

BINOCULAR VISUAL ENVIRONMENT PERCEPTION TECHNOLOGY FOR UNMANNED SURFACE VEHICLE

Abstract. Binocular vision system is an essential way for target localization in many fields, which has been widely used as payload of unmanned surface vehicles (USV). High resolution cameras, which can provide richer information, are utilized more often on a USV. This brings challenges of computing tremendous data for target detection and localization in real-time. In this paper, we propose an framework to automatically detect and localize target using high resolution binocular cameras for environment perception of USV. Instead of processing the whole image, the feature extraction and matching are executed within the target region of interest determined by a deep convolution network. Then the target can be localized using triangulation principle with calibrated binocular camera parameters. Experiments show that our proposed strategy can achieve both precise detection and high accurate localization results in real-time applications.

Research on obstacle detection and path planning based on visual navigation for mobile robot

Both obstacle and path planning are of significance for autonomous visual navigation of a mobile robot which is able to realize free movement and obstacle avoidance. In this paper, a scheme of obstacle detection which is based on binocular vision to identify obstacles is constructed and it is combined with improved A* algorithm for obstacle avoidance motion for mobile robot platform. First, a binocular vision system to detect the robot’s perception of the environment is used and the visual information is obtained. Then a Gaussian filter is used to reduce the noise in the image, and the grayscale image is obtained through the binocular vision system. The gray scale map is matched with SAD (Sum of Absolute Differences) in order to obtain the disparity image with comparison. Second, the location information of the obstacle is extracted through the environmental depth information contained in the disparity image, and the improved A* algorithm is used as the path planning algorithm in this paper, which can modify the path of the planning error. Furthermore, the detection experiment of the robot visual obstacle avoidance system was established and the optimal path was obtained. The examples show that, the obstacle avoidance system can complete the task of obstacle detection and give a better path for the mobile robot.

A Binocular MSCKF-Based Visual Inertial Odometry System Using LK Optical Flow

The odometry is an important part of intelligent mobile robots to achieve positioning and navigation functions. At present, the mainstream visual odometry locates only through the visual information obtained by camera sensors. Therefore, in the case of insufficient light, texture missing and camera jitter, the visual odometry is difficult to locate accurately. To solve the problem, we propose a binocular MSCKF-based visual inertial odometry system using Lucas-Kanade (LK) optical flow. Firstly, the Inertial Measurement Unit (IMU) is introduced to overcome the above problems. Moreover, LK optical flow algorithm is utilized to process the visual information obtained by the binocular camera, and MSCKF algorithm is employed to realize the fusion of visual information and inertial information, which improves the accuracy and efficiency of the visual inertial odometry system positioning. Finally, the proposed method is simulated on the European Robotics Challenge (EuRoc) dataset by Robot Operating System (ROS), and compared with two other advanced visual inertial odometry systems, ROVIO and MSCKF-mono. A large number of simulations verify that the proposed method can achieve accurate pose estimation, which is superior to the two existing advanced visual inertial odometry systems.

Cholinergic Modulation of Binocular Vision.

The endogenous neurotransmitter acetylcholine (ACh) is known to affect the excitatory/inhibitory (E/I) balance of primate visual cortex, enhancing feedforward thalamocortical gain while suppressing corticocortical synapses. Recent advances in the study of the human visual system suggest that ACh is a likely component underlying interocular interactions. However, our understanding of its precise role in binocular processes is currently lacking. Here we use binocular rivalry as a probe of interocular dynamics to determine ACh's effects, via the acetylcholinesterase inhibitor (AChEI) donepezil, on the binocular visual system. A total of 23 subjects (13 male) completed two crossover experimental sessions where binocular rivalry measurements were obtained before and after taking either donepezil (5 mg) or a placebo (lactose) pill. We report that enhanced cholinergic potentiation attenuates perceptual suppression during binocular rivalry, reducing the overall rate of interocular competition while enhancing the visibility of superimposition mixed percepts. Considering recent evidence that perceptual suppression during binocular rivalry is causally modulated by the inhibitory neurotransmitter GABA, our results suggest that cholinergic activity counteracts the effect of GABA with regards to interocular dynamics and may modulate the inhibitory drive within the visual cortex.SIGNIFICANCE STATEMENT Our research demonstrates that the cholinergic system is implicated in modulating binocular interactions in the human visual cortex. Potentiating the transmission of acetylcholine (ACh) via the cholinergic drug donepezil reduces the extent to which the eyes compete for perceptual dominance when presented two separate, incongruent images.

High-speed and robust infrared-guiding multiuser eye localization system for autostereoscopic display.

In order to localize the viewers' eyes, a high-speed and robust infrared-guiding multiuser eye localization system was fabricated in this paper for a binocular autostereoscopic display, which can project a pair of parallax images to corresponding eyes. The system is composed of a low-resolution thermal infrared camera, a pair of high-resolution left and right visible spectral cameras, and an industrial computer. The infrared camera and the left visible spectral camera, and the left and right visible spectral camera, can both form the binocular vision system. The thermal infrared camera can capture the thermography images. The left and right visible spectral cameras can capture the left and right visible spectral images, respectively. Owing to the temperature difference between the face and background, the features of the face in thermography images are prominent. We use the YOLO-V3 neural network to detect the viewers' faces in thermography images. Owing to the different features of the pseudo and real faces in the infrared spectral, in the thermography images, the pseudo-faces can be easily eliminated. According to the positions and sizes of potential bounding boxes of the detected faces in the thermography images, the industrial computer can be guided to determine the left candidate regions in the left visible spectral image. Then, the industrial computer can determine the right candidate regions in the right visible spectral image. In the left candidate regions, the industrial computer detects the faces and localize the eyes by using the SeetaFace algorithm. The template matching is performed between the left and right candidate regions to calculate the accurate distance between the viewer and the system. The average detection time of the proposed method is about 3-8 ms. Compared with traditional methods, the localization time is improved by 86.7%-90.1%. Further, the proposed method is hardly influenced by the pseudo-faces and the strong ambient light.

Grasping pose estimation for SCARA robot based on deep learning of point cloud

With the development of 3D measurement technology, 3D vision sensors and object pose estimation methods have been developed for robotic loading and unloading. In this work, an end-to-end deep learning method on point clouds, PointNetRGPE, is proposed to estimating the grasping pose of SCARA robot. In PointNetRGPE model, the point cloud and class number are fused into a point-class vector, and several PointNet-like networks are used to estimate the robot grasping pose, containing 3D translation and 1D rotation. Considering that rotational symmetry is very common in man-made and industrial environments, a novel architecture is introduced into PointNetRGPE to solve the pose estimation problem with rotational symmetry in the z-axis direction. Additionally, an experimental platform is built containing an industrial robot and a binocular stereo vision system, and a dataset with three subsets is set up. Finally, the PointNetRGPE is tested on the dataset, and the success rates of three subsets are 98.89%, 98.89%, and 94.44% respectively.

Anti-Shake HDR Imaging Using RAW Image Data

Camera shaking and object movement can cause the output images to suffer from blurring, noise, and other artifacts, leading to poor image quality and low dynamic range. Raw images contain minimally processed data from the image sensor compared with JPEG images. In this paper, an anti-shake high-dynamic-range imaging method is presented. This method is more robust to camera motion than previous techniques. An algorithm based on information entropy is employed to choose a reference image from the raw image sequence. To further improve the robustness of the proposed method, the Oriented FAST and Rotated BRIEF (ORB) algorithm is adopted to register the inputs, and a simple Laplacian pyramid fusion method is implanted to generate the high-dynamic-range image. Additionally, a large dataset with 435 various exposure image sequences is collected, which includes the corresponding JPEG image sequences to test the effectiveness of the proposed method. The experimental results illustrate that the proposed method achieves better performance in terms of anti-shake ability and preserves more details for real scene images than traditional algorithms. Furthermore, the proposed method is suitable for extreme-exposure image pairs, which can be applied to binocular vision systems to acquire high-quality real scene images, and has a lower algorithm complexity than deep learning-based fusion methods.

An improved artificial bee colony algorithm based on elite search strategy with segmentation application on robot vision system

SummaryAiming at accelerating the convergence speed and enhancing relative poor local search ability of the traditional artificial bee colony algorithm (ABC), this article introduces an ABC with a new elite search strategy. First, we propose a strategy of recording individuals with high performance. Then bees have more chances to learn from a real elite. In the onlooked bee phase, its updating equation is changed for having more opportunities to search in a valuable area. Furthermore, for saving the value of function evaluations, a new learning equation for the best onlooked bee is proposed. The image segmentation of a robot binocular stereo vision system is a key problem in mechanical robot vision system, but the computation time limits its application. The experimental results show that the proposed algorithm achieves better performance on 10 benchmark functions and the image segmentation problem of mechanical robot in comparison with several other state of the art algorithms.

Near-optimal combination of disparity across a log-polar scaled visual field.

The human visual system is foveated: we can see fine spatial details in central vision, whereas resolution is poor in our peripheral visual field, and this loss of resolution follows an approximately logarithmic decrease. Additionally, our brain organizes visual input in polar coordinates. Therefore, the image projection occurring between retina and primary visual cortex can be mathematically described by the log-polar transform. Here, we test and model how this space-variant visual processing affects how we process binocular disparity, a key component of human depth perception. We observe that the fovea preferentially processes disparities at fine spatial scales, whereas the visual periphery is tuned for coarse spatial scales, in line with the naturally occurring distributions of depths and disparities in the real-world. We further show that the visual system integrates disparity information across the visual field, in a near-optimal fashion. We develop a foveated, log-polar model that mimics the processing of depth information in primary visual cortex and that can process disparity directly in the cortical domain representation. This model takes real images as input and recreates the observed topography of human disparity sensitivity. Our findings support the notion that our foveated, binocular visual system has been moulded by the statistics of our visual environment.