Active Stereo Vision Research Articles

Optical imaging of cervical precancerous lesions based on active stereo vision and motion tracking

A 3-D optical imaging system based on active stereo vision and motion tracking is built to track the motion of patient and to register the time-sequenced images of cervix recorded during colposcopy. The imaging system is evaluated by examining human subjects in vivo before routine colposcopy examination procedures. The system tracks the motion of patient accurately. The temporal kinetics of the acetowhitening process in the area of interest can be quantitatively measured. The results demonstrate that the kinetics of acetowhitening may be potentially used for accurately differentiating the precancerous lesions from the normal and benign lesions, and grading the precancerous lesions.

An Active Stereo Vision System Based on Neural Pathways of Human Binocular Motor System

An active stereo vision system based on a model of neural pathways of human binocular motor system is proposed. With this model, it is guaranteed that the two cameras of the active stereo vision system can keep their lines of sight fixed on the same target object during smooth pursuit. This feature is very important for active stereo vision systems, since not only 3D reconstruction needs the two cameras have an overlapping field of vision, but also it can facilitate the 3D reconstruction algorithm. To evaluate the effectiveness of the proposed method, some software simulations are done to demonstrate the same target tracking characteristic in a virtual environment apt to mistracking easily. Here, mistracking means two eyes track two different objects separately. Then the proposed method is implemented in our active stereo vision system to perform real tracking task in a laboratory scene where several persons walk self-determining. Before the proposed model is implemented in the system, mistracking occurred frequently. After it is enabled, mistracking never occurred. The result shows that the vision system based on neural pathways of human binocular motor system can reliably avoid mistracking.

Optical imaging for medical diagnosis based on active stereo vision and motion tracking

This study aims to develop a novel imaging technique to improve the accuracy of the colposcolpic diagnosis of cervical cancer. An optical imaging system based on active stereo vision is built to measure the 3-D surface topology of cervix and track the motion of patient. The information of motion tracking are used to register the time-sequenced images of cervix recorded over the period of examination. The imaging system is evaluated by tracking the movements of cervix models. The results show that the error of 2-D image registration is 0.8 pixels, equivalent to the motion tracking error of 0.05 mm in the field-of-view. The imaging technique holds the promise to enable quantitative mapping of the acetowhitening kinetics over cervical surface for more accurate diagnosis of cervical cancer.

A COGNITIVE MODEL OF IMITATIVE DEVELOPMENT IN HUMANS AND MACHINES

Several algorithms and models have recently been proposed for imitation learning in humans and robots. However, few proposals offer a framework for imitation learning in noisy stochastic environments where the imitator must learn and act under real-time performance constraints. We present a novel probabilistic framework for imitation learning in stochastic environments with unreliable sensors. Bayesian algorithms, based on Meltzoff and Moore's AIM hypothesis for action imitation, implement the core of an imitation learning framework. Our algorithms are computationally efficient, allowing real-time learning and imitation in an active stereo vision robotic head and on a humanoid robot. We present simulated and real-world robotics results demonstrating the viability of our approach. We conclude by advocating a research agenda that promotes interaction between cognitive and robotic studies of imitation.

Active stereo vision-based mobile robot navigation for person tracking

In this paper, we propose a mobile robot architecture for person tracking, consisting of an active stereo vision module (ASVM) and a navigation module (NM). The first uses a stereo head equipped with a pan-tilt mechanism to track a moving target (sel

Active vision applications to cultural heritage acquisition and monitoring

Abstract This paper presents features and performances of a portable stereo active vision system (AVS), designed and built for data acquisition and monitoring of medium-scale monuments and environments of interest for cultural heritage. The research activity was partially carried out in the framework of PARNASO-SIINDA project 1 , a national project aimed at the development of new techniques for cultural heritage preservation, and the case-of-study of the activity was the Roman Theatre of Aosta (IT). Reliable data acquisition and monitoring of the surface alterations related to the monument degrade was a project demand that motivated the stereo vision approach because of its high spatial sampling rate, high accuracy and close correlation of shape and color measurements. The AVS system architecture is designed around three CCD cameras mounted on three independent pan-tilt stages under computer control. The cameras are equipped with different focal lens objectives, allowing the system to acquire both geometric and colorimetric information of an architectural surface (monuments, frescoes…) at different scales over areas of tens of square meters. Two cameras form a stereo pair and are equipped with long-focal objectives to sample a limited portion of the scene at high spatial resolution, while the third one is equipped with a wide-angle objective and it is intended for surveying the whole scene at low-resolution. Geometric measurements are carried out both by forward triangulation of the optical axes of the stereo pair during fixation of an object and by three-dimensional (3D) object reconstruction from the images of the stereo pair. The fixation process is performed by a PC-controlled image-based servo loop. The fixation approach supplies sparse but accurate 3D measurement in the AVS reference system, while the stereo acquisition of image pairs allows a dense 3D local reconstruction of the surface around the fixation point. Colorimetric measurements are performed at pixel level by processing the images of one of the stereo cameras, which is equipped with tristimulus filters. Results are supplied in terms of CIE-Lab components. A user-friendly interface for the system management allows performing several measurements automatically. On-site data processing allows the user to decide for further investigations immediately, if necessary, without expensive delays. The features of the AVS system, namely the working range of depths (2–10 m), the high spatial sampling rate, the accuracy and the close correlation of shape and color measurements, makes the system competitive with respect to other available instruments and techniques for data acquisition and monitoring of small and medium-scale works of art. In the following the AVS architecture is presented, and laboratory tests made to asses its performances are described. Results of the tests and of the acquisition campaign, made on the Roman Theatre of Aosta (IT), are reported and discussed in comparison with other techniques commonly employed in cultural heritage surveys.

Biologically motivated vergence control system using human-like selective attention model

We propose a new human-like vergence control method for an active stereo vision system. The proposed system uses a selective attention model to localize an interesting area in each camera. The selected object area in the master camera is compared with that in the slave camera to identify whether the two cameras find a same landmark. If the left and right cameras successfully find a same landmark, the implemented active vision system with two cameras focuses on the landmark. Using the motor encoder information, we can detect depth information automatically. Computer simulation and experimental results show that the proposed vergence control method is very effective in implementing the human-like active stereo vision system.

Optimised De Bruijn patterns for one-shot shape acquisition

Coded structured light is an optical technique based on active stereovision which allows shape acquisition. By projecting a suitable set of light patterns onto the surface of an object and capturing images with a camera, a large number of correspondences can be found and 3D points can be reconstructed by means of triangulation. One-shot techniques are based on projecting an unique pattern so that moving objects can be measured. A major group of techniques in this field define coloured multi-slit or stripe patterns in order to obtain dense reconstructions. The former type of patterns is suitable for locating intensity peaks in the image while the latter is aimed to locate edges. In this paper, we present a new way to design coloured stripe patterns so that both intensity peaks and edges can be located without loss of accuracy and reducing the number of hue levels included in the pattern. The results obtained by the new pattern are quantitatively and qualitatively compared to similar techniques. These results also contribute to a comparison between the peak-based and edge-based reconstruction strategies.

Development of a 3D Vision Range Sensor Using Equiphase Light Section Method

Many autonomous and industrial robots require three-dimensional (3D) environmental sensors. Stereovision or laser range finders are often used to detect distance to obstacles or targets. We propose 3D measurement system using active stereovision based on light section and phase shift. A special light source consisting of a rotating cylinder and a lamp projects planes of lights whose intensities vary sinusoidally over time. Each plane phase differs from those of other planes, so we distinguish which plane sectioned the target from the phase. Images from the video camera are processed into phases of each pixel that indicate distance to measured targets. This needs only simple calculation for phase detection and simultaneously operates multiple systems. The basic principle and expansion for practical use are proposed and experimental results are detailed using a trial, verifying the proposal’s effectiveness.

Active stereovision-based robot learning control for object tracking, fixating and grasping

In this paper, an active stereovision-based control approach is proposed for a robot to track, fixate, and grasp an object in an unknown environment. First, the functional mapping relationships between those joint angles of the active stereovision system and the three-dimensional (3-D) coordinates of the object are derived and expressed in the workspace frame. Second, two feed-forward neural networks are used to learn those functional mapping relationships, which are used for the robot tracking, fixating, and grasping control. Third, the present approach is verified by experiments based on the active stereovision system which is installed in the end-effecter of the robot. Last, the experimental results confirm the effectiveness of the present approach .

A Control Technique for 3D Object Tracking on Active Stereo Vision Robot

The paper describes a control technique for an active stereo vision robot. For the purpose of achieving both rigid tracking and accurate estimating, the proposed approach unifies the image-based high gain PD control law for tracking and the mechanical impedance control law for stable motion of the base axis.

3D Position Detection of A Moving Object in Active Stereo Vision System with Multi-DOF Motion

This paper describes a strategy to improve the accuracy of 3D position detection of a moving object. Stereo vision is one of powerful methods to detect a target position. However the measurement accuracy depends on the configuration of a target object and two cameras of vision system. In this case, these exist a camera configuration that enables to improve the measurement accuracy of object position. On the other hand, the camera system requires a multi-DOF motion to realize fast tracking of moving object. To address the above issues, a stereo vision system with multi-DOF motion is developed in this research. The constructed vision system has two hand-eye system that are controlled independently. In the proposed strategy, two performance indices are considered to determine the manipulator motion of hand-eye system. One is camera configuration index to evaluate the performance of the position detection by the stereo vision. The other is a manipulability measure of the manipulator in the hand-eye system. Considering both indices to determine the motion of hand-eye system, the tracking performance of stereo vision system is improved. Numerical and experimental results show the availability of the proposal method.

Three-dimensional rigid motion estimation using genetic algorithms from an image sequence in an active stereo vision system

This paper proposes a method for estimating the three-dimensional (3D) rigid motion parameters from an image sequence of a moving object. The 3D surface measurement is achieved using an active stereovision system composed of a camera and a light projector, which illuminates the objects to be analyzed by a pyramid-shaped laser beam. By associating the laser rays with the spots in the two-dimensional image, the 3D points corresponding to these spots are reconstructed. Each image of the sequence provides a set of 3D points, which is modeled by a B-spline surface. Therefore, estimating the 3D motion between two images of the sequence boils down to matching two B-spline surfaces. We consider the matching environment as an optimization problem and find an optimal solution using genetic algorithms. A chromosome is encoded by concatenating seven binary coded parameters, the angle, and the three components of the rotation vector axis, and the three translation vector components. We have defined an original fitness function for calculating the similarity measure between two surfaces. Experimental results with real and synthetic image sequences are presented to show the effectiveness and the robustness of the method.

Camera Calibration and 3-D Measurement with an Active Stereo Vision System for Handling Moving Objects

In this paper, we propose a fast, easy camera calibration and 3-D measurement method with an active stereo vision system for handling moving objects whose geometric models are known. We use stereo cameras that change direction independently to follow moving objects. To gain extrinsic camera parameters in real time, a baseline stereo camera (parallel stereo camera) model and projective transformation of stereo images are used by considering epipolar constraints. To make use of 3-D measurement results for a moving object, the manipulator hand approaches the object. When the manipulator hand and object are near enough to be situated in a single image, very accurate camera calibration is executed to calculate the manipulator size in the image. Our calibration is simple and practical because it does not need to calibrate all camera parameters. The computation time for real-time calibration is not large because we need only search for one parameter in real time by deciding the relationship between all parameters in advance. Our method does not need complicated image processing or matrix calculation. Experimental results show that the accuracy of 3-D reconstruction of a cubic box whose edge is 60 mm long is within 1.8 mm when the distance between the camera and the box is 500 mm. Total computation time for object tracking, camera calibration, and manipulation control is within 0.5 seconds.

3-D shape reconstruction in an active stereo vision system using genetic algorithms

The recovery of 3-D shape information (depth) using stereo vision analysis is one of the major areas in computer vision and has given rise to a great deal of literature in the recent past. The widely known stereo vision methods are the passive stereo vision approaches that use two cameras. Obtaining 3-D information involves the identification of the corresponding 2-D points between left and right images. Most existing methods tackle this matching task from singular points, i.e. finding points in both image planes with more or less the same neighborhood characteristics. One key problem we have to solve is that we are on the first instance unable to know a priori whether a point in the first image has a correspondence or not due to surface occlusion or simply because it has been projected out of the scope of the second camera. This makes the matching process very difficult and imposes a need of an a posteriori stage to remove false matching. In this paper we are concerned with the active stereo vision systems which offer an alternative to the passive stereo vision systems. In our system, a light projector that illuminates objects to be analyzed by a pyramid-shaped laser beam replaces one of the two cameras. The projections of laser rays on the objects are detected as spots in the image. In this particular case, only one image needs to be treated, and the stereo matching problem boils down to associating the laser rays and their corresponding real spots in the 2-D image. We have expressed this problem as a minimization of a global function that we propose to perform using Genetic Algorithms (GAs). We have implemented two different algorithms: in the first, GAs are performed after a deterministic search. In the second, data is partitioned into clusters and GAs are independently applied in each cluster. In our second contribution in this paper, we have described an efficient system calibration method. Experimental results are presented to illustrate the feasibility of our approach. The proposed method yields high accuracy 3-D reconstruction even for complex objects. We conclude that GAs can effectively be applied to this matching problem.

Miniaturized three-dimensional endoscopic imaging system based on active stereovision.

A miniaturized three-dimensional endoscopic imaging system is presented. The system consists of two imaging in channels that can be used to obtain an image from an object of interest and to project as tructured light onto the imaged object to measure the surface topology. The structured light was generated with a collimated monochromatic light source and a holographic binary phase grating. The imaging and projection channels were calibrated by use of a modified pinhole camera. The surface profile was extracted by use of triangulation between the projected feature points and the two channel ofthe endoscope. The imaging system was evaluated in three-dimensional measurements of several objects with known geometries. The results show that surface profiles of the objects with different surfaces and dimensions can be obtained at high accuracy. The in vivo measurements at tissue sites of human skin and an oral cavity demonstrated the potential of the technique for clinical applications.

OPL-3D: A novel, portable optical digitizer for fast acquisition of free-form surfaces

The paper presents OPL-3D, a novel, powerful three-dimensional optical digitizer based on structured, incoherent light illumination, developed in our laboratory for noncontact acquisition and digitization of free forms in space. The system exploits active stereovision using time multiplexing based on a combination of Gray code and phase-shifting techniques. The system has been specifically designed for applications to reverse engineering and rapid prototyping of free-form shapes, as well as for applications to measurement and quality control. Suitable estimation of the measurement parameters, as well as specifically developed two-dimensional image analysis, allows us to retrieve very dense point clouds in a few seconds. Special care has been devoted to the implementation of easy-to-use fast calibration procedures, and to enhancement of the flexibility of the system to the measuring problem as well as its portability. A kernel for elaboration of the views and for their alignment is provided. OPL-3D exhibits low-measurement uncertainty (120 μm) over large measurement areas (450 mm×340 mm), linearly scalable in the case of smaller areas. The output formats of the data files are fully compatible with the formats commonly used by elaboration environments dedicated to the production of polygonal models and to computer-aided design models of the shapes. The performance of OPL-3D has been tested in a number of applications, ranging from industry to biomedicine and virtual reality.

A New Approach to Automatic Reconstruction of a 3-D World Using Active Stereo Vision

In this paper, we propose a new automatic approach to reconstructing 3-D environments using an active binocular head. To efficiently store and access the depth estimates, we propose the use of an inverse polar octree which can transform both unbounded depth estimates and unbounded estimation errors into a bounded 3-D space with appropriate resolution. The depth estimates are computed by using the asymptotic Bayesian estimation method. Estimated depth values are then smoothed by using discontinuity-preserving Markov random fields. The path of the local motion required by the asymptotic Bayesian method is determined online automatically to reduce the ambiguity of stereo matching. Rules for checking the consistency between the new observation and the previous observations have been developed to properly update the inverse polar octree. Experimental results showed that the proposed approach is very promising for automatic generation of 3-D models which can be used for rendering a 3-D scene in a virtual reality system.

Active visual localization for multiple inspection robots

In the routine inspection of industrial or other areas, teams of robots with various sensors could operate together to great effect, but require reliable, accurate and flexible localization capabilities to be able to move around safely. We demonstrate accurate localization for an inspection team consisting of a robot with stereo active vision and its blind companion with an active lighting system, and show that in this case a single sensor can be used for measuring the position of known or unknown scene features, measuring the relative location of the two robots and actually carrying out an inspection task.

SAVI: an actively controlled teleconferencing system

A Stereo Active Vision Interface (SAVI) is introduced which detects frontal faces in real world environments and performs particular active control tasks dependent on hand gestures given by the person the system attends to. The SAVI system is thought of as a smart user interface for teleconferencing, telemedicine, and distance learning applications. To reduce the search space in the visual scene the processing is started with the detection of connected skin colour regions applying a new radial scanline algorithm. Subsequently, in the most salient skin colour region facial features are searched for while the skin colour blob is actively kept in the centre of the visual field of the camera system. After a successful evaluation of the facial features the associated person is able to give control commands to the system. For this contribution only visual control commands are investigated but there is no limitation for voice or any other commands. These control commands can either effect the observing system itself or any other active or robotic system wired to the principle observing system via TCP/IP sockets. The system is designed as a perception-action-cycle (PAC), processing sensory data of different kinds and qualities. Both the vision module and the head motion control module work at frame rate on a PC platform. Hence, the system is able to react instantaneously to changing conditions in the visual scene.