Binocular Tracking Research Articles

Gaze distance detection by binocular tracking for automatic zoom glasses

Gaze distance detection by binocular tracking for automatic zoom glasses

Energy-efficient design of a presbyopia correction wearable powered by mobile GPUs and FPGAs

This paper presents an energy-efficient design and evaluation of a novel portable device for the automatic correction of presbyopia in human eyes driven by the use of opto-electronic lenses and based on the dynamic pupil response of the subject. Due to the wearable nature of the proposed Dynamic Auto–Accommodation Glasses, in addition to the real-time requirement, an energy-efficient implementation is critical for the success of the device. In this work, the binocular pupil tracking of a subject, followed by the calculation of the eyes’ vergence, and the control of a pair of opto-electronic lenses are implemented and evaluated on several hardware platforms, including two mobile GPU/SoCs, a high-end FPGA, a low-cost FPGA, and a desktop GPU (as a reference point). The pupil tracking algorithm has been parallelized, applying different platform-specific optimizations for each case, to design a fast yet energy-efficient wearable. The hardware platforms have been evaluated to determine which one is the most appropriate for the presbyopia correction task. The experimental results show that the most energy-efficient platform is a mobile GPU (Samsung Exynos 8890) capable of processing frames at 0.016 Joules/frame, still allowing real-time processing (24 frames/sec).

A Normative Study of Objective Measures of Disparity Vergence and Saccades in Children 9 to 17 Years Old

This study establishes normative data for objective outcome measures of vergence and saccade eye movements for the pediatric population. These data should facilitate future clinical trial design. This study was designed to establish normative data for objective measures of disparity vergence and saccades in children between the ages 9 and 17 years using an objective binocular eye movement tracking system. Participants (aged 9 to 17 years) had a vision examination including refraction, accommodative, and binocular vision testing. Eligibility criteria included 20/25 visual acuity with best correction, normal accommodation, and binocular vision. The ISCAN RK-826PCI binocular tracking system (ISCAN, Woburn, MA) was used to objectively record horizontal, symmetrical disparity vergence, and saccadic eye movements. Parameters assessed included peak velocity, time to peak velocity, latency, and response amplitude for both disparity vergence and saccades. One hundred eighteen participants were recruited (54.94% female; mean age, 13.5 years), and 77.1% (91/118) of the participants completed the assessment with usable data. A sample of the normative data included peak velocity (°/s), which had a mean ± standard deviation of 25.4 ± 2.9, 22.0 ± 3.0, 225 ± 16.7, and 332.5 ± 20.5 for 4° convergence, 4° divergence, 5° saccades, and 10° saccades, respectively. The mean ± standard deviation for the latency (seconds) measures were 0.28 ± 0.1, 0.28 ± 0.16, 0.23 ± 0.05, and 0.23 ± 0.05 for 4° convergence, 4° divergence, 5° saccades, and 10° saccades, respectively. Normative data enable researchers to have benchmark results for comparison with patient populations with binocular dysfunction. These objective disparity vergence measures can serve as outcome measures in future clinical trials to assess the effectiveness of therapeutic interventions by determining whether post-treatment results are similar to normal data.

Eye movements in chameleons are not truly independent - evidence from simultaneous monocular tracking of two targets.

Chameleons perform large-amplitude eye movements that are frequently referred to as independent, or disconjugate. When prey (an insect) is detected, the chameleon's eyes converge to view it binocularly and 'lock' in their sockets so that subsequent visual tracking is by head movements. However, the extent of the eyes' independence is unclear. For example, can a chameleon visually track two small targets simultaneously and monocularly, i.e. one with each eye? This is of special interest because eye movements in ectotherms and birds are frequently independent, with optic nerves that are fully decussated and intertectal connections that are not as developed as in mammals. Here, we demonstrate that chameleons presented with two small targets moving in opposite directions can perform simultaneous, smooth, monocular, visual tracking. To our knowledge, this is the first demonstration of such a capacity. The fine patterns of the eye movements in monocular tracking were composed of alternating, longer, 'smooth' phases and abrupt 'step' events, similar to smooth pursuits and saccades. Monocular tracking differed significantly from binocular tracking with respect to both 'smooth' phases and 'step' events. We suggest that in chameleons, eye movements are not simply 'independent'. Rather, at the gross level, eye movements are (i) disconjugate during scanning, (ii) conjugate during binocular tracking and (iii) disconjugate, but coordinated, during monocular tracking. At the fine level, eye movements are disconjugate in all cases. These results support the view that in vertebrates, basic monocular control is under a higher level of regulation that dictates the eyes' level of coordination according to context.

Binocular Eye Movement Control and Motion Perception: What Is Being Tracked?

We investigated under what conditions humans can make independent slow phase eye movements. The ability to make independent movements of the two eyes generally is attributed to few specialized lateral eyed animal species, for example chameleons. In our study, we showed that humans also can move the eyes in different directions. To maintain binocular retinal correspondence independent slow phase movements of each eye are produced. We used the scleral search coil method to measure binocular eye movements in response to dichoptically viewed visual stimuli oscillating in orthogonal direction. Correlated stimuli led to orthogonal slow eye movements, while the binocularly perceived motion was the vector sum of the motion presented to each eye. The importance of binocular fusion on independency of the movements of the two eyes was investigated with anti-correlated stimuli. The perceived global motion pattern of anti-correlated dichoptic stimuli was perceived as an oblique oscillatory motion, as well as resulted in a conjugate oblique motion of the eyes. We propose that the ability to make independent slow phase eye movements in humans is used to maintain binocular retinal correspondence. Eye-of-origin and binocular information are used during the processing of binocular visual information, and it is decided at an early stage whether binocular or monocular motion information and independent slow phase eye movements of each eye are produced during binocular tracking.

THE HKUST MULTIMAP SYSTEM FOR ACTIVE VISION

This paper describes the HKUST MultiMap system, a hardware system we have developed to support the computation and integration of large numbers of neurally inspired feature maps at frame rates. The computed maps could serve as the basis for a generic image representation to enable robots to perform a variety of different tasks. In order to support the computation of feature representations with increasing complexity, the system can split the processing among a scalable number of processors, each on a different printed circuit board. We describe the hardware design of this board, as well as the communication protocol between different boards. Our experimental results on a four board system validate the performance of the proposed communication protocol. As an example of the applications enabled by this system, we describe a robotic binocular tracking system built by using it.

Physics-Based Person Tracking Using the Anthropomorphic Walker

We introduce a physics-based model for 3D person tracking. Based on a biomechanical characterization of lower-body dynamics, the model captures important physical properties of bipedal locomotion such as balance and ground contact. The model generalizes naturally to variations in style due to changes in speed, step-length, and mass, and avoids common problems (such as footskate) that arise with existing trackers. The dynamics comprise a two degree-of-freedom representation of human locomotion with inelastic ground contact. A stochastic controller generates impulsive forces during the toe-off stage of walking, and spring-like forces between the legs. A higher-dimensional kinematic body model is conditioned on the underlying dynamics. The combined model is used to track walking people in video, including examples with turning, occlusion, and varying gait. We also report quantitative monocular and binocular tracking results with the HumanEva dataset.

Binocular tracking with ultrasonic sensors

This paper proposes an Image-Based Visual Servo control (IBVS) method that integrates the kinematics of both a robotic binocular head and cameras with the dynamics of the binocular head and uses a fuzzy sliding mode control algorithm. The proposed method utilises ultrasonic sensors to supply distance information to the visual servo system. In image processing, an attention window is estimated to mark the 3D moving object: the size of the window can increase or decrease according to the size of the object. Finally, two experiments based on the multiple-sensor binocular head are performed to verify the theoretical results.

High-Performance Object Tracking and Fixation With an Online Neural Estimator

Vision-based target tracking and fixation to keep objects that move in three dimensions in view is important for many tasks in several fields including intelligent transportation systems and robotics. Much of the visual control literature has focused on the kinematics of visual control and ignored a number of significant dynamic control issues that limit performance. In line with this, this paper presents a neural network (NN)-based binocular tracking scheme for high-performance target tracking and fixation with minimum sensory information. The procedure allows the designer to take into account the physical (Lagrangian dynamics) properties of the vision system in the control law. The design objective is to synthesize a binocular tracking controller that explicitly takes the systems dynamics into account, yet needs no knowledge of dynamic nonlinearities and joint velocity sensory information. The combined neurocontroller-observer scheme can guarantee the uniform ultimate bounds of the tracking, observer, and NN weight estimation errors under fairly general conditions on the controller-observer gains. The controller is tested and verified via simulation tests in the presence of severe target motion changes.

A 3D Feature-Based Binocular Tracking Algorithm

This paper presents a 3D feature-based binocular tracking algorithm for tracking crowded people indoors. The algorithm consists of a two stage 3D feature points grouping method and a robust 3D feature-based tracking method. The two stage 3D feature points grouping method can use kernel-based ISODATA method to detect people accurately even though the part or almost full occlusion occurs among people in surveillance area. The robust 3D feature-based Tracking method combines interacting multiple model (IMM) method with a cascade multiple feature data association method. The robust 3D feature-based tracking method not only manages the generation and disappearance of a trajectory, but also can deal with the interaction of people and track people maneuvering. Experimental results demonstrate the robustness and efficiency of the proposed framework. It is real-time and not sensitive to the variable frame to frame interval time. It also can deal with the occlusion of people and do well in those cases that people rotate and wriggle.

Binocular motion tracking by gaze fixation control and three-dimensional shape reconstruction

It is an easy task for the human visual system to gaze continuously at an object moving in three-dimensional (3-D) space. While tracking the object, human vision seems able to comprehend its 3-D shape with binocular vision. We conjecture that, in the human visual system, the function of comprehending the 3-D shape is essential for robust tracking of a moving object. In order to examine this conjecture, we constructed an experimental system of binocular vision for motion tracking. The system is composed of a pair of active pan-tilt cameras and a robot arm. The cameras are for simulating the two eyes of a human while the robot arm is for simulating the motion of the human body below the neck. The two active cameras are controlled so as to fix their gaze at a particular point on an object surface. The shape of the object surface around the point is reconstructed in real-time from the two images taken by the cameras based on the differences in the image brightness. If the two cameras successfully gaze at a single point on the object surface, it is possible to reconstruct the local object shape in real-time. At the same time, the reconstructed shape is used for keeping a fixation point on the object surface for gazing, which enables robust tracking of the object. Thus these two processes, reconstruction of the 3-D shape and maintaining the fixation point, must be mutually connected and form one closed loop. We demonstrate the effectiveness of this framework for visual tracking through several experiments.

Self-Adaptive Output Tracking with Applications to Active Binocular Tracking

In this article we present a neurally-inspired self-adaptive active binocular tracking scheme and an efficient mathematical model for online computation of desired binocular-head trajectories. The self-adaptive neural network (NN) model is general and can be adopted in output tracking schemes of any partly known robotic systems. The tracking scheme ingeniously combines the conventional Resolved Velocity Control (RVC) technique and an adaptive compensating NN model constructed using SoftMax basis functions as nonlinear activation function. Desired trajectories to the servo controller are computed online by the use of a suitable linear kinematics mathematical model of the system. Online weight tuning algorithm guarantees tracking with small errors and error rates as well as bounded NN weights.

Subpixel resolution binocular visual tracking using analog VLSI vision sensors

This paper describes the application of an analog VLSI vision sensor to active binocular tracking. The sensor outputs are used to control the vergence angles of the two cameras and the tilt angle of the head so that the center pixels of the sensor arrays image the same point in the environment. One distinguishing feature of the sensor used here is the possibility to resolve target displacement with subpixel resolution via a phase-based algorithm, which integrates information over multiple pixels.

Binocular tracking: integrating perception and control

Presents an active binocular tracking system using log-polar images with contributions in both the perceptual and control aspects. The control part is based on the visual servoing framework, including kinematics and dynamics. We introduce a fixation constraint that simplifies the tracking problem by decoupling the visual kinematics and allowing us to express system dynamics in image coordinates. Simple dynamic controllers are designed for each degree of freedom directly from image features. In the perceptual part, we use a space variant sensor that emphasizes the center of the visual field (log-polar geometry). We present a disparity estimation algorithm for log-polar images and provide a theoretical analysis to illustrate the advantages of using space variant images. The overall system is implemented in the Medusa binocular head without any specific processing hardware. The use of log-polar images allows real-time performance (50 Hz). Tracking experiments are presented to illustrate system performance with different control strategies and objects of different shapes and motions.

Visual behaviours for binocular tracking

This paper presents a binocular tracking system based on the cooperation of visual behaviours. Biologically motivated behaviours, vergence and pursuit, are integrated as parallel and complementary processes in the tracking system. Their low internal coupling simplifies not only system design and control but also the acquisition of perceptual information. The use of a space variant image representation and low-level visual cues as feedback signals in a closed loop control architecture, allow real-time and reliable performance for each behaviour, despite the low precision of the algorithms and modelling errors. The overall system is implemented in a stereo head running at real-time (25 Hz), without any specific processing hardware. Results are presented for objects of different shapes and motions, illustrating that tracking can be robustly achieved by the cooperation of purposively designed behaviours, tuned to specific subgoals.

Binocular Tracking based on Retino-Cortical Mapping

This paper describes a new binocular tracking method using Log Polar Mapping (LPM) which approximately represents the mapping of the retina into the visual cortex in primate vision : Using LPM makes it possible not only to obtain both a high central resolution and a wide field of view, but also to significantly reduce processing image data. In this paper, LPM is performed in software by lookup table method. Our tracking method utilizes zero disparity filter (ZDF) for extracting the target object and virtual horopter method for estimating binocular disparities, respectively. The performance of both target extraction and disparity estimation is improved in comparison with the conventional methods, by using LPM. Some experimental results are also shown to demonstrate the effectiveness of the proposed method.

A Model for Binocular Vision

Publisher Summary This chapter describes a model for binocular vision. The present study aims to develop a mathematical model for the entire process of binocular vision and the tracking of a moving object using binocular vision. A model of this type would be useful in the areas of robotics and the diagnosing of abnormalities in human vision. This model for binocular vision and tracking involves the following stages. A simplified physical model of the eye is constructed and is used to develop a mathematical representation for the motion of the eye. This model is used to simulate a single eye tracking an object in two dimensions. This object is assumed to be a point object. The eye model is then expanded to the case of binocular vision, where there are two eyes tracking the object rather than one. Once the tracking model for the vision problem has been obtained, the question of observability is considered. A model for monocular vision with a single eye has been developed.

仮想ホロプタを用いた実時間両眼追跡

We present an efficient method to fixate a binocular gaze point on an object moving around in a complicated environment. Assuming the object is in the vicinity of the gaze point, the image features of the object can be isolated from others by using zero disparity filtering (ZDF). The object position in the depth direction, however, can not be estimated from such isolated features. In order to get the depth information simultaneously with the isolation by ZDF processing, we introduce the novel localization technique based on the idea of “virtual horopter”. The proposed method is implemented on our active vision system with much reduced programming efforts and resources in comparing with conventional methods. Several experiments demonstrate that the binocular tracking based on the proposed method works for various shape objects and in complicated environments.

Vergence control of central and peripheral disparities

The results of previous studies using either small or large target configurations suggest that open-loop (OL) vergence response dynamics may depend on the peripheral extent of the target. To investigate systemically the effect of peripheral target extent on dynamic vergence control, OL vergence responses to central, peripheral, and central-plus-peripheral targets were recorded. Open-loop disparity stimuli included steps, ramps, and sinusoids. It was found that the OL step and ramp responses to the central target were more rapid and exhibited significantly more step and multiple-step movements than those for either the peripheral or the central-plus-peripheral targets. During OL ramp stimulation, the maximum disparity for which tracking could be maintained was largest for the peripheral target. Sinusoidal responses showed the greatest gain for the central target. For all three types of stimuli, responses to the central-plus-peripheral target showed characteristics that were somewhat between those for the central and the peripheral targets. These results resolve some previously conflicting findings about open-loop vergence dynamics and suggest that large peripheral disparity stimulation initiates slow fusion of a visual scene, whereas small central disparity stimulation produces more rapid movements for precise binocular tracking of targets moving in depth.

Processing delays in amblyopic eyes: evidence from saccadic latencies.

Saccadic latencies were measured in amblyopes with constant strabismus, amblyopes without strabismus, and intermittent strabismics with or without amblyopia. Subjects tracked a small spot of light, either monocularly or binocularly, which moved with random horizontal step displacements of 0.25--8.5 deg over the central field. Increased saccadic latencies were observed in the amblyopic eyes of 6 of 11 subjects, with or without strabismus; saccadic latencies were similar in each eye of 2 subjects having intermittent strabismus without amblyopia. Amblyopia was a necessary condition for increased saccadic latencies and not strabismus. Evidence for normal motor control of eye movements in amblyopic subjects is as follows: normal saccadic durations in the amblyopic eyes, normal saccadic-latency distribution curves for binocular tracking and monocular tracking with the nonamblyopic eyes, and synchronous movements of the 2 eyes. Our results are interpreted in terms of a processing delay in the sensory pathways leading from the central region of the amblyopic eye to the centers involved in saccadic initiation.