Visual Target Tracking Research Articles

Scanning and tracking with independent cameras—a biologically motivated approach based on model predictive control

This paper presents a framework for visual scanning and target tracking with a set of independent pan-tilt cameras. The approach is systematic and based on Model Predictive Control (MPC), and was inspired by our understanding of the chameleon visual system. We make use of the most advanced results in the MPC theory in order to design the scanning and tracking controllers. The scanning algorithm combines information about the environment and a model for the motion of the target to perform optimal scanning based on stochastic MPC. The target tracking controller is a switched control combining smooth pursuit and saccades. Min-Max and minimum-time MPC theory is used for the design of the tracking control laws. We make use of the observed chameleon's behavior to guide the scanning and the tracking controller design procedures, the way they are combined together and their tuning. Finally, simulative and experimental validation of the approach on a robotic chameleon head composed of two independent Pan-Tilt cameras is presented.

Sensor fusion-based visual target tracking for autonomous vehicles with the out-of-sequence measurements solution

In this paper, a novel algorithm is proposed for the visual target tracking by Autonomous Guide Vehicles (AGV). This paper proposes a sensor data fusion system to estimate the dynamics of the target. Optical flow vectors, colour features, stereo pair disparities are used as the visual features while the vehicle’s inertial measurements are used to estimate the stereo cameras’ motion. The algorithm estimates the velocity and position of the target which is then used by the vehicle to track the target. In this sensor data fusion-based tracking system, the measurements from the same target can arrive out of sequence. This is called the “Out-Of-Sequence” Measurements (OOSM) problem. Thus the resulting problem - how to update the current state estimate with an “older” measurement - needs to be solved. In this paper the 1- step-lag OOSM solution from Bar-Shalom is applied for the Extended Kalman Filter-based target-state estimation. The performance of the proposed tracking algorithm with the OOSM solution is demonstrated through extensive experimental results.

Collaborative signal processing for target tracking in distributed wireless sensor networks

Target tracking, especially visual target tracking, in complex situations is challenging, which is always performed in single-view system. Because of the conflict between resolution and tracking range, however, single-view tracking is not robust and accurate. This paper presents a distributed multi-view tracking system using collaborative signal processing (CSP) in distributed wireless sensor networks (DWSNs). In the proposed tracking system, target detection and classification algorithms are based on single-node processing and target tracking is performed in sink node, whereas target localization algorithm is carried out by CSP between multisensor. For conquering the disadvantages of client/server based centralized data fusion, a progressive distributed data fusion are proposed. Finally, an indoor target tracking experiment is illustrated, and then tracking performance, execution time and energy consumption of progressive distributed data fusion are compared with client/server based centralized data fusion. Experimental results demonstrate that the CSP based distributed multi-view tracking system in DWSNs can accomplish multi-target extraction, classification, localization, tracking and association quickly and accurately with little congestion, energy consumption and execution time.

Postural coordination modes and transition: dynamical explanations

While research to date has been successful in quantifying postural behaviour, this paper examines the causes of transition between postural coordination mode using dynamical variables and, by inference, efficient control strategies underlying postural behaviour. To this end, six subjects in bipedal stance were instructed to maintain a constant distance between their head and a visual target that oscillated along the line of sight. Within sessions, participants were exposed to gradual changes in increasing target motion frequency. Kinematic results showed a sudden transition between in-phase and anti-phase postural coordination modes in visual target tracking. The dynamical analysis pointed out that (1) the center of pressure (CoP) position parameter is a crucial parameter in the determination of the adopted coordination mode, (2) the change occurred in response to limits bordered by the system: the interaction between equilibrium constraints (A/P displacements of CoP), physiological limits (net joint moments) support the emergence of different postural behaviours and, (3) finally, the anti-phase mode presents a better distribution of muscular moment between hip and ankle joints and is more effective to achieve high frequency oscillations with limited CoP displacements.

6.3 Segmental stabilization during upright standing underfixation and tracking of visual target

6.3 Segmental stabilization during upright standing underfixation and tracking of visual target

Optimization model predictions for postural coordination modes

This paper examines the ability of the dynamic optimization model to predict changes between in-phase and anti-phase postural modes of coordination and to evaluate influence of two particular environmental and intentional constraints on postural strategy. The task studied was based on an experimental paradigm that consisted in tracking a target motion with the head. An original optimal procedure was developed for cyclic problems to calculate hip and ankle angular trajectories during postural sway with a minimum torque change criterion. Optimization results give a good description of the sudden bifurcation phase between in-phase and anti-phase postural coordination modes in visual target tracking. Transition frequency and predicted effects of environmental and intentional constraints are also in line with experimental observations described in existing literature. In particular, these investigations pointed out that postural planning process can be related to the minimization of a dynamic cost criterion with an equilibrium constraint. In conclusion, the optimization technique is well suited for the prediction of postural modes of coordination and seems to offer many opportunities for better comprehension of neuromuscular movement control.

HAND MOVEMENT IMPROVEMENT ON VISUAL TARGET TRACKING BY MODEL-BASED COMPENSATOR

Brain dysfunction in the cerebral cortex, cerebellum and/or basal ganglia causes serious movement disorders such as cerebellar ataxia, Parkinson disease and so on. Compensation of hand movement by adding an external force will recover the motor function and will be helpful for improving the quality of patients' daily life. This paper proposes a method for compensating human hand movement on visual target tracking by adding an assistant force. Mathematical model was obtained from the measurement data of visual target tracking for each subject, and the compensator was constructed based on the model. Effectiveness of the compensation method was investigated through the experiment.

Modeling head tracking of visual targets

Control of the head involves somatosensory, vestibular, and visual feedback. The dynamics of these three feedback systems must be identified in order to gain a greater understanding of the head control system. We have completed one step in the development of a head control model by identifying the dynamics of the visual feedback system. A mathematical model of human head tracking of visual targets in the horizontal plane was fit to experimental data from seven subjects performing a visual head tracking task. The model incorporates components based on the underlying physiology of the head control system. Using optimization methods, we were able to identify neural processing delay, visual control gain, and neck viscosity parameters in each experimental subject.

Anticipatory control of hand and eye movements in humans during oculo-manual tracking.

Anticipatory activity of hand and eye has been examined during oculo-manual tracking of a constant velocity visual target with a hand cursor. Both target and cursor were presented briefly (< 480 ms), but repeatedly, at regular inter-stimulus intervals (ISI). In Expt 1, the build-up of hand and eye responses was examined for target velocities varying from 10-40 deg x s(-1) with an ISI of 2.4 s. The velocity 100 ms after target onset (i.e. prior to visual feedback) for both hand and eye (V100) progressively increased over the first four presentations but then attained a steady state (SS). SS V100 values for eye and hand increased in proportion to target velocity and were thus predictive of forthcoming movement. Hand velocity exceeded eye velocity but both exhibited similar anticipatory trajectories. In Expt 2, target velocity was constant (40 deg x s(-1)) but ISI varied from 0.48-3.74 s. Subjects made anticipatory eye movements for all ISIs but hand movements were often reactive at the longest ISI. If the target failed to appear as expected, subjects initiated predictive hand and eye responses with timing appropriate for the prevailing ISI. In Expt 3, predictive responses were compared with responses to randomised presentation. Peak hand velocity was greater in the randomised mode than in the predictive condition, whereas the converse was true for peak eye velocity. This difference is discussed in terms of the mechanisms of positional error correction in hand and eye. Results provide evidence of similar anticipatory mechanisms in hand and eye, using storage of velocity and timing to achieve rapid prediction of target motion.

Compensation of hand movement for patients by assistant force: relationship between human hand movement and robot arm motion.

As some functional diseases in the brain, such as cerebellum dysfunction and Parkinson's disease, cause the disability related to human movement control, a compensation method was developed for improving the performance of hand movement. The compensation can be carried out by adding assistant force, which is generated from artificial equipment attached to a human arm. From the experiment of visual target tracking, the tracking trajectories recorded from both healthy persons and patients with movement disability were analyzed. It was found that the tracking trajectories were represented sufficiently by a dynamic model of a robot arm in which the differences between healthy persons and patients were characterized by the model parameters. Based on the model, it was demonstrated that the hand movement of patients could be improved by introducing an appropriate compensation. The effectiveness of the proposed compensation method was verified from a simulation study of a robot arm. The design of artificial equipment for compensating the hand movement was also presented and discussed.

Assistant force compensation for hand movement of patients by using exogenous signals and/or neurophysiological signals

Because functional diseases of the brain can cause disabilities related to human movement control, a compensation method was developed for improving the performance of hand movements. The compensation for human hand movements can be carried out by adding an assistant force that is generated from artificial equipment attached to a human arm. From the experiment on visual target tracking, it was found that the tracking trajectory was adequately represented by a dynamic model of the motion of an articulated industrial robot arm, and the different abilities for movement control among healthy people and patients were classified by different model parameters as position loop gain, velocity loop gain, and response delay. Dynamic force compensation was approached by considering the different control features of the patients. The effectiveness of the proposed compensation method was verified in a simulation study on an actual industrial robot arm. A human-machine interface, e.g., a brain-computer interface (BCI), for realizing the control of artificial equipment to compensate for human hand movements is also presented and discussed.

Role of the cerebellar flocculus region in the coordination of eye and head movements during gaze pursuit.

The contribution of the flocculus region of the cerebellum to horizontal gaze pursuit was studied in squirrel monkeys. When the head was free to move, the monkeys pursued targets with a combination of smooth eye and head movements; with the majority of the gaze velocity produced by smooth tracking head movements. In the accompanying study we reported that the flocculus region was necessary for cancellation of the vestibuloocular reflex (VOR) evoked by passive whole body rotation. The question addressed in this study was whether the flocculus region of the cerebellum also plays a role in canceling the VOR produced by active head movements during gaze pursuit. The firing behavior of 121 Purkinje (Pk) cells that were sensitive to horizontal smooth pursuit eye movements was studied. The sample included 66 eye velocity Pk cells and 55 gaze velocity Pk cells. All of the cells remained sensitive to smooth pursuit eye movements during combined eye and head tracking. Eye velocity Pk cells were insensitive to smooth pursuit head movements. Gaze velocity Pk cells were nearly as sensitive to active smooth pursuit head movements as they were passive whole body rotation; but they were less than half as sensitive ( approximately 43%) to smooth pursuit head movements as they were to smooth pursuit eye movements. Considered as a whole, the Pk cells in the flocculus region of the cerebellar cortex were <20% as sensitive to smooth pursuit head movements as they were to smooth pursuit eye movements, which suggests that this region does not produce signals sufficient to cancel the VOR during smooth head tracking. The comparative effect of injections of muscimol into the flocculus region on smooth pursuit eye and head movements was studied in two monkeys. Muscimol inactivation of the flocculus region profoundly affected smooth pursuit eye movements but had little effect on smooth pursuit head movements or on smooth tracking of visual targets when the head was free to move. We conclude that the signals produced by flocculus region Pk cells are neither necessary nor sufficient to cancel the VOR during gaze pursuit.

The role of feedback in manual tracking of visual targets.

The role of visual feedback in manual tracking was investigated in 24 subjects who tracked 5-, 10-, and 40-mm/diameter targets, moving on a screen at 18 to 25 mm/sec., along various paths, by moving an unseen handle over a digitizing tablet. A cursor indicating instantaneous handle position was visible at all times on half the trials and hidden within a circle coaxial with the target but double its diameter in the other half. The handle had to be within the instantaneous target's digitizer-defined boundaries for the latter to keep moving. All tracking movements were segmented into small movement steps. A tendency to outrun the target was seen, indicating predictive control. Absence of visual feedback had negligible effect on movement velocity. Movement direction appeared to involve open-loop programming but improved significantly when subjects could see the cursor. Occasional corrective movements occurred only when visual feedback was given. Otherwise, a large positional error accumulates despite reasonable ability to control tracking direction.

A multiprocessor-oriented visual tracking system

The design and prototypal realization of a visual tracking system is presented. The approach to target identification is nonconventional, in that it relies on an architecture composed of multiple standard neural networks (multilayer perceptrons) and exploits the information contained in simple features extracted from images, performing a small number of operations. Therefore, the tracking functions are learned by examples, rather than implemented directly. The system demonstrates that a quite complex task such as visual target tracking can be easily obtained by a suitable neural architecture. The fast tracking algorithm and the parallel structure allow a true real-time operation. The system exploits a two-level neural-network hierarchy with a number of parallel networks and an "arbiter". The training set consists of various geometrical shapes, preprocessed to yield the data vectors. The experimental hardware implementation is based on multiple processing units, implementing the neural architecture, and serves as a prototype for the analysis of the system in practice. A small-sized realization can also be obtained.

Encoding of Smooth Pursuit Direction and Eye Position by Neurons of Area MSTd of Macaque Monkey

The spike activity of neurons was recorded from the dorsal bank of the superior temporal sulcus (area MSTd) of alert behaving macaque monkeys performing visual fixation or target tracking tasks, with the aim of studying the tuning features of these neurons with both the direction of slow eye movement and the position of gaze. One hundred thirty-two neurons were tested for several fixation points and tracking directions. Many of them (43%) tuned to the direction of pursuit, regardless of the angle of gaze. Some (18%) showed a tonic discharge modulated by the static position of the eyes without pursuit direction specificity. A substantial number of cells (22%) were characterized by a discharge rate tuned to pursuit direction but influenced also by angle of gaze. Tuning curves for eye movement direction presented an average bandwidth of 130 degrees and turned out to be continuously overlapping, suggesting a sort of vector coding of smooth pursuit direction. Gaze fields of eye position (EP) neurons were mostly ramp-like, with center of ramps shifted away from the straight ahead, implying a form of scalar coding of gaze eccentricity. The different categories of cells were intermingled and close to each other, suggesting possible reciprocal interactions within the same cortical area. These results show that EP and pursuit direction are signaled mainly by separate neuronal elements in area MSTd. Moreover, some cells can integrate both signals. Taking into account the visual responses of MSTd neurons to large, textured, moving fields, it is suggested that this area could be the site of interaction between visuo-oculomotor signals related to visual motion detection, slow eye movement direction, and EP. This signal interaction may be important for integrative functions such as analysis of external or self-induced visual motion, cortical control of pursuit eye movements, and eye/head coordination.

Multichannel Regularized Iterative Restoration of Motion Compensated Image Sequences

Restoration of image sequences is an important problem that can be encountered in many image processing applications, such as visual communications, robot guidance, and target tracking. The independent restoration of each frame in an image sequence is a suboptimal approach because the between-frame correlations are not explicitly taken into consideration. In this paper we address this problem by proposing a multichannel restoration approach. The multiple time-frames (channels) of the image sequence are restored simultaneously by using a multichannel regularized least-squares formulation of the problem. The regularization operator captures both within- and between-frame (channel) properties of the image sequence with the explicit use of the displacement vector field. We propose a number of different approaches to obtain the multichannel regularization operator, as well as an algorithm to iteratively compute the restored images. We present experiments that demonstrate the value of the proposed multichannel approach.

Absence of Subcortical Pathway Optokinetic Eye Movements in an Infant with Cortical Blindness

Tracking of large visual targets by normal neonates and by infants with visual cortex damage has led to the suggestion that infants use subcortical vision to guide visuomotor behaviors. To test this assertion, the author recorded eye movements in a five-month-old infant who became cor-tically-blind following an anoxic episode. Use of a full-field, high-contrast visual stimulus failed to elicit any evidence of optokinetic nystagmus. The findings in this infant lead to the conclusion that retinotectal visual inputs are incapable of driving eye movements in the young human in the absence of cortical vision.

Egocentric visual target position and velocity coding: role of ocular muscle proprioception.

Limited knowledge is available regarding the processes by which the brain codes the velocity of visual targets with respect to the observer. Two models have been previously proposed to describe the visual target localization mechanism. Both assume that the necessary information is derived from the coding of the position of the eye in the orbit, either through a copy of the muscular activation (out flow model) or through eye muscle proprioception (in flow model). Eye velocity coding might be derived from velocity sensitive ocular muscle proprioceptors or from position coding signals through differentiation. We used techniques based on manual pointing and manual tracking of visual target, combined with passive deviation of one covered eye, to demonstrate that ocular muscle proprioception is involved in (i) eye-in-head position coding, hence in target localization function; (ii) long-term maintenance of ocular alignment (phoria); and (iii) sensing of visual target velocity with respect to the head. These observations support other data now available, describing the processes by which the brain codes position and velocity of visual targets. Such findings might interest engineers in the field of robotics who are facing the problem of providing robots with the ability to sense object position and velocity in order to create an internal model of their working environment.

Active vision system integrated with angular velocity sensors

This paper describes intelligent sensing and sensor fusion technology, which are studied through the design of a visual target tracking system. At first, a gaze control model integrating visual information is proposed. It is derived from the requirement that the target should be captured at the center of the camera's field of view at a certain time. It is interesting that motions similar to both human saccadic and smooth pursuit eye movement appear in the model. Then a method of integrating the gaze control system and angular velocity sensors is proposed. This method enables unexpectedly disturbed cameras to perform gaze control like the human vestibulo-ocular reflex (VOR). The essence of this method is, in short, the decomposition of a required task into subsystems and the reconstruction of a simple integrating system from them. We made a proto-type system and evaluated the gaze control model and the integration method experimentally.

A Method for Analysing Characteristics of Manual Tracking of Visual Target by Using Artificial Neural Network

A Method for Analysing Characteristics of Manual Tracking of Visual Target by Using Artificial Neural Network