Head-mounted Camera Research Articles

How infants view natural scenes gathered from a head-mounted camera.

The role of early visual experience in human infant development has been inferred primarily by studies of visual deprivation (e.g., cataracts). Another approach, described here, is to provide a detailed description of the visual input gathered by normal infants in their natural environment. Recently, several labs have begun the laborious process of obtaining video images from a head-mounted camera to provide an infant's eye-view of their visual world. Preliminary findings from one such study are reviewed and discussed in the context of the power and limitations of this approach for revealing important insights about the role of early visual experience, as well as the broader implications for studies of cognitive, language, and social development.

EyeSeeCam: An Eye Movement–Driven Head Camera for the Examination of Natural Visual Exploration

The prototype of a gaze-controlled, head-mounted camera (EyeSeeCam) was developed that provides the functionality for fundamental studies on human gaze behavior even under dynamic conditions like locomotion. EyeSeeCam incorporates active visual exploration by saccades with image stabilization during head, object, and surround motion just as occurs in human ocular motor control. This prototype is a first attempt to combine free user mobility with image stabilization and unrestricted exploration of the visual surround in a man-made technical vision system. The gaze-driven camera is supplemented by an additional wide-angle, head-fixed scene camera. In this scene view, the focused gaze view is embedded with picture-in-picture functionality, which provides an approximation of the foveated retinal content. Such a combined video clip can be viewed more comfortably than the saccade-pervaded image of the gaze camera alone. EyeSeeCam consists of a video-oculography (VOG) device and a camera motion device. The benchmark for the evaluation of such a device is the vestibulo-ocular reflex (VOR), which requires a latency on the order of 10 msec between head and eye (camera) movements for proper image stabilization. A new lightweight VOG was developed that is able to synchronously measure binocular eye positions at up to 600 Hz. The camera motion device consists of a parallel kinematics setup with a backlash-free gimbal joint that is driven by piezo actuators with no reduction gears. As a result, the latency between the rotations of an artificial eye and the camera was 10 msec, which is VOR-like.

On‐Line Classification and Prediction of Eye Movements by Multiple‐Model Kalman Filtering

An extensible multiple-model Kalman filter framework for eye tracking and video-oculography (VOG) applications is proposed. The Kalman filter predicts future states of a system on the basis of a mathematical model and previous measurements. The predicted values are then compared against the current measurements. In a correcting step, the predicted state is enhanced by the measurements. In this work, the Kalman filter is used for smoothing the VOG data, for on-line classification of eye movements, as well as for predictive real-time control of a gaze-driven head-mounted camera (EyeSeeCam). With multiple models running in parallel, it was possible to distinguish between fixations, slow-phase eye movements, and saccades. Under the assumption that each class of eye movement follows a distinct model, one can decide which types of eye movement occurred by evaluating the probability for each model.

Feasibility Study of a Retinal Prosthesis

To demonstrate that an epiretinal prosthesis can produce patterned visual perception in patients blinded by photoreceptor degeneration who have no other treatment options. A totally blind subject with retinitis pigmentosa had a 16-electrode epiretinal prosthesis implanted. The implant is controlled wirelessly by an external computer or a head-mounted video camera. Spatial vision was assessed by measuring the subject's response to direct stimulation of patterns and by comparing the ability of the subject to identify the orientation of gratings with the system on and off. In response to stimulation of 2 orthogonal rows of electrodes, the subject drew 2 lines with a mean (SEM) angle of 87.4 degrees (1.8 degrees) between them. With the system on, the subject identified the orientation of the grating target up to a spatial resolution that matches the spacing between the adjacent electrodes. In contrast, with the system off, the subject could not detect or identify the target's orientation. Synchronized stimulation of different retinal locations with an epiretinal prosthesis implanted long-term can produce spatial vision with an acuity level determined by the distance between the electrodes. clinicaltrials.gov Identifier: NCT00279500.

Using a Head-Mounted Video Camera to Understand Social Worlds and Experiences

1.1 This paper considers the use of digital video as a research method for qualitative researchers; specifically the head-mounted video (headcam) and the opportunities it offers for deepening our understanding of social life. Headcam is one of a range of recent technological innovations relating to video that provide researchers with new means of addressing methodological challenges. To date, little has been written about the use of digital video in relation to social research (Henley 1998; Pink 2007a). Even less has been written on the specifics of headcam video techniques and less still on its potential outwith a positivist framework. Our exploratory research on mountain bikers' and walkers' embodied, multi-sensory ways of knowing and experiencing landscapes demonstrates some of the possibilities that advancements in headcam technology can offer interpretative and reflexive approaches to social research.

Real and virtual explorations of the environment and interactive tracking of movable objects for the blind on the basis of tactile-acoustical maps and 3D environment models

Purpose. This study describes the development of a multi-functional assistant system for the blind which combines localisation, real and virtual navigation within modelled environments and the identification and tracking of fixed and movable objects.Method. The approximate position of buildings is determined with a global positioning sensor (GPS), then the user establishes exact position at a specific landmark, like a door. This location initialises indoor navigation, based on an inertial sensor, a step recognition algorithm and map. Tracking of movable objects is provided by another inertial sensor and a head-mounted stereo camera, combined with 3D environmental models. This study developed an algorithm based on shape and colour to identify objects and used a common face detection algorithm to inform the user of the presence and position of others.Results. The system allows blind people to determine their position with ∼1 metre accuracy. Virtual exploration of the environment can be accomplished by moving one's finger on a touch screen of a small portable tablet PC. The name of rooms, building features and hazards, modelled objects and their positions are presented acoustically or in Braille.Conclusions. Given adequate environmental models, this system offers blind people the opportunity to navigate independently and safely, even within unknown environments. Additionally, the system facilitates education and rehabilitation by providing, in several languages, object names, features and relative positions.

Applications of augmented‐vision head‐mounted systems in vision rehabilitation

Vision loss typically affects either the wide peripheral vision (important for mobility), or central vision (important for seeing details). Traditional optical visual aids usually recover the lost visual function, but at a high cost for the remaining visual function. We have developed a novel concept of vision-multiplexing using augmented vision head-mounted display systems to address vision loss. Two applications are discussed in this paper. In the first, minified edge images from a head-mounted video camera are presented on a see-through display providing visual field expansion for people with peripheral vision loss, while still enabling the full resolution of the residual central vision to be maintained. The concept has been applied in daytime and nighttime devices. A series of studies suggested that the system could help with visual search, obstacle avoidance, and nighttime mobility. Subjects were positive in their ratings of device cosmetics and ergonomics. The second application is for people with central vision loss. Using an on-axis aligned camera and display system, central visibility is enhanced with 1:1 scale edge images, while still enabling the wide field of the unimpaired peripheral vision to be maintained. The registration error of the system was found to be low in laboratory testing.

New Products

In this issue, editors Eyal de Lara and Maria Ebling cover a mobile firewall with a USB form factor, a smart parking meter that can detect when a vehicle pulls in and out of a parking stall, and an initiative by the British government to outfit police officers with head-mounted cameras. They also present a Web-based voice communications service that lets users manage all their phones with a single virtual phone number and a service that turns the isolation of surfing the Web into a social networking opportunity.

Pragmatics of conversation and communication in noisy settings

Most analyses of discourse pragmatics assume a quiet setting that does not affect the interaction. This study examines two common, communicationally hostile environmental contexts that make demands on the perceptual, cognitive, and pragmatic dimensions of language and multimodal communication. It identifies strategies which discourse participants use to recover the information lost or degraded in noisy conversational interaction, and the repairs and conversational strategies they use if they recognize that communication has failed. We recorded the conversational discourse interaction of 6 normally-hearing adults in a restaurant setting and 24 normally-hearing children in elementary-school classrooms, using ear-level binaural microphones, head-mounted bullet cameras, and tripod-mounted video cameras. This yielded extensive audiotape and videotape data from the perspectives of individual listeners and speakers, and information about the interaction among participants. Our data indicate that the strategies employed in these settings are similar to those employed by people who are hard-of-hearing, and that usage-based linguistic theories and cognitivist theories of language processing, interaction, and pragmatics, that ignore language perception, are inadequate.

201 Imaging the world from a surgeon’s perspective with a gaze aligned head-mounted camera

201 Imaging the world from a surgeon’s perspective with a gaze aligned head-mounted camera

The state of visual prosthetics--Hype or promise?

The miracle of restoring sight to the blind, told in legends and myths throughout human cultures and history, is moving from dream to possibility in our times. Several dozen groups around the world now claim that electronic or neurochemical prostheses may soon restore some measure of functional vision. This editorial examines such claims by reviewing the proposed approaches and assessing their promise. We base our opinions on our involvement with some of these efforts and on presentations and discussions during the most recent biennial Eye and the Chip: World Congress on Artificial Vision organized by the Detroit Institute of Ophthalmology in June 2006; for a recent overview of many current activities in this area, see Dagnelie [1]. The retina extensively preprocesses and reduces image information, and the cortical map and magnification add further complexity; therefore, functional replacement of neural elements should be done at the earliest available stage of visual processing to limit surrogate processing that provides but a poor substitute for the native system. Thus, for example, a retinal prosthesis should be used if the nerve fibers and optic nerve are intact, such that the prosthesis stimulates the most distal remaining cell population available. Despite such considerations, the first attempts at vision restoration almost 40 years ago were directed at the visual cortex [2]. Early cortical implants used electrodes placed between the meninges, avoiding the complications of glial cell encapsulation plaguing most penetrating electrodes. A mapping procedure is used to determine the location of each electrode's phosphene in the visual field, and real-time image processing and stimulation software then extracts and projects contours onto the best-matching electrodes. These phosphenes do not convey real-form vision, yet they do allow the wearer crude localization of outlines in the scene. Newer penetrating probes such as the 100-electrode array developed at the University of Utah--currently being considered for use by the European CortiVis consortium--and the hatpin electrodes--developed at the National Institute of Neurological Disorders and Stroke and since adopted by the group at the Illinois Institute of Technology --may provide smaller phosphenes, and thus better resolution, at much lower and safer charge injection levels. In the early 1990s, intraoperative stimulation in patients with late-stage retinitis pigmentosa (RP) showed that phosphenes could be elicited from blind retinal areas long after photoreceptor degeneration [3]. Researchers are working on various implants in which a matrix of electrodes stimulates the surviving retinal cells with imagery collected by an external camera and reduced to match the resolution of the electrode array. Two research groups have progressed to the stage of prototype implants in patients with late-stage RP. A consortium formed by Second Sight Medical Products LLC and by the Doheny Retina Institute has performed six implants of 4 x 4 electrode arrays based on cochlear implant electronics; these patients can detect phosphenes at individual electrodes, discriminate crude shapes upon multiple electrode stimulation, and recognize simple stimuli presented via a head-mounted camera. A consortium formed by IIP Technologies GmbH and German universities has performed several 50-electrode implants and is reporting that patients have similar discrimination abilities with computer-generated stimuli. Other retinal prosthesis groups are exploring integrated concepts in which the eye's optics become part of a prosthetic imaging device, which in turn generates the signals to drive the secondary neurons. The simplest implementation of such a device, an array of small photodiodes under the retina, appears to have a neurotrophic beneficial effect in RP patients with some remaining vision; 30 patients have received this Optobionics ASR implant in Food and Drug Administration trials [4]. …

A body-mounted camera system for head-pose estimation and user-view image synthesis

In this paper, we propose a body-mounted system to capture user experience as audio/visual information. The proposed system consists of two cameras (head-detection and wide angle) and a microphone. The head-detection camera captures user head motions, while the wide angle color camera captures user frontal view images. An image region approximately corresponding to user view is then synthesized from the wide angle image based on estimated human head motions. The synthesized image and head-motion data are stored in a storage device with audio data. This system overcomes the disadvantages of head-mounted cameras in terms of ease of putting on/taking off the device. It also has less obtrusive visual impact on third persons. Using the proposed system, we can simultaneously record audio data, images in the user field of view, and head gestures (nodding, shaking, etc.) simultaneously. These data contain significant information for recording/analyzing human activities and can be used in wider application domains such as a digital diary or interaction analysis. Experimental results demonstrate the effectiveness of the proposed system.

A Robust Finger Tracking Method for Multimodal Wearable Computer Interfacing

Mobile wearable computers are intended to provide users with real-time access to information in a natural and unobtrusive manner. Computing and sensing in these devices must be reliable, easy to interact with, transparent, and configured to support different needs and complexities. This paper presents a vision-based robust finger tracking algorithm combined with audio-based control commands that is integrated into a multimodal unobtrusive user interface, wherein the interface may be used to segment out objects of interest in the environment by encircling them with the user's pointing fingertip. In order to quickly extract the objects encircled by the user from a complex scene, this unobtrusive interface uses a single head-mounted camera to capture color images, which are then processed using algorithms to perform: color segmentation, fingertip shape analysis, perturbation model learning, and robust fingertip tracking. This interface is designed to be robust to changes in the environment and user's movements by incorporating a state-space estimation with uncertain models algorithm, which attempts to control the influence of uncertain environment conditions on the system's fingertip tracking performance by adapting the tracking model to compensate for the uncertainties inherent in the data collected with a wearable computer

Playing checkers: detection and eye–hand coordination in simulated prosthetic vision

In order to assess the potential for visual inspection and eye–hand coordination without tactile feedback under conditions that may be available to future retinal prosthesis wearers, we studied the ability of sighted individuals to act upon pixelized visual information at very low resolution, equivalent to 20/2400 visual acuity. Live images from a head-mounted camera were low-pass filtered and presented in a raster of 6 × 10 circular Gaussian dots. Subjects could either freely move their gaze across the raster (free-viewing condition) or the raster position was locked to the subject's gaze by means of video-based pupil tracking (gaze-locked condition). Four normally sighted and one severely visually impaired subject with moderate nystagmus participated in a series of four experiments. Subjects' task was to count 1 to 16 white fields randomly distributed across an otherwise black checkerboard (counting task) or to place a black checker on each of the white fields (placing task). We found that all subjects were capable of learning both tasks after varying amounts of practice, both in the free-viewing and in the gaze-locked conditions. Normally sighted subjects all reached very similar performance levels independent of the condition. The practiced performance level of the visually impaired subject in the free-viewing condition was indistinguishable from that of the normally sighted subjects, but required approximately twice the amount of time to place checkers in the gaze-locked condition; this difference is most likely attributable to this subject's nystagmus. Thus, if early retinal prosthesis wearers can achieve crude form vision, then on the basis of these results they too should be able to perform simple eye–hand coordination tasks without tactile feedback.

Imaging the world from a surgeon's perspective with a gaze aligned head-mounted camera

Imaging the world from a surgeon's perspective with a gaze aligned head-mounted camera

Visual attention in orienteers at different levels of experience

This study explored how differences in orienteering experience are related to differences in how visual attention is allocated to the map, the environment and to travel. Twenty more experienced and 20 less experienced individuals orienteered while wearing a head-mounted video camera with microphone. The participants verbalized what they were attending to (map, environment or travel) at any given time. Each recorded film was coded at each point in time in terms of what the participant was attending to and whether the participant was moving or stationary. More experienced orienteers attended to the map markedly more while moving and spent less time stationary than less experienced orienteers. The participants' performance was significantly related to the ability to attend to the map while moving. The strategic control of attention is proposed to explain this ability. It is proposed that attentional training might enhance performance in sports characterized by multiple and dynamically varying elements.

Optical eye models for gaze tracking

The traditional bottom-up approach to video gaze tracking consists of measuring image features, such as the position of the pupil, corneal reflex, limbus, etc. These measurements are mapped to gaze angles using coefficients obtained from calibration data, collected as a cooperative subject voluntarily fixates a series of known targets. This may be contrasted with a top-down approach in which the pose parameters of a model of the eye are adjusted in conjunction with a camera model to obtain a match to image data. One advantage of the model-based approach is provided by robustness to changes in geometry, in particular the disambiguation of translation and rotation. A second advantage is that the pose estimates obtained are in absolute angular units (e.g., degrees); traditional calibration serves only to determine the relation between the visual and optical axes, and provide a check for the model. While traditional grid calibration methods may not need to be applied, a set of views of the eye in a variety of poses is needed to determine the model parameters for an individual. When relative motion between the head and the camera is eliminated (as with a head-mounted camera), the model parameters can be determined from as few as two images. A single point calibration is required to determine the angular offset between the line-of-sight and the observed optical axis.

THERE IS SOMETHING OUT THERE: DISTAL ATTRIBUTION IN SENSORY SUBSTITUTION, TWENTY YEARS LATER

Sensory substitution constitutes an interesting domain of study to consider the philosopher's classical question of distal attribution: how we can distinguish between a sensation and the perception of an object that causes this sensation. We tested the hypothesis that distal attribution consists of three distinct components: an object, a perceptual space, and a coupling between subjects' movements and stimulation. We equipped sixty participants with a visual-to-auditory substitution device, without any information about it. The device converts the video stream produced by a head-mounted camera into a sound stream. We investigated several experimental conditions: the existence or not of a correlation between movements and resulting stimulation, the direct or indirect manipulation of an object, and the presence of a background environment. Participants were asked to describe their impressions by rating their experiences in terms of seven possible "scenarios". These scenarios were carefully chosen to distinguish the degree to which the participants attributed their sensations to a distal cause. Participants rated the scenarios both before and after they were given the possibility to interrupt the stimulation with an obstacle. We were interested in several questions. Did participants extract laws of co-variation between their movements and resulting stimulation? Did they deduce the existence of a perceptual space originating from this coupling? Did they individuate objects that caused the sensations? Whatever the experimental conditions, participants were able to establish that there was a link between their movements and the resulting auditory stimulation. Detection of the existence of a coupling was more frequent than the inferences of distal space and object.

HAND–EYE COORDINATION THROUGH ENDPOINT CLOSED-LOOP AND LEARNED ENDPOINT OPEN-LOOP VISUAL SERVO CONTROL

We propose a hand-eye coordination system for a humanoid robot that supports bimanual reaching. The system combines endpoint closed-loop and open-loop visual servo control. The closed-loop component moves the eyes, head, arms, and torso, based on the position of the target and the robot's hands, as seen by the robot's head-mounted cameras. The open-loop component uses a motor-motor mapping that is learnt online to support movement when visual cues are not available.

Clinical Reasoning during Community Health Home Visits: Expert and Novice Differences

Very little is known about the differences between novice and expert clinical reasoning in community health practice. This article presents the findings of a study of the clinical reasoning of five expert and five novice community health occupational therapists (CHOTs) during the conducting of home visits. A head-mounted video camera was used to record the visits, followed by the participants reporting their clinical reasoning verbally using a video-assisted debriefing method. The transcripts from these verbal reports were analysed quantitatively and qualitatively. The quantitative results demonstrated many differences between experts and novices in terms of the amounts and types of clinical reasoning used. For example, the novices used more procedural reasoning whereas the experts used more conditional reasoning and mixes of different reasoning types. The qualitative results demonstrated that the experts used a free-flowing conversational approach when reasoning during home visits whereas the novices depended on external structures such as assessment forms to guide the process. Given their experience and familiarity with the process, the experts were confident and clear in their reasoning whereas the novices were more awkward and self-conscious. The experts handled sensitive issues whereas the novices seemed to avoid them. The study findings may provide insights for student and novice therapists concerning expert CHOTs' practice and promote reflection in general on the attainment of expertise in clinical practice.