Motion Parallax Improves Object Recognition in the Presence of Clutter in Simulated Prosthetic Vision.

Cheng Qiu,Kassandra R Lee,Eli Peli,Robert Goldstein,Jae-Hyun Jung

doi:10.1167/tvst.7.5.29

Cheng Qiu, Kassandra R Lee + Show 3 more

Open Access

https://doi.org/10.1167/tvst.7.5.29

Copy DOI

Abstract

PurposeEfficacy of current visual prostheses in object recognition is limited. Among various limitations to be addressed, such as low resolution and low dynamic range, here we focus on reducing the impact of background clutter on object recognition. We have proposed the use of motion parallax via head-mounted camera lateral scanning and computationally stabilizing the object of interest (OI) to support neural background decluttering. Simulations in head-mounted displays (HMD), mimicking the proposed effect, were used to test object recognition in normally sighted subjects.MethodsImages (24° field of view) were captured from multiple viewpoints and presented at a low resolution (20 × 20). All viewpoints were centered on the OI. Experimental conditions (2 × 3) included clutter (with or without) × head scanning (single viewpoint, 9 coherent viewpoints corresponding to subjects' head positions, and 9 randomly associated viewpoints). Subjects used lateral head movements to view OIs in the HMD. Each object was displayed only once for each subject.ResultsThe median recognition rate without clutter was 40% for all head scanning conditions. Performance with synthetic background clutter dropped to 10% in the static condition, but it was improved to 20% with the coherent and random head scanning (corrected P = 0.005 and P = 0.049, respectively).ConclusionsBackground decluttering using motion parallax cues but not the coherent multiple views of the OI improved object recognition in low-resolution images. The improvement did not fully eliminate the impact of background.Translational RelevanceMotion parallax is an effective but incomplete decluttering solution for object recognition with visual prostheses.

Highlights

An estimated 260,000 individuals in America are functionally blind.[1]
We demonstrated the usefulness of OI-stabilized motion parallax for improving object recognition, presumably by supporting object and background separation
Object recognition refers to a connection between a previously encountered stimulus and a new encounter with the same/similar stimulus.[31] Prior studies with visual prostheses (e.g., Alpha IMS, Argus II) and substitution devices (SSDs) (BrainPort) reported investigating object recognition, but used a few pretrained objects in their performance evaluation.[3,11,12,32] In the literature, this is referred to as pattern or object discrimination[33] in distinction from object recognition

Summary

Introduction

An estimated 260,000 individuals in America are functionally blind.[1] Blind individuals use mobility aids, such as long canes and guide dogs, and access text through braille and computer programs that convert text to speech. These tools do little in aiding search for and recognition of objects. Most visual prostheses and SSDs use video cameras to capture images and convert them into a format appropriate for the device These final ‘‘images’’ are restricted by the limitations of the devices and the physiological interface.

Objectives

Methods

Conclusion