ShopTalk: Independent Blind Shopping Through Verbal Route Directions and Barcode Scans

Abstract

Independent shopping in modern grocery stores that carry thousands of products is a great challenge for people with visual impairments. ShopTalk is a proof-of-concept wearable system designed to assist visually impaired shoppers with finding shelved products in grocery stores. Using synthetic verbal route directions and descriptions of the store lay- out, ShopTalk leverages the everyday orientation and mobility skills of independent visually impaired travelers to direct them to aisles with target products. Inside aisles, an off-the-shelf barcode scanner is used in conjunction with a software data structure, called a barcode connectivity matrix, to locate target product on shelves. Two experiments were performed at a real world supermarket. A successful earlier single-subject experiment is summarized and a new experiment involving ten visually impaired participants is presented. In both experiments, ShopTalk was successfully used to guide visually im- paired shoppers to multiple products located in aisles on shelves. ShopTalk is a feasible system for guiding visually im- paired shoppers who are skilled, independent travelers. Its design does not require any hardware instrumentation of the store and leads to low installation and maintenance costs.