Auditory Menus Research Articles

Menu Navigation With In-Vehicle Technologies: Auditory Menu Cues Improve Dual Task Performance, Preference, and Workload

Auditory display research for driving has mainly examined a limited range of tasks (e.g., collision warnings, cell phone tasks). In contrast, the goal of this project was to evaluate the effectiveness of enhanced auditory menu cues in a simulated driving context. The advanced auditory cues of “spearcons” (compressed speech cues) and “spindex” (a speech-based index cue) were predicted to improve both menu navigation and driving. Two experiments used a dual task paradigm in which users selected songs on the vehicle’s infotainment system. In Experiment 1, 24 undergraduates played a simple, perceptual-motor ball-catching game (the primary task; a surrogate for driving), and navigated through an alphabetized list of 150 song titles—rendered as an auditory menu—as a secondary task. The menu was presented either in the typical visual-only manner, or enhanced with text-to-speech (TTS), or TTS plus one of three types of additional auditory cues. In Experiment 2, 34 undergraduates conducted the same secondary task while driving in a simulator. In both experiments, performance on both the primary task (success rate of the game or driving performance) and the secondary task (menu search time) was better with the auditory menus than with no sound. Perceived workload scores as well as user preferences favored the enhanced auditory cue types. These results show that adding audio, and enhanced auditory cues in particular, can allow a driver to operate the menus of in-vehicle technologies more efficiently while driving more safely. Results are discussed with multiple resources theory.

Eyes-free interaction with free-hand gestures and auditory menus

Auditory interfaces can overcome visual interfaces when a primary task, such as driving, competes for the attention of a user controlling a device, such as radio. In emerging interfaces enabled by camera tracking, auditory displays may also provide viable alternatives to visual displays. This paper presents a user study of interoperable auditory and visual menus, in which control gestures remain the same in the visual and the auditory domain. Tested control methods included a novel free-hand gesture interaction with camera-based tracking, and touch screen interaction with a tablet. The task of the participants was to select numbers from a visual or an auditory menu including a circular layout and a numeric keypad layout. Results show, that even with participant's full attention to the task, the performance and accuracy of the auditory interface are the same or even slightly better than the visual when controlled with free-hand gestures. The auditory menu was measured to be slower in touch screen interaction, but questionnaire revealed that over half of the participants felt that the circular auditory menu was faster than the visual menu. Furthermore, visual and auditory feedback in touch screen interaction with numeric layout was measured fastest, touch screen with circular menu second fastest, and the free-hand gesture interface was slowest. The results suggest that auditory menus can potentially provide a fast and desirable interface to control devices with free-hand gestures.

Spearcons Improve Navigation Performance and Perceived Speediness in Korean Auditory Menus

For decades, auditory menus using both speech (usually text-to-speech, TTS) and non-speech sounds have been extensively studied. Researchers have developed situation-optimized auditory menus involving such cues as auditory icons, earcons, spearcons, and spindex. Spearcons have generally outperformed other cues in terms of providing both contextual information and item-specific information. However, little research has been devoted to exploration of spearcons in languages other than English, or the use of spearcon-only auditory menus. In this study, we evaluated the use of spearcons in Korean menus, as well as the use of spearcons alone. Twenty-five native Korean speakers navigated through a two-dimensional auditory menu presented via TTS, with or without spearcon enhancements. Korean spearcons were successful. Participants also rated the spearcon-enhanced menu as seeming speedier and more fun than the TTS-only menu. After a short learning period, mean time-to-target in the auditory menu was even faster with spearcons alone, compared to traditional TTS-only menus.

Spearcons (Speech-Based Earcons) Improve Navigation Performance in Advanced Auditory Menus

The goal of this project is to evaluate a new auditory cue, which the authors call spearcons, in comparison to other auditory cues with the aim of improving auditory menu navigation. With the shrinking displays of mobile devices and increasing technology use by visually impaired users, it becomes important to improve usability of non-graphical user interface (GUI) interfaces such as auditory menus. Using nonspeech sounds called auditory icons (i.e., representative real sounds of objects or events) or earcons (i.e., brief musical melody patterns) has been proposed to enhance menu navigation. To compensate for the weaknesses of traditional nonspeech auditory cues, the authors developed spearcons by speeding up a spoken phrase, even to the point where it is no longer recognized as speech. The authors conducted five empirical experiments. In Experiments 1 and 2, they measured menu navigation efficiency and accuracy among cues. In Experiments 3 and 4, they evaluated learning rate of cues and speech itself. In Experiment 5, they assessed spearcon enhancements compared to plain TTS (text to speech: speak out written menu items) in a two-dimensional auditory menu. Spearcons outperformed traditional and newer hybrid auditory cues in navigation efficiency, accuracy, and learning rate. Moreover, spearcons showed comparable learnability as normal speech and led to better performance than speech-only auditory cues in two-dimensional menu navigation. These results show that spearcons can be more effective than previous auditory cues in menu-based interfaces. Spearcons have broadened the taxonomy of nonspeech auditory cues. Users can benefit from the application of spearcons in real devices.

“Spindex” (Speech Index) Enhances Menus on Touch Screen Devices with Tapping, Wheeling, and Flicking

Users interact with many electronic devices via menus such as auditory or visual menus. Auditory menus can either complement or replace visual menus. We investigated how advanced auditory cues enhance auditory menus on a smartphone, with tapping, wheeling, and flicking input gestures. The study evaluated a spindex (speech index), in which audio cues inform users where they are in a menu; 122 undergraduates navigated through a menu of 150 songs. Study variables included auditory cue type (text-to-speech alone or TTS plus spindex), visual display mode (on or off), and input gesture (tapping, wheeling, or flicking). Target search time and subjective workload were lower with spindex than without for all input gestures regardless of visual display mode. The spindex condition was rated subjectively higher than plain speech. The effects of input method and display mode on navigation behaviors were analyzed with the two-stage navigation strategy model. Results are discussed in relation to attention theories and in terms of practical applications.

Spindex (Speech Index) Improves Auditory Menu Acceptance and Navigation Performance

Users interact with mobile devices through menus, which can include many items. Auditory menus have the potential to make those devices more accessible to a wide range of users. However, auditory menus are a relatively new concept, and there are few guidelines that describe how to design them. In this paper, we detail how visual menu concepts may be applied to auditory menus in order to help develop design guidelines. Specifically, we examine how to optimize the designs of a new contextual cue, called “spindex” (i.e., speech index). We developed and evaluated various design alternatives for spindex and iteratively refined the design with sighted users and visually impaired users. As a result, the “attenuated” spindex was the best in terms of preference as well as performance, across user groups. Nevertheless, sighted and visually impaired participants showed slightly different responses and feedback. Results are discussed in terms of acoustical theory, practical display design, and assistive technology design.

Enhanced Auditory Menu Cues on a Mobile Phone Improve Time-Shared Performance of a Driving-Like Dual Task

The growing trend of using mobile phones and other in-vehicle technologies (IVT) while driving has spurred research on driver distraction, its effects and alleviation (Ashley, 2001; Young & Regan, 2007). The present study used a dual task in which 21 undergraduates navigated a mobile phone contact list for a target name (secondary task) while playing a computer game representative of driving (primary task). The phone menu was enhanced with two audio navigation cues: traditional text-to-speech (TTS) and spearcons (i.e., compressed speech). These cues were tested with and without visual display of the contact list. Spearcons in conjunction with TTS enhanced performance on the primary task while having no negative effect on the secondary task. Auditory menus reduced perceived workload and increased subjective ratings. Results are discussed in terms of multiple resources theory and practical mobile phone menu design.

"Spindex": Accelerated Initial Speech Sounds Improve Navigation Performance in Auditory Menus

"Spindex": Accelerated Initial Speech Sounds Improve Navigation Performance in Auditory Menus

“Spindex”: Accelerated Initial Speech Sounds Improve Navigation Performance in Auditory Menus

Users interact with mobile devices through menus, which can include many items. Auditory menus can supplement or even replace visual menus. Unfortunately, little research has been devoted to enhancing the usability of large auditory menus. We evaluated a novel auditory menu enhancement called a “spindex” (i.e., speech index), in which brief audio cues inform the user where she is in a long menu. In the current implementation, each item in a menu is preceded by a sound based on the item's initial letter. 25 undergraduates navigated through an alphabetized contact list of 50 or 150 names. The menu was presented with text-to-speech (TTS) alone, or TTS plus spindex, and with the visual menu displayed or not. Search time was faster with the spindex-enhanced menu, especially for long lists. Subjective ratings also favored the spindex. Results are discussed in terms of theory and practical applications.

Navigation Efficiency of Two Dimensional Auditory Menus Using Spearcon Enhancements

A total of 28 undergraduates navigated to specific items in a two dimensional menu that was displayed using only sound. The auditory menu consisted of either text-to-speech (TTS) only, or TTS enhanced with spearcons. Spearcons are brief sound cues created by compressing the original TTS sound file. Speed of navigation to target items in the auditory menu was found to be significantly faster in the spearcon condition than in the condition using only TTS. There was also a smaller per-item cost in terms of speed for the spearcon-enhanced menu, leading to increasingly better performance as menu length increased. These results provide further evidence that spearcon enhancements can lead to faster navigational performance in auditory menus, when compared to text-to-speech alone.

A Comparison of Broad Versus Deep Auditory Menu Structures

The primary purpose of this experiment was to gain a greater understanding of the utilization of working memory when interacting with a speech-enabled interactive voice response (IVR) system. A widely promoted guideline advises limiting IVR menus to five or fewer items because of constraints of the human memory system, commonly citing Miller's (1956) paper. The authors argue that Miller's paper does not, in fact, support this guideline. Furthermore, applying modern theories of working memory leads to the opposite conclusion--that reducing menu length by creating a deeper structure is actually more demanding of users' working memories and leads to poorer performance and satisfaction. Participants took a working memory capacity test and then attempted to complete a series of e-mail tasks using one of two IVR designs (functionally equivalent, but one with a broad menu structure and the other with a deep structure). Users of the broad-structure IVR performed better and were more satisfied than users of the deep-structure IVR. Furthermore, this effect was more pronounced for those with low working memory capacity. Results indicate that creating a deeper structure is more demanding of working memory resource than the alternative of longer, shallower menus. This experiment has important practical implications for all systems with auditory menus (particularly IVRs) because it provides empirical evidence refuting a widely promoted design practice.

User Preferences for Auditory Device-driven Menu Navigation

A device-driven auditory menu interface that “pushes” the menu options to the user and requires minimal user input can be beneficial to users with both limited vision and input range. Presentation rate is important for auditory menus; they need to be slow enough for comprehension and responding, but fast enough to prevent frustration. User preferences for inter-item gap on an auditory device-driven cell phone menu were investigated. Participants ( n=11) navigated a two-level auditory menu to target a randomly chosen item. Before each trial participants set the inter-item gap by manipulating a slider. Results showed a mean preference for .853 second inter-item gap in experimental trials, and a significant decrease in preferences between practice (15) and experimental (30) trials [t(10) = 4.28, p = .002]. Accuracy was 91.5% over all the trials. The findings indicate that it is feasible to design a limited-input cell phone with an auditory device-driven menu.

Optimal number of choices per menu for naive users of a computer-answered telephone system

With the high volume of telephone calls being placed to businesses and community organizations, computer technology is being used to answer and route calls. In order to access information, callers go through a series of auditory menus from which they must select the appropriate option from each menu. Callers must rely on short-term memory since information is rarely preserved for review visually. This paper investigates the structure of the menus as they are arranged in a hierarchical tree. Three configurations were chosen for study: menus with 2, 4, 0r 8 choices per menu. Eighteen subjects were tested on a simulated system, placing calls as they might at home or the office. With 64 items in the database, 2-choice menus require the subject to listen to and choose from 6 different menus. The 4- and 8-choice menus require 3 and 2 menus, respectively. Subjects navigated through the system of menus and reached the correct information in less time and with fewer errors with the 4-choice and 8-choice menus than they did with the 2-choice menus. In a questionnaire, two-thirds of the subjects preferred 4-choice menus. Experience with other computer-answered telephone systems was not a significant factor in the subjects' performance.