Visual Scene Displays Research Articles

A comparison of differing organizational formats for teaching requesting skills to children with autism

The selection of high-tech AAC for children diagnosed with autism spectrum disorder can be a challenging process due to the vast array of options available. One of the decisions that clinicians need to make involves how vocabulary will be organized on the display. This study aimed to compare a visual scene display (VSD) with a grid display using a multiple-probe design across participants with an embedded adapted alternating treatment design. Four young children with autism spectrum disorder who were beginning communicators were recruited and taught to request preferred items using two display formats: VSD and grid layout on a mainstream tablet with an AAC app. Two of the participants achieved criterion with both displays, the other two participants failed to achieve criterion in either display. For all participants, progress was similar in both displays. The results are discussed through the lens of each participant’s characteristics with suggestions for clinical decision-making.

Visual fixation patterns to AAC displays are significantly correlated with motor selection for individuals with Down syndrome or individuals on the autism spectrum

Eye tracking research technologies are often used to study how individuals attend visually to different types of AAC displays (e.g. visual scene displays, grid displays). The assumption is that efficiency of visual search may relate to efficiency of motor selection necessary for communication via aided AAC; however, this assumption has not received direct empirical study. We examined the relation between speed of visual search and speed of motor selection of symbols. Ten individuals on the autism spectrum (AS; Study 1) and nine with Down syndrome (DS; Study 2) participated in a search task using simulated AAC displays with a main visual scene display (VSD) and a navigation bar of thumbnail VSDs. Participants were given an auditory prompt to find one of four thumbnail VSDs in the navigation bar. Eye tracking technologies measured how long it took participants to fixate visually on the thumbnail VSD, and recorded how long it took participants to select the thumbnail VSD with a finger. A statistically significant relationship emerged between visual fixation and selection latencies, confirming the positive relationship between visual processing and motor selection for both groups of participants. Eye tracking data may serve as a useful proxy measure for evaluating how display design influences selection of AAC symbols, especially when individuals are unwilling or unable to comply with traditional behaviorally-based assessment tasks.

Visual Scene Displays for Children and Adults: Using Case Studies to Bridge Research and Clinical Practice

Purpose: The purpose of this clinical focus article is to share research and clinical perspectives related to visual scene display (VSD) technology for people who use augmentative and alternative communication (AAC). Furthermore, the authors will help bridge the gap between research and clinical practice as speech-language pathologists (SLPs) consider supplementing AAC systems with VSD technology. The authors will review research and clinical practice about the development and use of VSD technology for both children and adults as a tool to support language, literacy, and social interactions. SLPs will be informed about key principles from VSD research to assist them to use this technology to supplement multimodal communication methods. Case studies will provide examples for SLPs to consider how they can create and use VSD technology in their clinical practice. Conclusions: Previous AAC research supports the development and use of VSDs for emerging communicators and people with aphasia. Based on the authors' clinical observations and review of the relevant research, challenges remain with VSD implementation as SLPs determine how to develop and effectively integrate this newer technology with current AAC interface design. Recommendations and ideas will be provided to SLPs who are considering VSDs as a visual support to promote language development and increase social interactions.

Examining the Visual Attention Patterns and Identification Accuracy of Adults With Aphasia for Grids and Visual Scene Displays.

We compared the degree of cognitive processing needed by people with aphasia to identify themes depicted in grids and visual scene displays (VSDs). We also compared the accuracy of theme identification for both display types. Eye-tracking technology was employed to measure the visual processing patterns of 21 adults with aphasia when interpreting themes presented through grids and VSDs. Additionally, we assessed theme identification accuracy by having participants select themes from four choices after viewing each display. Participants more rapidly identified VSDs than grid displays, and VSDs required fewer visual fixations to process than grids. No significant differences were noted between grids and VSDs for theme identification accuracy; however, results indicate a ceiling effect for the variable, as participant accuracy levels were nearly 100% for both display conditions. Results from this study add to a growing body of evidence supporting the use of VSDs for adults with aphasia. Both display types were accurately identified; however, VSDs were processed more efficiently than grids indicating that both display types may prove effective for people with aphasia; however, VSDs may require less cognitive effort to effectively use than grid displays.

Exploring high-technology augmentative and alternative communication interfaces: the effect of age and technology experience

Purpose Many people with aphasia have inadequate language skills for functional communication thereby necessitating well-designed augmentative and alternative communication (AAC) interfaces. Using both a grid and a VSD interface with single word hotspots by neurologically healthy adults, across ages and with different technology experiences, this study examines the utility of these interfaces prior to investigating their use with people with aphasia in future studies. Method Participants include 18 young adults, 24 older adults with technology experience, and 20 older adults with limited technology experience. The older adult groups were matched for mean age and for sex. Participants described pictures with each interface, and performance was measured based on four dependent variables: (a) the total number of correct information units (CIUs), (b) the percentage of CIUs, (c) CIUs per minute, and (d) preferred interface. Results There was a significant difference between older adults and the young adult group for the total number of CIUs (p < 0.001) and CIUs per minute (p < 0.001). Despite the changes in technology experience between the two older groups, there was no significant difference in performance. Additionally, there was no significant difference in performance across the two interfaces in any of the groups. Conclusion Findings suggest age does impact performance on AAC. However, the difference in technology experience in older adult groups did not affect performance. Furthermore, both interfaces used in the current study were equally beneficial when describing pictures by healthy adults. Implications for Rehabilitation Augmentative and alternative communication (AAC) systems involve various interface designs. Consequently, the comparative utility of these interfaces must be examined across populations and communicative functions. Advancement in technology has given rise to numerous high-tech AAC interface designs. However, older adults with acquired language difficulties whose technology experience is limited may become reluctant to use high-tech AAC. Hence, the influence of technology experience on high-tech AAC performance should be investigated. Prior to examining interfaces with individuals with acquired language difficulties, the current study examined the use of a) grid display and b) visual scene display (VSD) with single word hotspots, by healthy adults of different ages and technology experiences. The study revealed that older adults with limited technology experience were equally competent in using high-tech AAC interfaces, as those with technology experience. Hence, an individual’s technology experience should not be considered a deterrent to using high-tech AAC interfaces. Additionally, the study found no differences in performance between the two interfaces, making it essential to consider user's personal preference when adopting AAC interface designs.

The Effect of AAC Display Types on Message Production and Its Relationship with Reading Ability in Patients with Cognitive-Communication Disorders

Objectives: To provide empirical evidence that can be applied in medical settings, this study aimed to examine the effects of AAC display types on message production ability in patients with cognitive-communication disorders who are hospitalized and the correlation with the patients’ reading abilities.Methods: Twenty patients with cerebral vascular accidents who were diagnosed with cognitive-communication disorders participated in this study. Message production abilities were examined by measuring accuracy scores and response times under three AAC display conditions: (1) text-based visual scene display (VSD), (2) mixed type (text and graphic symbol-based) of VSD, and (3) graphic symbol-based GRID.Results: The graphic symbol-based GRID showed significantly high accuracy scores and short response time compared to the text-based VSD. There was no significant difference between the graphic symbol-based GRID and the mixed type of VSD in terms of accuracy scores and response time. In addition, significant correlations between the VSD type of display and the reading task scores were found.Conclusion: This study showed that the VSD types of AAC display do not enhance the communication efficacy more than the graphic symbol-based GRID in medical settings. It is at odds with a previous study that claims the effectiveness of text-based VSD without considering patients’ reading ability. These results suggest that rather than perceiving VSD itself to be more effective than other display types, it is important to provide individual graphic symbols so that reading impaired patients can understand the meaning of each message.

초기 의사소통자 가상 사례를 통한 보완대체의사소통(AAC) 화면 디자인에 대한 설문 조사

Background and Objective: The purpose of this study was to identify Korean speech-language pathologists‘ (SLPs) decisions and perspectives on AAC display design for a hypothetical beginner communicator. Method: Forty six Korean SLPs with experience in AAC responded to questions regarding AAC display design when provided a compilation case study of a beginner communicator. They were asked what type of AAC display they would design for the child communicating during a music class. They selected one among 1) grid display, 2) Visual Scene Displays(VSDs), and 3) hybrid of grid display and VSDs and provided elaborated information for each. Participants’ responses were analyzed by calculating the frequency and the percentage of each. Results: A majority of Korean SLPs chose VSDs/hybrid for the specific case and communication setting (i.e., music class). Participants provided opinions on how they would design VSDs/hybrid or grid displays with specific suggestions. In particular, they suggested which components should be included in the VSD for the case. They were also asked to respond to the follow-up questions about which components should be considered important when they design a grid display for the case. The results were also compared with those from American SLPs who participated in Thistle &amp; Wilkinson’s (2020) survey with a similar compilation case study. Conclusion: The results revealed that SLPs select different types of displays to best support different AAC users and using VSDs/hybrid may be suggested for interactive communication, in particular, for the activity-based communication settings.

Strategies for highlighting items within visual scene displays to support augmentative and alternative communication access for those with physical impairments

Purpose In contrast to the traditional grid-based display, visual scene displays (VSDs) offer a new paradigm for aided communication. For individuals who cannot select items from an AAC display by direct selection due to physical impairments, AAC access can be supported via methods such as item scanning. Item scanning sequentially highlights items on a display until the individual signals for selection. How items are highlighted or scanned for AAC access can impact performance outcomes. Further, the effectiveness of a VSD interface may be enhanced through consultation with experts in visual communication. Therefore, to support AAC access for those with physical impairments, the aim of this study was to evaluate the perspectives of experts in visual communication regarding effective methods for highlighting VSD elements. Methods Thirteen participants with expertise related to visual communication (e.g., photographers, artists) completed semi-structured interviews regarding techniques for item highlighting. Results Study findings identified four main themes to inform how AAC items may be highlighted or scanned, including (1) use of contrast related to light and dark, (2) use of contrast as it relates to colour, (3) outline highlighting, and (4) use of scale and motion. Conclusion By identifying how compositional techniques can be utilized to highlight VSD elements, study findings may inform current practice for scanning-based AAC access, along with other selection techniques where feedback or highlighting is used (e.g., eye-gaze, brain-computer interface). Further, avenues for just-in-time programming are discussed to support effective implementation for those with physical impairments. IMPLICATIONS FOR REHABILITATION Findings identify multiple potential techniques to improve scanning through items in a photograph for individuals with severe motor impairments using alternative access strategies. Study findings inform current practice for scanning-based AAC access, along with other selection techniques where feedback or highlighting is used (e.g., eye-gaze, brain-computer interface). Avenues for just in time programming of AAC displays are discussed to decrease programming demands and support effective implementation of study findings.

Visual Attention Patterns in Adults According to the Use of Things in Aided AAC VSDs: An Eye-tracking Study

Objectives: This study aims to investigate patterns of visual attention on a target object in VSDs (Visual Scene Displays) when they are designed with/without an action of usage of the object. We used eye-tracking technology to evaluate how the action of usage of an object in still photographs influenced the visual attention of adults without disabilities. We tried to examine visual attention on the contents of visual scene displays (VSDs).Methods: 25 college students participated in the study. Eye-tracking technology recorded point-of-gaze while participants viewed 20 photographs. Data from eye-tracking provided information on where participants were visually fixated and paid more attention on the presented VSDs including a target object.Results: Both total fixation duration and average fixation count were statistically significant. Participants visually fixated on the target object longer and more often when the object was being used in the presented VSDs. For AOI (Area Of Interest) time of the first fixation, after analyzing only a partial group that had the data match due to the difference in gaze pattern per subject, the average AOI time of the first fixation was shown to be faster when using an object in 6 out of 10 objects.Conclusion: This study supports the inclusion of an action of an object usage in VSDs suggesting that the act of object usage can partially influence the visual attention pattern of a user.

What's in a Photograph? The Perspectives of Composition Experts on Factors Impacting Visual Scene Display Complexity for Augmentative and Alternative Communication and Strategies for Improving Visual Communication.

Purpose Visual scene displays (VSDs) can support augmentative and alternative communication (AAC) success for children and adults with complex communication needs. Static VSDs incorporate contextual photographs that include meaningful events, places, and people. Although the processing of VSDs has been studied, their power as a medium to effectively convey meaning may benefit from the perspective of individuals who regularly engage in visual storytelling. The aim of this study was to evaluate the perspectives of individuals with expertise in photographic and/or artistic composition regarding factors contributing to VSD complexity and how to limit the time and effort required to apply principles of photographic composition. Method Semistructured interviews were completed with 13 participants with expertise in photographic and/or artistic composition. Results Four main themes were noted, including (a) factors increasing photographic image complexity and decreasing cohesion, (b) how complexity impacts the viewer, (c) composition strategies to decrease photographic image complexity and increase cohesion, and (d) strategies to support the quick application of composition strategies in a just-in-time setting. Findings both support and extend existing research regarding best practice for VSD design. Conclusions Findings provide an initial framework for understanding photographic image complexity and how it differs from drawn AAC symbols. Furthermore, findings outline a toolbox of composition principles that may help limit VSD complexity, along with providing recommendations for AAC development to support the quick application of compositional principles to limit burdens associated with capturing photographic images. Supplemental Material https://doi.org/10.23641/asha.15032700.

Personalization of Visual Scene Displays

Visual scene displays (VSDs) are becoming an increasingly popular method of message representation within augmentative and alternative communication (AAC) supports; however, design factors can influence the effectiveness of these images as communication supports. One issue that has come to light in recent years is the fact that selecting personalized VSDs, which depict the person with complex communication needs or an individual with whom they are familiar, is preferred over generic VSDs, which depict unfamiliar individuals. Although personalization is likely an important factor in the usability of VSDs, these images may be difficult for clinicians to obtain. As such, compromises must be identified. The purpose of this study was to explore the effects of controlling personal relevance factors (i.e., age and gender of the people depicted in generic VSDs) on the image preference patterns of adults with and without aphasia. Results from three very preliminary study summaries indicate that gender and age are both mitigating factors in image preference, as males tended to indicate preference for VSDs containing males over those containing females. In addition, females tended to indicate preference for females of a similar age depicted in VSDs.

Personalized AAC Intervention to Increase Participation and Communication for a Young Adult With Down Syndrome

Many adults with Down syndrome (DS) experience difficulty with speech production, and intelligibility challenges and communication breakdowns are common. Augmentative and alternative communication (AAC) intervention can provide important supports for persons with these complex communication needs, but must be customized to address the goals, strengths, and needs of the individual. This article provides a description of a personalized AAC intervention for a young adult with DS whose speech was frequently unintelligible. The AAC intervention made use of a video visual scene display (VSD) approach, and was investigated in two separate studies in two key community settings: An inclusive post-secondary education program (a single-case reversal ABAB design), and a community shopping activity (a non-experimental AB case study design). The participant demonstrated sharp increases in successful communication and participation in both settings following the introduction of the video VSD, and both the participant and key stakeholders viewed the intervention positively. The results provide preliminary evidence that personalized AAC intervention, including the use of a video VSD approach, can provide important supports for communication and participation in community settings for adults with DS and complex communication needs.

Effectiveness and Utility of Visual Scene Displays as Communication Supports for Adults and Children With Complex Communication Needs: Perspectives of Clinicians and Researchers.

Purpose The aims of this research study were to describe the perspectives of clinicians and researchers regarding the effectiveness of visual scene displays (VSDs) as communication supports in order to inform the translation of research to clinical practice. Method An online survey was completed by 298 clinicians and nine VSD researchers. Participants rated the effectiveness of VSDs in supporting a variety of communication functions and their strength of agreement regarding statements of VSD utility. Results Regardless of population served, clinicians and researchers rated VSDs as effective at supporting a range of communication functions. Participants also tended to agree with statements of VSD utility. Conclusions The current study highlights the perceptions of clinicians and researchers regarding the uses and benefits of VSDs for adults and children with complex communication needs and provides an initial measure of the current translation of research to clinical practice. Results indicate that VSDs can be used to support a variety of communication functions and utilities; however, further work is necessary to explore additional communication functions and delineate similarities and differences of VSD uses for differing populations. Supplemental Material https://doi.org/10.23641/asha.14551344.

Clinical and Research Perspectives on Visual Scene Displays

Purpose The purposes of this study were to measure the perceptions of speech-language pathologists (SLPs) and researchers regarding the design of visual scene displays (VSDs) and to investigate the perceived difficulties of SLPs in regard to the acquisition and programming of VSDs. Method We recruited 270 SLPs and nine researchers who focus on VSDs to complete an online survey. The SLP survey addressed both perception of VSD design and difficulties associated with implementing these images as communication supports in the clinical realm. The researcher group was surveyed solely regarding their opinions associated with VSD design. Results SLPs and researchers both ranked high-context VSDs as more effective at supporting communication than low-context VSDs. In addition, both groups ranked VSDs with task-engaged people as more effective at supporting communication than VSDs with camera-engaged people or no people. SLPs reported difficulties with acquiring high-quality VSDs from online sources as well as difficulties with maintenance and upkeep of VSDs. In addition, a large portion of SLPs reported challenges with representing complex concepts through VSDs. Conclusions Results from this study provide evidence of general agreement between SLPs and researchers regarding perception of VSD design. SLPs continue to report challenges with the acquisition and programming of VSDs. Further research is needed to develop evidence-based guidelines to support clinical endeavors in these areas.

Visual Attention to Cued Targets in Simulated Aided Augmentative and Alternative Communication Displays for Individuals With Intellectual and Developmental Disabilities

Purpose Many aided augmentative and alternative communication (AAC) systems require the use of an external display that is represented via a visual modality. It is critical to evaluate and understand visual–perceptual processing in individuals with disabilities who could benefit from AAC. One way to evaluate how individuals process visual materials is through research-based automated eye-tracking technologies that obtain a fine-grained stream of data concerning gaze paths of visual attention. Method The current study examined how individuals with autism spectrum disorder ( n = 13), Down syndrome ( n = 13), intellectual and developmental disabilities ( n = 9), or typical development ( n = 20) responded to a spoken prompt to find a thumbnail-sized navigation key within a complex AAC display, including a main visual scene display (VSD) and a navigation bar of four thumbnail-sized VSDs. Stimuli were presented on a monitor containing automated eye-tracking research technology that recorded patterns of visual attention. Results Participants across groups spent more time fixating on a target thumbnail VSD navigation image after the presentation of the spoken cue to look at the target, compared to before the presentation of the spoken cue; they also spent more time looking at the target thumbnail VSD than the other thumbnail-sized VSDs in the navigation bar after the cue. Discussion Participants were able to locate the target thumbnail VSDs, even within the context of a visually complex AAC display. Implications for the design of AAC displays and for assessment of comprehension are discussed.

Exploration of multimodal alternative access for individuals with severe motor impairments: Proof of concept

ABSTRACT Many individuals with complex communication needs and severe motor impairments are unable to control technologies through conventional means and require alternative access techniques to achieve accurate and efficient access. With current alternative access techniques, individuals with severe motor impairments are limited in that they can only use one access technique at a time. The purpose of this project was to test proof of concept of a new multimodal access technique which integrated eye gaze and single switch scanning selection techniques. The aims were to investigate the learning patterns of two adults with severe cerebral palsy when first introduced to the multimodal access technique and then to compare the accuracy and efficiency of multimodal to single-modality access when selecting targets on an AAC visual scene display. The participants learned to use the multimodal access technique; they demonstrated improvements in their accuracy of selection across sessions and attained at least 80% accuracy within 3-15 training sessions. Both participants achieved similar accuracy with multimodal access compared to single-modality, but took longer to select targets with multimodal access compared to single-modality. The potential utility of multimodal access is explored as well as important avenues for future research.

Supporting Peer Interactions in the Inclusive Preschool Classroom Using Visual Scene Displays

Inclusive preschool classrooms have become increasingly common in recent decades, affording opportunities for children with and without disabilities opportunities to interact and develop positive relationships. Children with disabilities may be unintentionally excluded from these interactions due to communication differences. This paper discusses one assistive technology tool, called visual scene displays (VSDs), for supporting communication and interaction between children. Specifically, the paper aims to: (a) describe VSDs; (b) discuss why VSDs are particularly well-suited for supporting social communication between young children; (c) describe how VSDs are created and introduced to children; and (d) provide examples of how VSDs may be implemented during two common preschool activities: object play and shared storybook reading. Finally, resources for creating VSDs are offered.

Video Visual Scene Displays with Dynamic Text: Effect on Single-Word Reading by an Adolescent with Cerebral Palsy.

The purpose of this study was to investigate the effect of the Transition to Literacy (T2L) feature within video visual scene displays (VSDs) on the single-word reading of academic vocabulary concepts (i.e., weather concepts) by a pre-literate adolescent with cerebral palsy and minimal speech. A single-subject, multiple probe, across word sets design was used to evaluate the effects of the intervention. The intervention used an AAC app programmed with video VSDs embedded with hotspots with the T2L feature to teach the adolescent 12 academic vocabulary words. The adolescent acquired all target words successfully with only minimal exposure to the written words through the app and was able to generalize her learning to two novel tasks. Using Tau-U to evaluate the size of the observed effects, there were very large effects across all word sets. The findings from this study demonstrate the effectiveness of the T2L feature to improve single-word reading in pre-literate individuals with minimal speech. The use of video VSDs and T2L technology together may offer professionals a unique way to complement current AAC devices and literacy instruction for adolescents with minimal speech who are pre-literate.

Supporting Communication and Participation in Shared Storybook Reading Using Visual Scene Displays.

Supporting Communication and Participation in Shared Storybook Reading Using Visual Scene Displays.

Speech-Language pathologists’ decisions when designing an aided AAC display for a compilation case study of a beginning communicator

Purpose The current study sought to identify decisions speech-language pathologists (SLPs) make and their rationale for those decisions when designing an AAC display for a hypothetical school-aged child who was a beginning communicator. Augmentative and alternative communication (AAC) displays support communicative interactions for children who are unable to use spoken language to meet their communication needs. Children who are at the early stages of learning language, often termed beginning or emerging communicators, may rely heavily on the AAC display to support comprehension and expression. Thus, it is important for speech-language pathologists (SLPs) to design a display that facilitates beginning communicators’ communication development. Methods This paper reports on the answers related to a specific case study presented in an online survey completed by 77 SLPs with experience in AAC. Results Data illustrated design decisions driven by child-specific characteristics and informed by research regarding type of layout, representation, and the importance of various design features. Conclusions Suggestions for future research are provided, including identifying goals and communicative functions best supported by different types of displays, clinicians’ challenges/successes growing a system with the child, and determining the importance of symbol arrangement and organisation in communicative tasks by AAC users. Implications for Rehabilitation Identification of current practices reported by speech-language pathologists given a case study to develop and implement future research. A gap between research and practice suggests the need for better dissemination and application of research supporting the inclusion of people engaged in the event depicted in a visual scene display. Foundations for future research to provide guidance regarding best design of and communication goals supported by visual scene displays and grid-based displays.