Augmentative Communication Research Articles

Making my voice and owning its future

This article explores disabled experience and the future of technologies relating to augmentative and alternative communication (AAC). This field includes people’s use of AAC devices, typically in combination with other...

Tact Training with Augmentative Gestural Support for Language Disorder and Challenging Behaviors: A Case Study in an Italian Community-Based Setting

Background: Gestures or manual signing are valid options for augmentative and alternative communication. However, the data in the literature are limited to a few neurodevelopmental disorders, and less is known about its application in the community setting. Objectives: This case report explores the feasibility and preliminary efficacy of tact training with augmentative gestural support intervention for a child affected by a language disorder with challenging behaviors in a community setting. Methods: Baseline assessments were conducted using the Verbal Behavior Milestone Assessment and Placement Program (VB-MAPP) and Griffiths Mental Developmental Scale-III (GMDS-III). The patient received six months of standard treatment, consisting of neuropsychomotor and speech therapy each twice a week, with improved cooperation in proposed activities, but no improvement in language. Afterward, a total of 24 sessions of tact training with augmentative gestural support interventions were performed. Data were collected by two independent observers and analyzed to measure language and behavioral outcomes. Results: VB-MAPP scores increased form minimal communication and social interaction at T0 (baseline) to improved compliance but unchanged language skills at T1 (after standard therapy). After tack training with augmentative gestural support (T2), VB-MAPP scores showed significant improvements, with notable increases in verbal operants, independence in communication, and intersubjectivity skills. GMDS-III scores at T2 also demonstrated growth in social, communicative, and cognitive skills. Additionally, challenging behaviors were reduced by more than 70% and nearly resolved by the end of the intervention. Conclusions: Personalized approaches appear to be essential for interventions tailored to developmental age. Further research is needed to determine the effectiveness of these approaches for other neurodevelopmental disorders, identify patient characteristics that may be predictors of outcomes to tailor the intervention, and explore the generalization of the results obtained with these strategies.

Communication Prototype for Post-Stroke Patients Using Electrooculography (EOG)

Background this prototype is intended for post-stroke patients experiencing disabilities in their daily activities, particularly in communication with others. The challenge they face is difficulty in communication, leading to a diminished quality of life for post-stroke patients. The purpose of innovating this communication aid prototype is to facilitate communication between post-stroke patients and caregivers. The method employed in the post-stroke communication aid prototype utilizes Electrooculography (EOG) signals generated from eye muscle movements during eye gazes, captured by the MaM Sense sensor. The variation in Analog-to-Digital Converter (ADC) values in the MaM Sense sensor is exploited to produce various forms of EOG signals. The resultant command signals from this method are processed by a microcontroller and displayed on a 20 x 4 Character LCD. Testing was conducted on 9 healthy individuals, comprising 5 males and 4 females. To ensure the prototype's functionality, testing was also performed on 1 post-stroke patient. The success rate of MaM Sense sensor readings was 80.5% for the 4 communication modes employed, involving 4 eye gaze movements: right gaze, left gaze, upward gaze, and downward gaze. Thus, the post-stroke communication aid prototype proves effective in assisting communication for post-stroke patients and aiding caregivers in understanding the patients' desires. In the future, a wireless system may be implemented for the acquisition of EOG signals attached to the face to minimize the use of cables.

Use of Augmentative and Alternative Communication Technology to Retrain Attention and Enhance Functional Communication Following Severe Traumatic Brain Injury: A Clinical Case Report

Purpose: This case report outlines the use of high-tech assistive technology in the form of an augmentative and alternative communication device to assess and treat a veteran with severe speech, language, and cognitive impairments status post severe blast injury. Method: The patient is a right-handed male Veteran of the United States Marine Corps who sustained catastrophic injury after stepping on a dismounted improvised explosive device. He presented to the outpatient clinic 4 years status post injury for continued speech, language, and cognitive interventions. Based on assessments and observations, the patient was diagnosed with spastic anarthria, probable apraxia of speech, aphasia, and attentional impairments. His functional communication was restricted to limited facial expression and eye movements interpreted by those who knew him well. This case reviews the detailed informal and novel assessment techniques and training methods used to implement high-tech assistive technologies as part of the rehabilitation plan designed to enhance functional communication. Results: Through systematic instruction and retraining visual and sustained attention, the patient was able to engage meaningfully with a high-tech device to increase functional personal choice when supported by a partner. Conclusion: This case demonstrates the role high-tech assistive technologies can play in the rehabilitation and functional communication efforts for persons with severe traumatic brain injury and cerebrovascular accidents.

Brain-computer interface applications in the aging population with Alzheimers and Parkinsons disease

Abstract. The rapidly aging population faces increasing prevalence of neurodegenerative conditions like Alzheimer's Disease (AD) and Parkinson's Disease (PD), which significantly impair cognitive and motor functions. There is an urgent need for increased research, awareness, and development of effective treatments to mitigate their impact on the affected individuals and their families. This paper delves into the history, mechanisms, and applications of brain-computer interfaces (BCIs) as innovative solutions to mitigate these impacts. BCIs enable direct brain-to-device communication, circumventing conventional neuromuscular routes. We highlight how neurofeedback and neuroplasticity facilitated by BCIs not only restore motor skills and cognitive functions but also enhance the overall quality of life for sufferers. Through applications in cognitive training, motor assistance, and advanced communication aids, BCIs present a significant promise despite the ethical and practical challenges they pose. The continual advancement of this technology promises to a future when its incorporation into treatment plans will be widely used and provide significant advantages for those with PD and AD.

Using large language models to accelerate communication for eye gaze typing users with ALS

Accelerating text input in augmentative and alternative communication (AAC) is a long-standing area of research with bearings on the quality of life in individuals with profound motor impairments. Recent advances in large language models (LLMs) pose opportunities for re-thinking strategies for enhanced text entry in AAC. In this paper, we present SpeakFaster, consisting of an LLM-powered user interface for text entry in a highly-abbreviated form, saving 57% more motor actions than traditional predictive keyboards in offline simulation. A pilot study on a mobile device with 19 non-AAC participants demonstrated motor savings in line with simulation and relatively small changes in typing speed. Lab and field testing on two eye-gaze AAC users with amyotrophic lateral sclerosis demonstrated text-entry rates 29–60% above baselines, due to significant saving of expensive keystrokes based on LLM predictions. These findings form a foundation for further exploration of LLM-assisted text entry in AAC and other user interfaces.

Imagining alternative futures with augmentative and alternative communication: a manifesto.

This manifesto seeks to challenge dominant narratives about the future of augmentative and alternative communication (AAC). Current predictions are mainly driven by technological developments-technologies usually being developed for different markets-and are often based on ableist assumptions. In online conversations and a discussion panel at the 2023 International Society for Augmentative and Alternative Communication conference, we explored alternative futures by adopting different starting positions. Our case is presented under five headings: questioning the dominance of predictions that artificial intelligence and brain-computer interfaces will define the future of AAC; resisting disability being framed medically, as a problem to be solved, yet acknowledging both the pleasures and pains of being disabled; declaring that people who use AAC-as cyborgs of necessity rather than choice-should have choice and ownership of our technologies; challenging notions of independence as the necessary end goal for disabled bodies and considering interdependence as a human right; imagining alternative futures in which all people who use AAC are accepted and embraced for our communication and self-expression. This manifesto is an invitation for further discussion, and we welcome responses. While our focus is AAC, and three of the authors use AAC, we believe that our stance could be relevant to other disability communities in turn. This paper is about who gets to imagine disability futures and whose voices are left out. It is about how uncritical these futures can be, often presuming values that disabled people, in all their diversity, may not share.

Ability to Select AAC Graphic Symbols by Severity of Alzheimer’s Disease and Number of Symbols

Purpose: This study aimed to examine the graphic-symbol selection ability of patients with dementia of Alzheimer's type (DAT) based on the severity of dementia by conducting symbol identification and symbol discrimination tasks with increasing numbers of symbols. These tasks were performed with both normal elderly individuals and DAT patients to explore how cognitive load affects them differently.Methods: A total of 21 normal elderly individuals, 17 mild DAT patients, and 20 moderate to severe DAT patients were assessed. They performed symbol identification and symbol discrimination tasks using the Augmentative and Alternative Communication Symbol Assessment Tool. The number of symbols increased in six steps, from 4 to 72. Accuracy and response time were measured for statistical analysis.Results: In the symbol identification task, there was no significant difference in accuracy among the three groups. However, normal group showed significantly faster responses compared to moderate to severe DAT group. In the symbol discrimination task, normal group showed higher accuracy and significantly faster responses than moderate to severe DAT group. As the number of symbols increased, response times increased across all groups, with more time required for the symbol discrimination task.Conclusion: It was confirmed that the greater the severity of DAT, the more cognitive load and impaired information processing were reflected in symbol selection. Findings suggest that response time is a more sensitive indicator of cognitive decline in DAT patients and highlight the need to adjust the number of symbols and presentation methods according to the severity of the patient’s condition in dementia interventions.

Probing clarity: AI-generated simplified breast imaging reports for enhanced patient comprehension powered by ChatGPT-4o

BackgroundTo assess the reliability and comprehensibility of breast radiology reports simplified by artificial intelligence using the large language model (LLM) ChatGPT-4o.MethodsA radiologist with 20 years’ experience selected 21 anonymized breast radiology reports, 7 mammography, 7 breast ultrasound, and 7 breast magnetic resonance imaging (MRI), categorized according to breast imaging reporting and data system (BI-RADS). These reports underwent simplification by prompting ChatGPT-4o with “Explain this medical report to a patient using simple language”. Five breast radiologists assessed the quality of these simplified reports for factual accuracy, completeness, and potential harm with a 5-point Likert scale from 1 (strongly agree) to 5 (strongly disagree). Another breast radiologist evaluated the text comprehension of five non-healthcare personnel readers using a 5-point Likert scale from 1 (excellent) to 5 (poor). Descriptive statistics, Cronbach’s α, and the Kruskal–Wallis test were used.ResultsMammography, ultrasound, and MRI showed high factual accuracy (median 2) and completeness (median 2) across radiologists, with low potential harm scores (median 5); no significant group differences (p ≥ 0.780), and high internal consistency (α > 0.80) were observed. Non-healthcare readers showed high comprehension (median 2 for mammography and MRI and 1 for ultrasound); no significant group differences across modalities (p = 0.368), and high internal consistency (α > 0.85) were observed. BI-RADS 0, 1, and 2 reports were accurately explained, while BI-RADS 3–6 reports were challenging.ConclusionThe model demonstrated reliability and clarity, offering promise for patients with diverse backgrounds. LLMs like ChatGPT-4o could simplify breast radiology reports, aid in communication, and enhance patient care.Relevance statementSimplified breast radiology reports generated by ChatGPT-4o show potential in enhancing communication with patients, improving comprehension across varying educational backgrounds, and contributing to patient-centered care in radiology practice.Key PointsAI simplifies complex breast imaging reports, enhancing patient understanding.Simplified reports from AI maintain accuracy, improving patient comprehension significantly.Implementing AI reports enhances patient engagement and communication in breast imaging.Graphical

Machine learning for predictive AAC: Improving speech and gesture-based communication systems

Augmentative and alternative communication (AAC) systems play a crucial role in supporting individuals with severe communication disabilities by providing accessible means of expression and engagement. However, many conventional AAC devices rely on manual input or basic predictive functions, which can limit communication efficiency and responsiveness. The application of machine learning (ML) to AAC offers new opportunities to enhance these systems, enabling them to provide faster, more accurate, and contextually relevant communication assistance. Advances in ML, particularly in predictive text, speech recognition, and gesture interpretation, allow AAC systems to adapt more intuitively to user needs, predicting intent based on usage patterns and multimodal data, such as voice and gestures. Current research highlights the potential of ML to address key gaps in AAC technology by creating more responsive, personalized systems that align with individual user behaviours. This study proposes a novel ML framework designed to integrate these capabilities, promising improvements in communication speed, user autonomy, and accuracy. By addressing the challenges and limitations of traditional AAC devices, this research aims to advance accessible communication solutions that empower users and improve quality of life.

Ethical Implications of AI-Driven AAC Systems: Ensuring Inclusivity and Equity in Assistive Technologies

This study investigates the ethical implications of Artificial Intelligence (AI) integration into Augmentative and Alternative Communication (AAC) systems, focusing on inclusivity, accessibility, and data privacy in assistive technologies. AI-driven AAC systems enhance communication for individuals with impairments by offering features like predictive text, speech recognition, and symbol-to-text translation. However, ethical challenges arise as these systems increasingly rely on sensitive user data, which could jeopardize privacy without strict safeguards. Furthermore, limitations in linguistic diversity within AI training data compromise inclusivity, potentially excluding users from varied cultural and linguistic backgrounds. Accessibility also remains an issue, as AI-driven AAC systems are often financially inaccessible for marginalized communities, exacerbating social inequities. Through a comprehensive analysis of current systems, including Proloquo2Go, Tobii Dynavox and Google’s Project Euphonia, this paper evaluates both the advancements and ethical shortcomings in AI-driven AAC technologies. The findings advocate for privacy-preserving AI practices, culturally diverse training datasets, and affordable AAC solutions to foster a more inclusive, accessible, and equitable digital communication environment.

Educação Inclusiva e Comportamento no Transtorno do Espectro Autista: Desafios e Estratégias de Intervenção para a Promoção do Desenvolvimento Escolar

Inclusive education aims to provide an accessible learning environment for all, including students with Autism Spectrum Disorder (ASD). This study investigates the main challenges faced by schools in the inclusion of students with ASD and presents pedagogical and behavioral intervention strategies that promote their academic and social development. Among the challenges identified, difficulties in communication, socialization, and adaptation to school routines stand out. Inclusive pedagogical practices, such as the use of Augmentative and Alternative Communication (AAC) and structured teaching, have proven to be effective in overcoming these barriers. In addition, the support of a multidisciplinary team, composed of psychologists, speech therapists, and occupational therapists, is essential to ensure that students with ASD receive the necessary support for their full academic and emotional development. This study reinforces the importance of ongoing teacher training and collaboration between schools, families, and health professionals in the inclusion process.

Preliminary Investigation of Context-Aware AAC with Automated Just-in-Time Cloze Phrase Response Options for Social Participation from Children on the Autism Spectrum.

Social participation for emerging symbolic communicators on the autism spectrum is often restricted. This is due in part to the time and effort required for both children and partners to use traditional augmentative and alternative communication (AAC) technologies during fast-paced social routines. Innovations in artificial intelligence provide the potential for context-aware AAC technology that can provide just-in-time (JIT) communication options based on linguistic input from partners to minimize the time and effort needed to use AAC technologies for social participation. This preliminary study used an alternating treatment design to compare the effects of a context-aware AAC prototype with automated cloze phrase response options to traditional AAC for supporting three young children who were emerging symbolic communicators on the autism spectrum in participating within a social routine. Visual analysis and effect size estimates suggest the context-aware AAC condition resulted in increases in linguistic participation, vocal approximations, and visual attention for all three children. While this study was only an initial exploration and results are preliminary, context-aware AAC technologies have the potential to enhance participation and communication outcomes for young emerging symbolic communicators on the autism spectrum and more research is needed.

SignifAI: French sign language translation based on deep learning and large language models

In this project, we propose SignifAI which is an AI-powered software that translates French Sign Language into spoken French. The goal of SignifAI is to remove the language barrier that separates the deaf and hearing communities. The translation process proves to be a major challenge for existing methods, as most of them rely on traditional Natural Language Processing (NLP) methods which solve the task in a similar way as spoken languages (e.g., paying more attention to the logics and reasoning). The drastic syntactical difference between sign and spoken languages, however, makes such approaches not as effective and efficient. In SignifAI, we propose to leverage a combination of deep learning models for sign gesture recognition and large language models (LLM) for sentence generation from recognized words. The former allows for precise detection of individual words, while the latter takes the recognized words as inputs and returns a fluently reformulated translated sentence. With extremely limited data available for French sign language, we created our own enhanced dataset through synthetically augmenting existing videos. Utilizing the SlowFast model for this task, we have achieved a maximum test accuracy of 98.9%, and a fluid translation process after implementing ChatGPTs API. The potential applications of SignifAI range from aiding communication between the deaf and the hearing, to helping the deaf place phone calls in numerical services. Ultimately, SignifAI strives to improve all aspects of a deaf persons life, and compensates for their speaking disabilities.

Environmental factors influencing choice of spelling and graphic symbols in communicative interactions of adolescents who use communication aids

This exploratory interpretative qualitative study aimed to investigate environmental factors influencing “in the moment” decisions about use of graphic symbols or spelling in face-to-face communicative interactions, by adolescents who use communication aids and are learning how to spell. The participants were six adolescents who used speech generating devices and their mothers. Data collection consisted of seven to eight communicative interactions between adolescents and their mothers and follow up interviews with the participants. Each dyad took part in three Zoom sessions. Researchers identified the communication mode shifts between graphic symbols and spelling during interactions and subsequently discussed with participants the reasons for these shifts. The interview data were analyzed using Charmaz’s constructivist grounded theory approach to coding. The analysis revealed three themes explaining environmental factors relevant to choosing spelling or graphic symbols in communicative interactions, which were: (a) features of the communication aid; (b) communication partner’s skills and their knowledge of the shared experience; and (c) opportunities to practice and use spelling. The findings provide insights into the importance of providing opportunities to practice and use spelling, communication partner’s skills and knowledge of shared experience and communication aid technology design which can help to facilitate spelling during communicative interactions.

Improving communication during damage control surgery: a survey of adult major trauma centres in England.

Deficiencies in non-technical skills can severely impede the functioning of teams in high-intensity scenarios, such as in damage control surgery for the critically injured trauma patient. Truncated preoperative checklists, modified from the standard World Health Organization preoperative checklist, and situational reporting at intervals during surgery are long-established practices in the military, and are recommended in the National Health Service guidelines on major incidents. These tools allow the multiprofessional team to create a shared mental model of the anaesthetic and operative plan, thereby improving team efficiency. Our aim was to establish whether adult major trauma centres in England are using truncated preoperative checklists and situational reporting for damage control surgery. An online survey was devised and distributed via the national programme of care for trauma in November 2020. Responses were received from all 23 adult major trauma centres in England. Nine centres (39.1%) reported using a truncated preoperative checklist for damage control surgery albeit in a variety of formats. Common components were blood products received and/or available, presence of allergies, tranexamic acid and antibiotic administration, availability of viscoelastic tests, equipment required, availability of cell saver, role allocation and reference to other personnel needed, and discussion of the plan. Twelve centres (52.2%) have formal policies in place for situational reporting. Again, these were in multiple formats but all focused on patient physiology to direct surgical planning. We have identified key components to advanced communication aids for damage control surgery, providing a foundation on which other major trauma centres can build their own versions of these potentially lifesaving tools.

An integrated prototyping tool to enhance interdisciplinary communication in smart product design

The increasing integration of smart products into society is noteworthy in the field of Prospective Ergonomics (PE), as they herald the emergence of novel products featuring yet-to-be-discovered interaction modalities. Literature highlights a lack of front-end prototyping approaches that transcend disciplinary skills and aid communication across diverse design fields, highly relevant in early-stage smart product development. This paper introduces an integrated, low-fidelity prototyping tool to enhance interdisciplinary communication and understanding of future products’ interactive and environmental qualities. A survey provides preliminary validation of the integrated tool’s format and content. Findings endorse the tool’s structure and flexibility in supporting understanding and communication of interactive and environmental qualities of smart product concepts across a range of design disciplines. As such we advance the discourse on PE by envisioning a low-fidelity prototyping tool to create intermediary objects supporting design efforts towards future products and their new and emerging interaction modalities.

Child, family and professional views on valued communication outcomes for non-verbal children with neurodisability: A qualitative meta-synthesis.

There are many children with neurodisability who are unable to rely on speech to communicate and so use a range of augmentative and alternative communication (AAC) methods and strategies to get their message across. Current instruments designed to measure the outcomes of speech and language therapy interventions lack specific attention to communication outcomes that are valued by non-verbal children with neurodisability, their families and support networks. This qualitative meta-synthesis was conducted to identify valued communication outcomes to inform the next stage of developing a novel outcome measure. To systematically identify and synthesise the qualitative evidence about which communication outcomes non-verbal children with neurodisability, their family members, healthcare professionals and educators think are important to achieve, specifically which communication outcomes are most valued by: (1) non-verbal children with neurodisability; (2) parents or other family members of non-verbal children with neurodisability; and (3) professionals who work with non-verbal children with neurodisability. A systematic search of bibliographic databases and the grey literature was undertaken to identify qualitative studies that included evidence of views expressed by children, family members, healthcare professionals and educators on outcomes in relation to the communication of non-verbal children with neurodisability. All papers meeting the inclusion criteria were quality appraised using the Critical Appraisal Skills Programme Qualitative checklist, although none were excluded on this basis. The data synthesis involved organising coded data into descriptive themes which were then synthesised into analytical themes. We found 47 papers containing qualitative data meeting the inclusion criteria from research situated in 14 countries. The views of 35 children, 183 parents, six other family members, 42 healthcare professionals and 18 educators are represented in the review. The included studies contained very few data reported by children themselves; most data were provided by adults, especially parents. Three main analytical themes were identified: Experiences of communication and expectations; adapting to and acceptance of AAC; and becoming an autonomous communicator. This meta-synthesis brings together the limited qualitative research findings about what parents, professionals and children consider are important communication outcomes for non-verbal children with neurodisability. The synthesis identifies key gaps in our knowledge about the perspectives of children and their siblings. This synthesis will inform primary research to understand valued communication outcomes in this group, and ultimately the development of a patient-reported outcome measure (PROM) that can be used to demonstrate the effect of interventions, at both clinical and service levels. What is already known on the subject Studies of children with cerebral palsy and autism spectrum disorder indicate that at least 25% of children with these conditions are non-verbal. Studies on the health outcomes of children with neurodisability have identified that communication is rated as important by parents and health professionals. There is an evidence gap about which communication outcomes are important to non-verbal children, their families and the people who work with them. What this paper adds to the existing knowledge This is the first synthesis of data that relates to communication outcomes for non-verbal children with neurodisability. This qualitative meta-synthesis identifies from previous research studies the communication outcomes valued by children who are non-verbal, their parents or other family members, and the professionals who work with them. The findings will be used to shape further primary research and the development of a novel patient-reported communication outcome measure for non-verbal children with neurodisability. It is anticipated that this will be used by clinicians to measure the effect of their interventions. What are the practical and clinical implications of this work? Clinicians should reflect on parents' experiences of communication with their child before discussing potential outcomes with them. Gaining insight into the lived experience of communication for non-verbal children and their families will help healthcare professionals to understand which goals are important to them and why. Few studies have specifically asked which communication outcomes are important for non-verbal children with neurodisability. Further exploration is needed to determine which communication outcomes non-verbal children and their families would like to see included in outcome measures used by clinicians.

Executive functioning and nonverbal cognitive factors associated with response to technology-assisted language intervention

ObjectiveTo explore potential differences in the relationship between executive function (EF) skills and language development when integrating augmentative and alternative communication technology into speech-language therapy for deaf/hard of hearing (DHH) children. MethodRandomized trial data were analysed to investigate this relationship among children who participated in a Technology-Assisted Language Intervention (TALI) compared to treatment as usual (TAU). Language samples were assessed for pre-post-intervention changes, including mean length of utterance in morphemes (MLU), mean turn length (MTL), and number of different words spoken (NDW). EF skills were measured with standardized parent-report assessment. ResultsThirty-seven DHH children were included (TALI n = 19 and TAU n = 18). Results of regression models indicated that higher EF skills were significantly (p < 0.05) associated with improvements in MLU, MTL, and NDW among children who received TAU. No significant associations between EF skills and any of the measures (MLU, MTL, NDW) were seen in children who received TALI. ConclusionThese results suggest that TALI may offer language learning support, particularly for DHH children with EF difficulties. Future research should investigate the direct relationship between EF measurements and language outcomes in TALI recipients. Establishing baseline EF measurements in DHH children could inform personalized strategies within language interventions and therapy.

Implications of Constraints on the Language of Emerging Aided Communicators and Intervention Decisions: A Tutorial

This tutorial offers speech-language pathologists (SLPs) an opportunity to consider the theoretical constructs of language acquisition theory and the multifactorial determinants that can impinge on the language acquisition process for children with complex communication needs. The system of constraints observed through the lens of optimality theory is discussed alongside intrinsic and extrinsic barriers associated with children who acquire their first language simultaneous to their introduction to aided augmentative and alternative communication (AAC) systems. This examination calls into question whether the language competence of emerging aided communicators may be masked by their ability to produce language that accurately represents their understanding of their native language. Specific to this question is whether vocabularies available to emerging aided communicators, opportunities to access aided language models, and intervention decisions made by SLPs may be implicated in the language acquisition and development process. Suggestions for clinical practice are offered as a means of mitigating adverse outcomes related to extrinsic constraints.