From characters to communication: using AI to Tailor personally relevant AAC systems for children

Communication is essential to each individual's quality of life. One must be able to communicate with others to build relationships, to contribute to society through a vocation, and most of all, to live independently. However, many people are unable to communicate in the most prevalent mode: speech. As many as eight to twelve per 1 ,000 people require a means to communicate in addition to, or as an alternative to, speech since they are unable to meet their communication needs with speech (Beukelman & Ansel, 1995; Light, Beukelman, & Reichle, 2003).When an individual cannot use speech to communicate competently enough to meet their daily needs, another means of communication is needed. Augmentative and alternative communication (AAC) is the use of expressive mediums, in addition to or in place of speech, to communicate. It is represented in two forms: unaided and aided (Johnston, Reichle, & Evans, 2004). Unaided AAC does not require the use of additional objects or materials; aided does require materials that are external to the user. Unaided AAC may take the form of pointing, gestures, or sign language. In contrast, aided AAC requires materials such as pictures, objects, or electronic aids.While various AAC systems offer options and benefits for communication, most individuals utilize a combination of aided and unaided AAC (Mirenda, 2009). In addition to aided AAC, individuals employ other forms of communication such as eye gaze, vocalizations, gestures, and facial expressions. Persons with AAC systems may use these conventional means of communication in addition to his/her AAC system, as the premise behind AAC is to allow the individual to utilize as many available skills and means of communication to promote effective and efficient communication (Downey & Hurtig, 2003). A multi-modal approach recognizes that an individual needs a communication system that is flexible and responsive to their needs in various communicative environments and situations. Other means of communication that may be paired with an aided AAC system are sign language and the use of residual speech (Beukelman & Mirenda, 2005).Music therapists provide therapeutic services to individuals with communication disorders. Though music therapists frequently work with clients to improve their communication skills, their training does not typically include use and maintenance of AAC systems. Therefore, simply having an AAC system does not guarantee its optimal use by the client or by the music therapist. Many times, the system may not be present in the communicative environment, the system might not be turned on, or the individual may not be prompted to use the AAC system (Johnston, Reichle, & Evans, 2004). Indeed, survey results of music therapists working with clients with autism indicated that few music therapists are consistently utilizing aided AAC with their clients who already have aided AAC systems (Gadberry, 201 1).Intentional CommunicationWhen a person is learning to communicate with the AAC system, focus is placed on supporting intentional communication. Since intentional communication requires another individual to be involved, intentional communication requires joint attention. Joint attention is a shared focus between two or more people on a common object, topic, or event (Reichle, Beukelman, & Light, 2002). Most authors agree that intentional communication involves joint attention and the purposeful conveyance of a message to another person.PromptsWhen assisting a person in development of expressive, intentional communication a therapist, educator, or caregiver often needs to prompt the individual to respond. Prompts are behaviors that encourage individuals to communicate. There are three types of prompts: (a) verbal, (b) gestural, and (c) physical (Bingham, 2005). Pointing is considered gestural; touching the individual's arm (as an indication for him to move it towards his AAC system) is a physical prompt. …

Introduction: The achievement of effective overall communication skills can be complex and challenging for children with Special Educational Needs (SEN) who use augmentative and alternative communication (AAC) systems (Light and McNaughton, 2014). LEGO®-based therapy is primarily been used with people on the autistic spectrum, as a social intervention to target communication and interaction skills (LeGoff, 2004). This presentation outlines the novel use of LEGO®-based therapy that has been adapted to enhance the development of communicative competence of children aged 7-11 who use AAC systems. Methods: This adapted LEGO®-based therapy approach is based on the principles of Baron Cohen’s Empathising-Systemising model (Baron-Cohen et al, 2005) and social constructivism. Children take turns to play different roles (a supplier, an engineer, and a builder). They are provided with a LEGO® set or allowed to freestyle, in order to build a model as a team. The role of the adult facilitator is to prompt the resolution of any social or communication difficulties that may arise. The gamification approach provides a balance between the development of social interaction and gaming (Mosley and Whitton, 2014) as the children collaboratively create a LEGO® model as a team. Outcomes: The initial findings from a series of case studies highlighted the potential benefits of using a gamification approach featuring adapted LEGO®-based therapy, to develop single or multiple communication competences in young children using AAC devices. The case studies provide insights into the development of multiple (linguistic, operational, social, strategic) communication competencies coupled with the influence of psychosocial factors and environmental support. These case studies highlighted the additional development of social skills, (including joint attention, problem solving, negotiation and both verbal and non-verbal communication), which enhance overall communicative competence. The project highlighted the importance of recognising and responding to linguistic, operational, social and strategic challenges faced by children using AAC devices during therapeutic intervention. Discussion, conclusions and recommendations: The initial findings illustrated in the case studies indicates the successful adaption of LeGoff’s (2004) original LEGO®-based therapy, as a therapeutic intervention to enhance single and multiple communicative competences of young AAC users. The use of adapted LEGO®-based therapy with AAC users is not without challenges, but these are not insurmountable. Practical and theoretical recommendations for the integration of adapted LEGO®-based therapy in educational and healthcare settings will be shared.

You have accessThe ASHA LeaderGet Social1 Jul 2015Where Social Media Meets AACThe tech-heavy world of augmentative and alternative communication is well-suited for social media sharing. Vicki ClarkeMS, CCC-SLP Vicki Clarke Google Scholar More articles by this author , MS, CCC-SLP https://doi.org/10.1044/leader.GS.20072015.np SectionsAbout ToolsAdd to favorites ShareFacebookTwitterLinked In The families in our clinic pushed me into the world of social media several years ago. After a little investigation, we jumped wholeheartedly into online socializing with a Facebook page, Twitter and Tumblr accounts, Pinterest boards, an updated blog, secondary blog and, more recently, Instagram. Although social media has its share of selfies, dessert pictures and complaints about traffic, it’s also a wonderful source of information, resource sharing and connections for speech-language pathologists interested in augmentative and alternative communication (AAC). As professionals and families enjoy better access to AAC through mobile technology, the need for support, advice and resource-sharing also increases. AAC specialists are in short supply and demand is growing. Social media allow specialists to quickly share information with parents, SLPs and teachers. Clients and pros can find information on AAC in a variety of social media formats. There are blogs, video/photo-sharing, link shares and chats. Each has its advantages, so let’s take a look! Blogs By far, blogs give the most detailed information about techniques, equipment, apps and concepts. Some of my favorite bloggers are parents of children using AAC. These writers bring us distinctive views into the realities of raising AAC users. We celebrate, empathize and learn with these parents as they discuss daily experiences. My favorites: Kreed’s World Uncommon Sense There are wonderful professional blogs as well—my top three: PrAACtical AAC Teaching Learners With Multiple Needs Speak For Yourself Twitter Twitter users typically share links, photos and brief tips. Look for resources from our sister specialties of assistive technology and special education, which often also are relevant to AAC. I follow people from a variety of these related specialties. People highlight posts through hashtags, so search #augcomm or #aac to find pertinent tweets. #WeSpeechies posts frequently about AAC as well. Here are some “peeps” I follow: Kate Munro @speechiekate. ISAAC @ISAAC_AAC. US Society for AAC @USSAAC. Bronwyn Hemsley @BronwynHemsley. Gail Bennell @gailbennell. Jeremy Legaspi @azspeechguy. Facebook Facebook users post prolifically about AAC. Families, companies, individuals and AAC users themselves post pictures, videos, links, articles and personal stories. There are also Facebook groups that support AAC users, families and professionals. Here are a few to look up: Augmentative Communication Resources & Help AAC Language Lab AAC for the SLP There are also specific groups for AAC users, including: AAC Users! AAC- Alternative Awesome Communicators Instagram and YouTube Instagram is a photo- and video-sharing site where AAC families and professionals post materials, ideas, successes and equipment. I just started dabbling on Instagram and find it a great place to share quick videos of techniques and materials or brag on my patients and their families. AAC doesn’t yet have a significant presence on Instagram, but I see value in expanding topical knowledge to a broader audience through Instagram, which also uses hashtags. Search for #augcomm, #aac and #ashaigers. YouTube offers a wide variety of videos about AAC. AAC users and their families share stories, professionals give technique tips, and manufacturers and application developers offer training on products. Type a topic of interest into the search field to find videos. Pinterest Pinterest allows users to organize images with captions and links according to customizable “boards” or topics. You can follow “pinners” or specific boards. It’s also easy to search for AAC pins, enthusiasts or topics. Some pinners I follow include: Lauren Enders PrAACtical AAC ASHA Community The ASHA community provides the opportunity for SLPs to ask and answer specialized questions about AAC-related subjects. In the platform’s discussion area for ASHA Special Interest Group 12, Augmentative and Alternative Communication, SLPs can collaborate, troubleshoot and share regarding AAC. Social media offer easy access to a lot of AAC knowledge from and for professionals and consumers. Various sites better serve different learning styles, so check them all out. Most professionals and families sharing on social media use several different sites. You can expand your resources by finding favorites on one site and seeking them out on other social media, exploring their followers and the people they follow, too! Author Notes Vicki Clarke, MS, CCC-SLP, owns Dynamic Therapy Associates, a private practice specializing in AAC in Kennesaw, Georgia. With more than 25 years in the AAC world, she also evaluates, treats, trains and consults at schools, conferences and hospitals. [email protected] Advertising Disclaimer | Advertise With Us Advertising Disclaimer | Advertise With Us Additional Resources FiguresSourcesRelatedDetails Volume 20Issue 7July 2015 Get Permissions Add to your Mendeley library History Published in print: Jul 1, 2015 Metrics Current downloads: 530 Topicsasha-topicsleader_do_tagasha-article-typesleader-topicsCopyright & Permissions© 2015 American Speech-Language-Hearing AssociationLoading ...

Supporting Families of Children Who Use AAC

Background and Objectives: For people who have difficulties in language, locationbased Augmentative and Alternative Communication (AAC) mobile applications recommend AAC symbols based on their location to assist communication in daily life. Using AAC symbols available at a specific location with smart devices having small screens requires additional actions like swiping, which can be burden for [This research was supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number : HJ21C0007). This research extends the poster presentation of ‘An AAC Board Hierarchy and User Interface for Communication Situation Switching in Location-Based AAC’ in ‘The 3rd East Asian Regional Conference on Augmentative and Alternative Communication: AAC Beyond Pandemic’ of Korean Society of Augmentative and Alternative Communication (KSAAC). Correspondence to Ki-Hyung Hong, Ph.D., Department of Service Design Engineering, Sungshin Women’s University, Seoul, Korea, e-mail: khhong@sungshin.ac.kr, tel: + 82 2 920 7525] AAC users. This study aims to make it easier for AAC users to explore symbols and improve efficiency of facilitation by reusing AAC resources. Method: We define AAC situation boards as collections of AAC symbols specifically needed for certain communication situations. Group AAC boards, on the other hand, are collections of AAC symbols that belong to the same categories. We designed a hierarchy to allow these situation and group AAC boards to be included as sub-boards in general and location AAC boards. We also developed a user interface that allows facilitators to manage AAC resources based on the hierarchy. The interface makes it easy for AAC users to select appropriate AAC symbols for the current situation through a situation-based symbol circulation and switching method. Result & Conclusion: We enhanced and expanded location and situation-based AAC mobile applications by utilizing a hierarchy of AAC resources and integrating a user interface for efficient management and utilization of these resources. To assess its usability, we conducted a user evaluation involving 44 facilitators. The results of the evaluation indicate that the location and situation-based AAC mobile application, with its ability to switch situations based on the AAC resource hierarchy, received high ratings in terms of satisfaction, convenience, and stability.

Background and Objectives: For people who have difficulties in language, locationbased Augmentative and Alternative Communication (AAC) mobile applications recommend AAC symbols based on their location to assist communication in daily life. Using AAC symbols available at a specific location with smart devices having small screens requires additional actions like swiping, which can be burden for [This research was supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number : HJ21C0007). This research extends the poster presentation of ‘An AAC Board Hierarchy and User Interface for Communication Situation Switching in Location-Based AAC’ in ‘The 3rd East Asian Regional Conference on Augmentative and Alternative Communication: AAC Beyond Pandemic’ of Korean Society of Augmentative and Alternative Communication (KSAAC). Correspondence to Ki-Hyung Hong, Ph.D., Department of Service Design Engineering, Sungshin Women’s University, Seoul, Korea, e-mail: khhong@sungshin.ac.kr, tel: + 82 2 920 7525] AAC users. This study aims to make it easier for AAC users to explore symbols and improve efficiency of facilitation by reusing AAC resources. Method: We define AAC situation boards as collections of AAC symbols specifically needed for certain communication situations. Group AAC boards, on the other hand, are collections of AAC symbols that belong to the same categories. We designed a hierarchy to allow these situation and group AAC boards to be included as sub-boards in general and location AAC boards. We also developed a user interface that allows facilitators to manage AAC resources based on the hierarchy. The interface makes it easy for AAC users to select appropriate AAC symbols for the current situation through a situation-based symbol circulation and switching method. Result & Conclusion: We enhanced and expanded location and situation-based AAC mobile applications by utilizing a hierarchy of AAC resources and integrating a user interface for efficient management and utilization of these resources. To assess its usability, we conducted a user evaluation involving 44 facilitators. The results of the evaluation indicate that the location and situation-based AAC mobile application, with its ability to switch situations based on the AAC resource hierarchy, received high ratings in terms of satisfaction, convenience, and stability.

Access to Justice: An SLP’s Guide to Helping Persons with Complex Communication Needs Voice Their Case

Artificial intelligence is increasingly being explored in augmentative and alternative communication (AAC) to support personalized and responsive communication. Integrating emotion recognition into AAC systems may help users convey emotion content in real time, enhancing social interaction and communication quality. This preliminary study utilized DeepFace, a facial emotion recognition tool, as a starting point for integrating emotion recognition into AAC systems. Twelve neurotypical children and four children with autism, aged 8-13 years, completed two tasks: an imitated (copy) emotion condition using static images and a semispontaneous emotion-elicitation condition. Similar to existing AAC devices, each participant completed an individualized calibration procedure to help tailor the model to their facial expressions. Number scale ratings of the user experience were also collected. Overall, the system demonstrated promising provisional accuracy, particularly for happiness, sadness, and calm (neutrality), although performance varied across emotions, participants, and task types. User experience ratings were generally positive, suggesting satisfaction and engagement with the system, with variability observed. Findings highlight the value of calibration and personalization in AAC. Future work should expand training data sets to better represent children's facial expressions, incorporate multimodal inputs, and explore the alignment of synthetic speech output with detected emotions. Broader studies including active AAC users, diverse developmental and cultural backgrounds, and more spontaneous real-world interactions are needed. These results support the preliminary utility of DeepFace as a possible starting point for integrating emotion recognition into AAC systems, but further research is warranted. Directions for user-centered development are also discussed. https://doi.org/10.23641/asha.32511465.

Training and Intervention in South Africa

A Non-Traditional Clinical Experience in Costa Rica

Non-speaking individuals with motor disabilities heavily rely on augmentative and alternative communication (AAC) text entry systems to communicate. However, designing, developing, and evaluating AAC text entry systems for users and researchers gives rise to several challenges, such as the difficulty of eliciting requirements for a highly heterogeneous user population using a wide variety of assistive devices. This research aims to address such challenges and bridge the gaps between AAC stakeholders while exploring the potential of artificial intelligence (AI)-powered AAC systems to cater to diverse AAC needs and wants. To achieve these goals, we propose an imperfect surrogate user model for AAC system design and evaluation, develop a dedicated machine learning language model for AAC purposes, and design an AI-driven text entry system Tinkerable AAC (TAAC) which is ready for user testing. The objective of these efforts is to ultimately reduce the communication gap between AAC users and their speaking partners.

In this qualitative case study, we investigated the perspectives of users of high tech augmentative and alternative communication (AAC) systems regarding their AAC systems, suggestions for others, and their successes and barriers. The five participants in this study had cerebral palsy and were at the postsecondary education level or the employment status. We found 5 main themes during the investigation: (a) day-to-day challenges; (b) experiences with AAC; (c) perceptions of preferable features and future AAC; (d) suggestions to others, including communication partners, professionals in the AAC field, and/or potential AAC users; and (e) adopted strategies. We will interpret these findings from three perspectives: (a) the participants' perspectives on AAC; (b) their perspectives on other people; and (c) their personal insights on themselves. Overall, participants indicated that AAC systems were indispensable in their daily lives, school settings, and work environments. However, all of the participants preferred to use their natural speech if possible, and thought that their AAC device was not a replacement for speech. Rather, they perceived their AAC device as a tool for useful and flexible communication.

Purpose: Shared book reading (SBR) is an evidence-based literacy activity that promotes the development of oral language and emergent literacy skills in young children; however, literature regarding best practices for parents of children with developmental disabilities under the age of 5 years, specifically those with complex communication needs, is severely limited. In this descriptive study, we aimed to better understand how families of toddlers/preschoolers incorporated augmentative and alternative communication (AAC) into SBR interactions. Method: Videos of SBR interactions were collected across five families with children between 35 and 48 months of age who used AAC systems. Three families had previous training in AAC use during SBR; two families did not have previous training and reportedly did not use their child's AAC system during typical SBR interactions at home. Thus, SBR without AAC was considered “typical” for these two families, and behaviors were compared across two conditions: SBR without AAC and SBR with AAC. Each video was coded for a variety of parent and child behaviors, including asking questions, repetitions, modeling/commenting, expanding/extending, defining vocabulary, affirmations, initiations, and responses. The communication modality used was coded for each behavior. Results: Analysis of the video coding revealed that parents used a variety of strategies and that children relied on multimodal means of communication during SBR. For the two families who had videos collected across two conditions, namely, SBR without AAC and SBR with AAC, family AAC use increased the overall duration of SBR activities. For all families, the frequency of parent SBR strategy use increased the SBR duration. Conclusions: This study adds to the limited research on SBR practices with toddlers/preschoolers who use AAC. Directions for future research and the implications of the study are discussed. Supplemental Material: https://doi.org/10.23641/asha.26240408

You have accessThe ASHA LeaderFrom My Perspective1 Jul 2015Are We Ready for Growing AAC Needs in Group Homes?As more adults with intellectual disabilities live longer and lose their parents, they need increased support with communication systems. Carrie KaneMS, CCC-SLP Carrie Kane Google Scholar More articles by this author , MS, CCC-SLP https://doi.org/10.1044/leader.FMP.20072015.5 SectionsAbout ToolsAdd to favorites ShareFacebookTwitterLinked In People with intellectual disabilities and complex communication disorders are living longer, higher-quality, independent and more productive lives. And that’s thanks, in part, to augmentative and alternative communication (AAC) systems and the services of speech-language pathologists. But beyond the skills of recommending the right AAC for this population and providing training, SLPs need to better understand the settings in which these adults live. No longer do they live in large institutions, but in more intimate and natural independent or small group homes. The days of state-supported institutional nursing facilities for people with developmental disabilities have slowly ended over the past few decades. It is more natural and comfortable—and more cost-effective—for these AAC users to live in the community, where they enjoy a higher quality of life. A crisis may be at hand, however, as aging parents lose the ability to care for adult children with intellectual disabilities and complex communication disorders who live at home. According to the 2013 State of the States in Developmental Disabilities report, 71.5 percent of people with these disabilities in 2011 lived with family caregivers. Over the next few decades, this population will flood the group home system as their parents age. A crisis may be at hand as aging parents lose the ability to care for adult children with intellectual disabilities and complex communication disorders who live at home. Higher-stakes AAC Communication is critical to a person’s independence. Family caregivers may tend to speak for the adult with a disability and anticipate the person’s needs more than staff at a group home. Independent means of communication, therefore, becomes that much more important when that adult moves into a new environment. In this transition, the SLP has a major responsibility in finding the most appropriate, functional, evidence-based AAC intervention. AAC intervention does not necessarily involve a high-tech speech-generating device (SGD). I use the term “communication system” to describe my AAC recommendations, because no one uses an SGD alone. A communication system is a person’s overall expressive arsenal, encompassing gestures, pointing, facial expressions, vocalizations, verbalizations, yes/no responses—and an SGD when applicable. But many other factors—beyond the skills of the adult with intellectual disabilities and our communication recommendations—affect successful AAC use. Communicative success is a team effort among the client, the SLP, families, and paid caregivers and group home staff. Some staff members are highly supportive; some are not. Informal assessment of the AAC user’s living environment is crucial. It can be a delicate process to help staff members see the purpose of a communication intervention. However, if they don’t buy in to the communication-system recommendation and plan, there is a high risk of abandonment. (Encouragingly, in my state of Pennsylvania, the group home system clearly values communication—I have seen a higher number of clients from these homes in my practice.) Challenges Though now accepted as a better solution than nursing homes for people without complex medical conditions, group homes have their own challenges. Moving to a group home is a major adjustment for people who typically have lived their entire lives with their families and who often struggle with change. Another challenge in group homes is staff turnover. Hourly wages for workers in community intellectual/developmental programs averaged $10.14 per hour, according to the 2013 State of the States in Developmental Disabilities report, and a 2011 Paraprofessional Healthcare Institute report noted that almost half of direct-care workers (including group home staff) live below the federal poverty level. The work can be rewarding, but is often psychologically and physically challenging, so staff turnover is high. Frequent turnover, unfortunately, is confusing and frightening, and can lower residents’ quality of life. I have seen many adults with intellectual disabilities and complex communication disorders go years, if not decades, without AAC intervention. It is especially painful when they used AAC in school, but transitioned into the adult world with no reliable means of expression because the device belonged to the school, was abandoned or became obsolete. And although nursing homes provide speech support onsite, group home residents must receive it as outpatients and rely on group home staff for AAC support. Our charge SLPs must be advocates, even cheerleaders, for our patients and residents. We need to help the caregivers understand how to develop and foster communication with their residents instead of just anticipating their needs. Before receiving AAC, for example, a resident may have been able to communicate only using yes/no responses, so we need to teach caregivers about the communication system’s range, access methods and time requirements (someone who scans will communicate more slowly than someone with direct-selection access, for example). We should also see that group home staff add communication goals to clients’ mandated plans of care. We can do this by modifying our documented goals to fit the group home setting with simple, straightforward, easy-to-measure goals. We must also ensure that residents’ outpatient treatment incorporates staff training on device care, maintenance and programming. Residents’ successful use of AAC is directly linked to the caregivers around them. Speech-language treatment sessions are short-term—a few months at the most. It is ultimately up to the system around the resident to carry over our recommendations and techniques. If caregivers are not educated and invested in improving the resident’s use of the device, abandonment is inevitable. If caregivers are not educated and invested in improving the resident’s use of the device, abandonment is inevitable. Adults with intellectual disabilities and complex communication needs are living longer, and communication interventions will continue to be a medical necessity. An AAC recommendation is not a one-time process, as SGD users may need many device upgrades throughout their lives. Determining their best communication system is not the end of the process—it is only the beginning. Author Notes Carrie Kane, MS, CCC-SLP, is an assistive-technology professional at the Good Shepherd Rehabilitation Network in Allentown, Pennsylvania. She is an affiliate of ASHA Special Interest Group 12, Augmentative and Alternative Communication. [email protected] Advertising Disclaimer | Advertise With Us Advertising Disclaimer | Advertise With Us Additional Resources FiguresSourcesRelatedDetails Volume 20Issue 7July 2015 Get Permissions Add to your Mendeley library History Published in print: Jul 1, 2015 Metrics Current downloads: 753 Topicsasha-topicsleader_do_tagasha-article-typesleader-topicsCopyright & Permissions© 2015 American Speech-Language-Hearing AssociationLoading ...

Many non-speaking people use augmentative and alternative communication (AAC) systems to assist them to communicate with other people. Access to an AAC system is generally slow for its user, who may have other disabilities as well as being non-speaking. An AAC system can contain stored words, messages and stories for use in communication, and there can be a large quantity of such information for the user to search through and select from while he or she is trying to participate in conversation. New user interface techniques are required to assist a user to navigate through the information stored in an AAC system and select appropriate items for use during live conversation. Information visualisation techniques may be able to assist a user to overview and appraise the contents of an AAC system as part of that selection process, and therefore improve its efficiency of use. This paper outlines key AAC issues and explores visualisation techniques which may be of value in an AAC context.