Spoken Word Research Articles

Documentary songwriting in prison: ‘I am good ‘bout myself’

The purpose of this study was to investigate the nature of prison residents’ songwriting experiences while engaged in Documentary Songwriters (DocSong) collaborative songwriting pedagogy. The research implemented a qualitative, case study portraiture design inside a prison in which the experiences of an individual ‘story source’ and group were explored. Key themes originated from the observational, spoken, audio-recorded and sung data procured from a week-long DocSong workshop. The songwriting participants and facilitator created a community that transformed the prison space into an environment in which the residents experienced individual actualization and collective liberation and liberated and restorative music creativity. The songwriting community members shared personal narratives that were replete with emotion, while also demonstrating and supporting each other’s emotional vulnerability and resilience. The group collaboratively created songs from spoken word using the DocSong approach, forming a strong emotional bond throughout the songwriting process. The individuals’ interactions within the group suggested that confidence, patience and the willingness to share were also practised within the songwriting context.

Spoken Word in M. S. Shchepkin’s Art Work

Spoken Word in M. S. Shchepkin’s Art Work

Silent Signals to Spoken Words: A Sign Language Conversion to Text and Speech

Abstract: By leveraging machine learning, an innovative system has emerged to connect sign language with text-to-speech capabilities. Powered by sophisticated algorithms, this pioneering solution conversion of visual expressions into textual representations and then vocalizes it into spoken words. This fusion of technology stans as a transformative asset, facilitating seamless communication for individuals facing hearing impairments. By dismantling obstacles, this advancement fosters inclusive and accessible interactions, ultimately elevating the standard of living for members of the hearing- impaired population

Can Words Speak Louder Than Actions? Using Top Management Teams’ Language to Predict Myopic Marketing Spending

Myopic marketing spending—curtailing marketing and research and development expenses to boost earnings—damages firms’ long-term value. Despite this, top management teams are often myopic, and by the time investors or boards detect such short-termism, it is too late to react or intervene. This research introduces a novel prediction method by analyzing the language top management teams use in earnings calls, specifically focusing on marketing and earnings emphasis, to predict future instances of myopic marketing spending. Through linguistic dependency parsing of almost 11 million sentences extracted from nearly 25,000 quarterly earnings call transcripts of 1,197 firms between 2008 and 2019, the authors demonstrate that the proposed approach can predict myopic marketing spending at a quarterly frequency for up to one year in advance. They find that an increase of one standard deviation in earnings emphasis is associated with a 23.68% increase in the likelihood of future myopic marketing spending. Investments based on the proposed approach produce 1.61% additional annual abnormal returns compared with models that exclusively use known predictors of myopic marketing spending, while offering earlier foresight and more frequent opportunities for intervention. This reduces information asymmetry for investors and boards of directors.

Delineating memory reactivation in sleep with verbal and non-verbal retrieval cues.

Sleep supports memory consolidation via the reactivation of newly formed memory traces. One way to investigate memory reactivation in sleep is by exposing the sleeping brain to auditory retrieval cues; a paradigm known as targeted memory reactivation. To what extent the acoustic properties of memory cues influence the effectiveness of targeted memory reactivation, however, has received limited attention. We addressed this question by exploring how verbal and non-verbal memory cues affect oscillatory activity linked to memory reactivation in sleep. Fifty-one healthy male adults learned to associate visual stimuli with spoken words (verbal cues) and environmental sounds (non-verbal cues). Subsets of the verbal and non-verbal memory cues were then replayed during sleep. The voice of the verbal cues was either matched or mismatched to learning. Memory cues (relative to unheard control cues) prompted an increase in theta/alpha and spindle power, which have been heavily implicated in sleep-associated memory processing. Moreover, verbal memory cues were associated with a stronger increase in spindle power than non-verbal memory cues. There were no significant differences between the matched and mismatched verbal cues. Our findings suggest that verbal memory cues may be most effective for triggering memory reactivation in sleep, as indicated by an amplified spindle response.

Transforming Written and Spoken Words into Quantitative Data for Palliative Care Research Using Natural Language Processing (ME301)

Transforming Written and Spoken Words into Quantitative Data for Palliative Care Research Using Natural Language Processing (ME301)

Presence and Pronouns: An Exploratory Investigation into the Language of Social VR

As social virtual reality (VR) continues to grow as a medium for digital communication, sustaining presence among communicators remains one of the main constructs that researchers and practitioners use to assess the quality of user experience. In the present paper, we explore language patterns as a behavioral link to presence. We accomplished this through an exploratory text analysis of over 4,800 min of conversation in social VR, consisting of over 130,000 spoken words from 126 participants. We observed that the use of self-references and collective references positively correlated to social presence and spatial presence. Furthermore, median interpersonal distance between communicators was positively associated with using impersonal pronouns, suggesting that participants who stood farther apart from their interlocutors tended to speak in more impersonal terms. Our work sheds light on the possible psychological mechanisms behind presence and the potential of using speech data to help build systems that enhance user engagement.

Nilai-Nilai Pendidikan Karakter dalam Buku “Saleh Ritual Saleh Sosial” Karya Kh. A. Mustofa Bisri

Character education has always been an important pillar of education in facing various problems. The problem with our education is that we often encounter students who summarize the things they have done, fight, have low respect for parents and teachers, speak impolite words in communication and often use harsh and even dirty words.This study focuses on the values of character education contained in the book "Saleh Ritual Saleh Sosial" by KH. A. Mustofa Bisri. In this study, the author aims to reveal some of the character education values contained in the book. The approach used in this research is a qualitative approach. While the method used is content analysis, a method used to reveal a large text. organized into main summaries.As for the values of character education found, there are seventen values of character education, namely 1) Adil, 2) Dermawan`, 3) Musyawah 4)Tawadhu` 5) Saleh, 6) Manusiawi, 7) Sabar, 8) Syukur, 9)Takwa, 10) Ridha, 11) Guyub, 12) Jejeg, 13) Istiqamah, 14) Iqtishat, 15)Halim, 16) I`tidal, 17) Sak Madyo.

SayHello: An intelligent peer-to-peer polyglot voice-to-voice conversation application

As the entire world is becoming increasingly a global village, the need for reliable, smooth, and easy-to-use applications that facilitate the communication process between people speaking different languages worldwide becomes essential, especially in the tourism industry. While numerous online and mobile applications attempt to bridge the linguistic gap using text-to-text, text-to-voice, or voice-to-text-to-voice translators, they often fall short due to constraints such as the need for a single shared device, manual setup of speaker’s gender and preferred language, and an inability to communicate from a distance. These applications struggle to mimic the practical nature of real-time multilingual conversations where immediate and clear communication is paramount. This paper introduces an intelligent peer-to-peer polyglot voice-to-voice mobile application to facilitate the communication of people speaking different languages worldwide transparently mimicking a live conversation whether the involved parties are close to each other or at a nearby distance. People can interact with others transparently using their preferred language, irrespective of others’ languages, while the application automatically recognizes the language, gender of the speaker, and spoken words with very high accuracy. Five languages were implemented in the developed application as a proof-of-concept, and it is designed to smoothly and simply adapt more in future updates.

Making mindfulness accessible: Reflections on the Calming Course

A mindfulness intervention was introduced in a residential setting to reduce individuals felt anxiety or stress in challenging situations. Previous attempts at mindfulness in this setting were difficult for residents to access leading to the implementation of the Calming Course. Fourteen individuals took part in the intervention with ten completing the course. Here reflections are made on the process of designing the course as well as changes made. Reflections also explore the challenges with adequate outcome measures, potential stigma and whether adaptations may be reductionist.Easy to read summary■Practising mindfulness may decrease feelings of stress and anxiety■In a residential setting mindfulness exercises needed some changes to support individuals to engage in them■Some of these changes included clearer instructions and less spoken words■Thoughts about what was successful, what could be better and things to think about for future work are discussed

Enhancing Lip Reading: A Deep Learning Approach with CNN and RNN Integration

This research introduces an innovative approach to enhance lip reading-based text extraction and translation through the integration of a double Convolutional Neural Network (CNN) coupled with Recurrent Neural Network (RNN) architecture. The proposed model aims to leverage the strengths of both CNN and RNN to achieve superior accuracy in lip movement interpretation and subsequent text extraction. The methodology involves training the double CNN+RNN model on extensive datasets containing synchronized lip movements and corresponding linguistic expressions. The initial layers of the model utilize CNNs to effectively capture spatial features from the visual input of lip images. The extracted features are then fed into RNN layers, allowing the model to grasp temporal dependencies and contextual information crucial for accurate lip reading. The trained model showcases its proficiency in extracting textual content from spoken words, demonstrating an advanced capability to decipher nuances in lip gestures. Furthermore, the extracted text undergoes a translation process, enabling the conversion of spoken language into various target languages. This research not only contributes to the advancement of lip reading technologies but also establishes a robust foundation for real-world applications such as accessibility solutions for individuals with hearing impairments, real-time multilingual translation services, and improved communication in challenging acoustic environments. The abstract concludes with a discussion on the potential impact of the double CNN+RNN model in pushing the boundaries of human-computer interaction, emphasizing the synergy between deep learning, lip reading, and translation technologies

ASL Spelling Detection Using Deep Learning

Abstract: Understanding American Sign Language is a difficult task for anyone who wish to communicate with the deaf and dumb. It creates a gap between people. This paper introduces a finger spelling recognition system designed to help bridge that gap. Our system uses a camera and specialized CNN architecture to understand hand gestures that represent letters. This approach offers a few key benefits: it uses the familiar alphabet, can be learned quickly, and works for people with different levels of hearing loss or sign language experience. This technology has the potential to make communication much easier for many people, especially in situations where spoken words or traditional signs might not be the best option. It detects the ASL alphabets and constructs words and sentences, also known as Finger-Spelling

InsightStream: A Real-Time Perspective on Classroom Environment

Abstract: Traditionally, understanding classroom environment relies on subjective observations and post-hoc surveys. "Insight Stream" proposes a paradigm shift, offering a real-time, data-driven perspective through machine learning-powered facial emotion detection. This project leverages AI to analyse student facial expressions during class, capturing the emotional undercurrents in real-time. By delving beyond spoken words, "Insight Stream" aims to: Quantify classroom engagement: Detect emotions like boredom, confusion, and excitement to gauge real-time student engagement and adapt teaching methods accordingly. Identify hidden anxieties: Uncover subtle cues of anxiety or discomfort that may go unnoticed, allowing for proactive support and personalized interventions. Optimize teaching delivery: Track shifts in emotional response to different teaching styles and materials, enabling instructors to fine-tune their methods for maximal impact. Foster well-being: Monitor overall emotional climate to ensure a positive and supportive learning environment, contributing to student well-being and academic success. "Insight Stream" goes beyond just observing the classroom - it delves into the hearts and minds of students, offering a real-time window into their emotional tapestry. This project holds immense potential to revolutionize teaching and learning, creating a dynamic and data-driven environment that caters to the holistic needs of every student.

Achieving creative collaboration between language teachers and artists

Inspired by academic thinking and practice-based work on creativity and education, and creative approaches in language education, this article is based on outcomes of the project ‘Creating Welcoming Learning Environments: Disseminating Arts-Based Approaches to Including All Learners’, funded by the UK Arts & Humanities Research Council. The project brought together creative artists specializing in filmmaking, drama, crafting, poetry and the spoken word, textiles and music with teachers in the south-west of England, who collaborated on integrating arts-based practices into language teaching, learning and assessment. The focus of the project was to generate new ways of teaching a) children developing English as an Additional Language (EAL) in mainstream primary, secondary and special schools and b) all children in schools where there is a commitment to building an ethos of inclusion and diversity in relation to languages and cultures. The article reports on how co-operative, collaborative workshops were used to bring teachers, creative artists and researchers together in a way which valued everyone’s knowledge and expertise. A thematic analysis of data from evaluation interviews undertaken with the participating teachers is provided and the potential for workshop-based professional development opportunities as a way of achieving creative collaboration in schools is discussed.

I know how you'll say it: evidence of speaker-specific speech prediction.

Most models of language comprehension assume that the linguistic system is able to pre-activate phonological information. However, the evidence for phonological prediction is mixed and controversial. In this study, we implement a paradigm that capitalizes on the fact that foreign speakers usually make phonological errors. We investigate whether speaker identity (native vs. foreign) is used to make specific phonological predictions. Fifty-two participants were recruited to read sentence frames followed by a last spoken word which was uttered by either a native or a foreign speaker. They were required to perform a lexical decision on the last spoken word, which could be either semantically predictable or not. Speaker identity (native vs. foreign) may or may not be cued by the face of the speaker. We observed that the face cue is effective in speeding up the lexical decision when the word is predictable, but it is not effective when the word is not predictable. This result shows that speech prediction takes into account the phonological variability between speakers, suggesting that it is possible to pre-activate in a detailed and specific way the phonological representation of a predictable word.

Animations and the Role of Emotions

How can an animator capture and analyse emotion? Might the act of animating something itself be a way to access meaning that earlier studies were unable to do? Animation has the ability to both accentuate and hide emotions that are displayed through body language and gesture. We are exposed to both the spoken words and the subtle variations in body motions when we watch live action (human interview) documentary film. What may be gained by interpreting documentary video via the animator's personal and aesthetic perspective, or how much might be lost when it is converted into animation? This study investigates the outcomes of the initial of a series of animations produced utilising research via practise approach, drawing on my prior expertise as a game's animator. The process of manipulating images to provide them the appearance of motion is referred to as animation. By creating a series of images, or frameworks, with one frame being a bit distinct relative to the last, a representation of movement is produced. Cartoons are among the best forms of animation. Animated videos have always been hand-drawn, using a great deal of visuals made with minor adjustments using color as well as a sharpie. A lot of contemporary animation was produced using an arsenal of specialized software programs as computer technology and animation software proliferated.

Factors associated with sleep disturbances in children and adolescents with Angelman Syndrome

BackgroundAngelman Syndrome (AS) is a rare genetic disorder characterised by hyperactivity, overexcitability, developmental delays, and lack of speech. MethodsThis study used secondary data analysis to investigate sleep disturbances in children and adolescents (n = 212) who are enrolled in the Global Angelman Syndrome Registry. Participants were divided into two groups based on the presence or absence of sleep disturbance. The cut-off score of 40 on the Sleep Disturbance Scale for Children was used to indicate the presence or absence of sleep disturbances. Sleep disturbances and their association with co-occurring conditions were examined regarding challenging behaviour, language and communication, infancy history, gastrointestinal symptoms, and epilepsy. Multiple regression was then conducted to investigate possible predictors for sleep disturbances. ResultsChildren and adolescents with AS, with and without sleep disturbances, differed considerably regarding anxiety. Sleep disturbances were significantly associated with an ability to use spoken words and computerised communication devices, and anxiety was a predictor of sleep disturbances. ConclusionFuture research is necessary to replicate this novel research, and to advance the clinical treatment of sleep disturbances in children and adolescents with AS.

Using music to assist language learning in autistic children with minimal verbal language: The MAP feasibility RCT.

Research has shown that autistic individuals often have unusually good musical skills and that combining words and music helps autistic individuals to focus on spoken words. This study tests the idea that music will help with early language learning of preschool autistic children. The results show that when caregivers sing words to autistic children, the children pay more attention to the caregiver than when the words are spoken and that they learn word combinations more easily.

Audio-visual Stroop matching task with first- and second-language color words and color associates

AbstractIn the audio-visual Stroop matching task, participants compare one Stroop stimulus dimension (e.g., the color of a written word) to a second stimulus (e.g., a spoken word) and indicate whether these two stimuli match or mismatch. Slower responses on certain trials can be due to conflict which occurs between color representations (semantic conflict) or due to conflict between responses evoked by task comparisons (response conflict). The contribution of these conflicts has been investigated with color word distracters. This is the first study which explores how two types of first- and second-language words affect audio-visual matching. Native French speakers performed a bilingual Stroop matching task with intermixed French (L1) and English (L2) color words (Experiment 1) and color associates (Experiment 2) presented in congruent and incongruent colors simultaneously with spoken French color words. Participants were instructed to indicate whether the spoken word “matches” or “mismatches” the font color, while ignoring written word meaning. Interestingly, the results were similar for the critical “mismatch” trials for both French and English words. The responses were the fastest on trials in which task comparisons activate fewer response alternatives, supporting the assumption of the response conflict account.

An exploratory investigation of acoustic features underlying arousal and valence perception of vocalizations from non-speaking individuals

Emotion perception of vocalizations, especially from individuals with no or few spoken words, remains an underexplored topic in acoustical research. The aim of this exploratory study was to identify acoustic features within non-speech vocalizations correlated with perceived arousal and valence. 364 vocalizations were selected from the open-access ReCANVo dataset, comprising non-speech communicative sounds by non-speaking individuals with autism and neurodevelopmental disorders. 108 listeners independently rated each vocalization for arousal and valence on a 5-point Likert scale (78624 total ratings). We then used random forest and elastic net regression techniques to determine correlations between acoustic-based features and perceived affect. Using 4 published (e.g., GeMAPs) and 2 custom feature sets (dim 12, 28) encompassing duration, intensity, pitch, formant, and spectral centroid features, we evaluated each model using R-squared and mean-squared error. Our 12-feature custom elastic net model achieved the highest performance for predicting arousal (R-sq = 0.763, MSE = 0.075), with intensity features weighted highest, comparable to other sets. Our 28-feature random forest model, which included vocal quality features, significantly outperformed other models in valence prediction (R- sq = 0.491; MSE = 0.139, p < 0.001). While further validation on a held- out test set is essential, these exploratory results expand our understanding of affective acoustic features of non-speech sounds from individuals with neurodevelopmental disorders.