Voice Activity Research Articles

End-to-end elevator voice command recognition system based on improved convolutional neural network

Abstract As a common life scenario, the use of voice control in elevator operation can effectively reduce public health risks and improve user experience in the post-epidemic era. However, due to the characteristics of its closure and public space, it is difficult to deploy the elevator. To solve this problem, an end-to-end speech command recognition algorithm based on a new deep neural network is proposed. The algorithm uses the self-built corpus for training, and uses the speech segment interception algorithm to obtain audio segments under the premise of speech stream as input, puts them into the network model in real time for reasoning, and drives the elevator to run according to the output results. The results show that compared with the recognition model using MFCC-CNN, the network has achieved about 10% improvement in accuracy under the premise of smaller computation, and compared with the DTW algorithm, the network has improved the accuracy by about 25%. Finally, a typical deployment environment is constructed to prove the correctness and effectiveness of the method in practical application.

‘It's not the boogeyman’: How voice assistant technology is bridging the digital divide for older people

Technologically illiterate users are frequently excluded from access to technology and its benefits, creating a digital divide. Some groups of older adults are among those considered technologically illiterate and experiencing vulnerability. For this population, traditional technologies that require physical interactions have proven to be stressful and create a phenomenon known as ‘technophobia’. Herein lies the potential value of touchless technologies such as voice assistants, which may offer a new avenue for older adult inclusion. Thus, our study delves deep into the lifeworld of older people to understand how voice assistants can impact inclusion and well-being. Adopting a longitudinal approach, we observed and interviewed participants (n = 32) in their homes, augmenting this qualitative data with secondary data from voice assistant device reports. Our initial results show that older people perceived the effect of illiteracy in their first encounter with the technology, but over time, they learnt to overcome initial challenges and started building relationships with artificial intelligence. In the final part of the study, participants identified the value of using technology and recognised that technology ‘is not the boogeyman’ but rather enables tech-inclusion and well-being. Under the lens of transformative service research, we provide a framework that illustrates how artificial intelligence is healing older people's digital divide, showing core elements of inclusion (access to new technology, support with activities, sense of participation, independence, convenience and source of information), and well-being (sense of pride, entertainment option, eased loneliness, improved self-esteem).

Nonbinary Voices for Digital Assistants—An Investigation of User Perceptions and Gender Stereotypes

Due to the wide adoption of digital voice assistants (DVAs), interactions with technology have also changed our perceptions, highlighting and reinforcing (mostly) negative gender stereotypes. Regarding the ongoing advancements in the field of human–machine interaction, a developed and improved understanding of and awareness of the reciprocity of gender and DVA technology use is thus crucial. Our work in this field expands prior research by including a nonbinary voice option as a means to eschew gender stereotypes. We used a between-subject quasi-experimental questionnaire study (female voice vs. male voice vs. nonbinary voice), in which n=318 participants provided feedback on gender stereotypes connected to voice perceptions and personality traits. Our findings show that the overall gender perception of our nonbinary voice leaned towards male on the gender spectrum, whereas the female-gendered and male-gendered voices were clearly identified as such. Furthermore, we found that feminine attributes were clearly tied to our female-gendered voice, whereas the connection of masculine attributes to the male voice was less pronounced. Most notably, however, we did not find gender-stereotypical trait attributions with our nonbinary voice. Results also show that the likability of our female-gendered and nonbinary voices was lower than it was with our male-gendered voice, and that, particularly with the nonbinary voice, this likability was affected by people’s personality traits. Thus, overall, our findings contribute (1) additional theoretical grounding for gender-studies in human–machine interaction, and (2) insights concerning peoples’ perceptions of nonbinary voices, providing additional guidance for researchers, technology designers, and DVA providers.

ATC-SD Net: Radiotelephone Communications Speaker Diarization Network

This study addresses the challenges that high-noise environments and complex multi-speaker scenarios present in civil aviation radio communications. A novel radiotelephone communications speaker diffraction network is developed specifically for these circumstances. To improve the precision of the speaker diarization network, three core modules are designed: voice activity detection (VAD), end-to-end speaker separation for air–ground communication (EESS), and probabilistic knowledge-based text clustering (PKTC). First, the VAD module uses attention mechanisms to separate silence from irrelevant noise, resulting in pure dialogue commands. Subsequently, the EESS module distinguishes between controllers and pilots by levying voice print differences, resulting in effective speaker segmentation. Finally, the PKTC module addresses the issue of pilot voice print ambiguity using text clustering, introducing a novel flight prior knowledge-based text-related clustering model. To achieve robust speaker diarization in multi-pilot scenarios, this model uses prior knowledge-based graph construction, radar data-based graph correction, and probabilistic optimization. This study also includes the development of the specialized ATCSPEECH dataset, which demonstrates significant performance improvements over both the AMI and ATCO2 PROJECT datasets.

Believe It or Not? Investigating the Credibility of Voice Assistants in the Context of Social Roles and Relationship Types

Recent advancements in artificial intelligence have empowered voice assistants to comprehend user input and provide contextually relevant responses more effectively. This paper aims to explore people’s relationship with voice assistants, examining factors like the sense of power, control, closeness, and parasocial relationships, and how relationship types relate to the credibility of voice assistants. To investigate this, we built on the foundation of interpersonal relationships and applied parasocial relationship theory as a framework for understanding the human–agent relationship. We conducted an online survey (N = 393). Our results show that people mainly perceive their relationship with voice assistants as either a servant–master relationship, student–teacher relationship, or equal partnership. This study highlights that voice assistants’ mimicry of human characteristics can make them more than tools—they can also be perceived as servants, teachers, or equal partners. Notably, a significant association exists between relationship type and credibility judgments towards voice assistants.

An embedded end-to-end voice assistant

Voice assistants are spreading in various environments, such as houses and cars, bringing the possibility of controlling heterogeneous Internet of Things devices with simple voice commands. However, massive use of the cloud connection for speech processing requires an efficient and robust Internet connection and raises concerns in terms of privacy. Therefore, we propose an end-to-end solution able to work totally offline, based on a system architecture combining different Deep Learning models to implement all the steps of the speech elaboration process. Being interested in targeting the Italian language, we exploited the transfer learning paradigm, which allows leveraging models trained in English on large datasets and fine-tuning them to the target language on a smaller dataset. The proposed system architecture is configurable and easily extensible to other languages. Experimental results in an automotive application use case show that our solution outperforms the other embedded models and achieves performance comparable to state-of-the-art cloud-connected solutions for Automatic Speech Recognition. Moreover, overall latency is significantly reduced by eliminating the need to connect to the cloud.

Analysis of artificial intelligence models for the smart home industry

Since ChatGPT's release, large AI models have gained significant attention, particularly in the integration with Smart Home 3.0, marking a new research direction. This paper reviews cutting-edge research on the intersection of smart homes and large AI models, highlighting challenges and trends. We focus on data, models, and execution to explore the advancement of smart home platforms globally. We discuss data collection and feature research progress within smart homes, using intelligent voice assistants like Amazon Alexa and Google Bard as examples. We examine the applications, potentials, and challenges of AI models in smart homes and offer insights into future applications of large AI models in this field.

Performance Optimization of Voice-Assisted File Management Systems

In this paper, we present a novel approach for managing the file system in Linux using a voice assistant. Our system allows users to perform file system operations such as creating directories, renaming files, and deleting files by issuing voice commands. We develop a voice assistant using Python libraries and integrate it with the file system in Linux. The voice assistant is capable of understanding natural language and executing commands based on the user’s voice inputs. We conduct experiments to evaluate the performance of the system and demonstrate that our approach is effective and efficient in managing the file system using voice commands. Our system can enhance the accessibility and usability of the file system in Linux for individuals with disabilities or those who prefer a hands-free approach to file management.

Comparing and modeling the use of online recommender systems

This study explores a new way to model the adoption of AI, specifically online recommender systems. It aims to find factors that can explain the variation in usage in terms of differences between individuals and differences over technologies. We analyzed survey data from users of online platforms in the U.S. using a two-level structural equation model (SEM) (N = 1007). In this model, the dependent variable was the usage rate, which was defined as the share of time a person used a particular recommender system (e.g., “People You May Know”) when they use the platform (e.g., Facebook). The individual responses (within-systems level) were clustered in the 26 recommender systems (between-systems level). We hypothesized that three technology-specific factors, adapted from the Diffusion of Innovations (DOI) theory and the Unified Theory of Acceptance and Use of Technology 2 (UTAUT2), could explain the variations in usage at both levels: perceived performance expectancy (PE), perceived effort expectancy (EE), and perceived hedonic motivation (HM). Our estimated model showed that usage was associated with PE and HM at the within-system level and only with PE at the between-system level. A considerable part of the variation in usage across the 26 systems could be explained by PE only (R2 = 0.30). The most important contribution to practitioners is that this study provides evidence for the idea that there are inherent, measurable differences across recommender technologies that affect their usage rates, and specifically it finds usefulness to be a key factor. This is potentially valuable for app developers and marketeers who look to promote the adoption of novel recommender systems. The main contribution to the literature is that it presents a proof-of-concept of a two-level model for AI adoption, conceptualizing it as an effect of both variations over users and variations over applications. This finding is potentially valuable for policymakers, as better predictive models might enable improved assessments of AI's social implications. In future studies, the two-level approach presented here could be applied to other forms of AI, such as voice assistants, chatbots, or Internet of Things (IoT).

Uncovering post-adoption usage of AI-based voice assistants: a technology affordance lens using a mixed-methods approach

ABSTRACT Despite the growing proliferation of AI-based voice assistants in our daily lives, how different functions of AI-based voice assistants affect users’ post-adoption usage behaviours remains an under-investigated research question. This study explores the nature and causation of post-adoption usage behaviours (effective utilisation after initial adoption or diffusion) within the context of AI-based voice assistants. Using a sequential mixed-methods research design, we first identify the post-adoption usage behaviours of AI-based voice assistants as a multidimensional concept comprised of routine use and extended use, then develop a contextualised model by revealing technology-specific antecedents, cognitive beliefs, and boundary conditions. By integrating results from the quantitative study and qualitative study, we find that three technology affordances (i.e., anthropomorphism affordance, interactivity affordance, and personalisation affordance) are salient antecedents of two cognitive beliefs, which further affect users’ routine use and extended use of AI-based voice assistants. Additionally, we uncover use frequency as a boundary condition and obtain a complementary view of post-adoption usage of AI-based voice assistants. The empirical research findings can extend the post-adoption IS usage literature and provide practical implications for crafting user-centred functionalities to facilitate effective human-AI interactions.

Towards interculturally adaptive conversational AI

Abstract Among the many ways that AI technologies are becoming embedded in our social worlds is the proliferation of Conversational User Interfaces, such as voice assistants (e.g. Apple Siri and Amazon Alexa), chatbots and voice-based conversational agents. Such conversational AI technologies are designed to draw upon the designers’ understanding of interactional practices employed in human–human conversation, and therefore have implications for intercultural communication (ICC). In this paper, we highlight some of the current shortcomings of conversational AI, and how these relate to ICC. We also draw on findings from Conversation Analysis to discuss how pragmatic norms vary across linguacultural groups (see Risager, Karen. 2019. Linguaculture. In Carol A. Chapelle (ed.). Encyclopedia of applied linguistics. Wiley-Blackwell for a discussion of the term ‘linguaculture’), noting that this poses further challenges for designers of conversational AI systems. We argue that the solution is to work towards what we call interculturally adaptive conversational AI. Finally, we propose a framework for how this can be conceptualised and researched, and argue that researchers with expertise in language and ICC are uniquely placed to contribute to this endeavour.

The impact of self-conscious emotions on the continuance intention of digital voice assistants in private and public contexts

Digital voice assistants (DVAs) are application programs that can understand and interpret natural language voice commands. They are embedded in various products like smartphones and smart speakers and have thus become integral in everyday life, interpersonal communications, and social relationships. Despite users tend to humanize DVAs, emotions that arise in interpersonal interactions such as self-conscious emotions (i.e., pride, shame, guilt, and vicarious embarrassment) and their influence on human-machine interaction remain unstudied. Grounding on regulatory focus theory, we argue that these emotions are fundamental for promoting or preventing future DVA use. Additionally, drawing on social influence theory, we contend that the influence of self-conscious emotions on continuous DVA use varies across specific usage situations. Thus, we extend the expectation-confirmation model with self-conscious emotions and empirically compare user reactions between different social scenarios (alone and with friends in private vs. public places). Analyzing 860 DVA user responses through structural equation modeling and multigroup analysis, our findings reveal that pride consistently positively influences continuance intention across all social contexts. Furthermore, shame acts as an important inhibitor of continuance intentions in public, while guilt inhibits continuance intentions in private places. Vicarious embarrassment, however, does not exhibit significant effects in any scenario. These results carry valuable implications for both research and management in understanding and optimizing DVA user experiences.

Voice Assistants for Mental Health Services: Designing Dialogues with Homebound Older Adults.

The number of older adults who are homebound with depressive symptoms is increasing. Due to their homebound status, they have limited access to trained mental healthcare support, which leaves this support often to untrained family caregivers. To increase access, a growing interest is placed on using technology-mediated solutions, such as voice-assisted intelligent personal assistants (VIPAs), to deliver mental health services to older adults. To better understand how older adults and family caregivers intend to interact with a VIPA for mental health interventions, we conducted a participatory design study during which 6 older adults and 7 caregivers designed VIPA-human dialogues for various scenarios. Using conversation style preferences as a starting point, we present aspects of human-likeness older adults and family caregivers perceived as helpful or uncanny, specifically in the context of the delivery of mental health interventions, which helps inform potential roles VIPAs can play in mental healthcare for older adults.

PP01.128 Quality of Smoking Cessation Counseling Provided by Artificial Intelligence Voice Assistants

PP01.128 Quality of Smoking Cessation Counseling Provided by Artificial Intelligence Voice Assistants

Artificial Intelligence (AI) Vs Academic Integrity (AI) in Law and Society

Artificial Intelligence (AI) having its ever-evolving nature and far-reaching implications in law and society has become too great to be ignored by individuals, educators, businesses, and regulators globally. AI systems with specific sets of rules to conduct a particular task existed in our society for many decades in the form of voice assistants like Siri or Alexa, Google's search or Google map’s algorithm, and even the recommendation engines on Netflix or Amazon have been through the use of AI all these years. So why is this sudden need for thorough examination of this technology having remarkable capabilities and its unprecedented adoption facing the controversy of harming contemporary education’s integrity? For Bangladesh, to a large extent, it has been the rise of tech startups accelerating AI penetration where the abundance of collecting and mining educational data of the learners came with ethical implications. A survey conducted for this paper identified the impact of AI on college and university teachers which was critical for addressing the concerns about AI and its short and long-term impacts on education systems generally. This paper attempts to explore the intricate dual relationship between AI and academic integrity (ai), in law making it imperative to critically examine the unintended consequences by addressing some of the misconceptions regarding the use of AI in education, while acknowledging its potential benefits for society and educational institutions without compromising on academic integrity in law and society. By having a responsive approach to the changes brought about by AI in law & to uphold academic integrity in our society, rethinking and redeveloping the educational programs is the way to make an impact on our education and training systems which will only grow further and further in the future of Smart Bangladesh.

Beyond Binary Gender: Navigating Gender Ambiguity in Voice User Interfaces

This paper delves into the nuanced realm of gender ambiguity within Voice User Interfaces (VUIs) and explores the challenges and opportunities it presents. In a world that is increasingly recognizing gender diversity and moving beyond traditional binary constructs[1], [2], the design and implementation of VUIs must adapt to accommodate a spectrum of gender identities[3]. This article examines the current landscape of VUIs and identifies the limitations of binary gender options in voice interactions[4], [5], [6]. Furthermore, it investigates the implications of gendered language and voice modulation in VUIs, considering their impact on user experience and inclusivity[3], [7]. drawing on insights from gender studies, linguistics, and human-computer interaction[8], [9], this study proposes strategies for navigating gender ambiguity in VUI design[7]. These strategies encompass incorporating non-binary gender options, offering customizable voice settings, and employing gender-neutral language in VUI interactions[3], [7]. Additionally, the paper discusses the ethical considerations involved in designing gender-inclusive VUIs, including privacy concerns and potential biases in data collection and processing[3]. by addressing the complexities of gender identity within VUIs[10], this research aims to foster inclusivity and accessibility in voice-driven technology. It highlights the importance of embracing diversity in design practices and emphasizes the role of technology in reflecting and respecting the multifaceted nature of human identity[11], [12], [13]. Ultimately, this paper advocates for a shift towards gender-inclusive design principles in VUI development, contributing to a more equitable and empowering digital environment for all users[11], [12], [13].

Exploring the impact of gratifications (hedonic, utilitarian, symbolic and social benefits) on the behavioral intention of voice assistants’ users

Exploring the impact of gratifications (hedonic, utilitarian, symbolic and social benefits) on the behavioral intention of voice assistants’ users

Help Them Understand: Testing and Improving Voice User Interfaces

Voice-based virtual assistants are becoming increasingly popular. Such systems provide frameworks to developers for building custom apps. End-users can interact with such apps through a Voice User Interface (VUI), which allows the user to use natural language commands to perform actions. Testing such apps is not trivial: The same command can be expressed in different semantically equivalent ways. In this article, we introduce VUI-UPSET, an approach that adapts chatbot-testing approaches to VUI-testing. We conducted an empirical study to understand how VUI-UPSET compares to two state-of-the-art approaches (i.e., a chatbot testing technique and ChatGPT) in terms of (i) correctness of the generated paraphrases, and (ii) capability of revealing bugs. To this aim, we analyzed 14,898 generated paraphrases for 40 Alexa Skills. Our results show that VUI-UPSET generates more bug-revealing paraphrases than the two baselines with, however, ChatGPT being the approach generating the highest percentage of correct paraphrases. We also tried to use the generated paraphrases to improve the skills. We tried to include in the voice interaction models of the skills (i) only the bug-revealing paraphrases, (ii) all the valid paraphrases. We observed that including only bug-revealing paraphrases is sometimes not sufficient to make all the tests pass.

AraSpot: Arabic Spoken Command Spotting

Spoken keyword spotting (KWS) is the task of identifying a keyword in an audio stream and is widely used in smart devices at the edge in order to activate voice assistants and perform hands-free tasks. The task is daunting as there is a need, on the one hand, to achieve high accuracy while at the same time ensuring that such systems continue to run efficiently on low power and possibly limited computational capabilities devices. This work presents AraSpot for Arabic keyword spotting trained on 40 Arabic keywords, using different online data augmentation, and introducing ConformerGRU model architecture. Finally, we further improve the performance of the model by training a text-to-speech model for synthetic data generation. AraSpot achieved a State-of-the-Art SOTA 99.59% result outperforming previous approaches.

Evaluating Quantized Llama 2 Models for IoT Privacy Policy Language Generation

Quantized large language models are large language models (LLMs) optimized for model size while preserving their efficacy. They can be executed on consumer-grade computers without the powerful features of dedicated servers needed to execute regular (non-quantized) LLMs. Because of their ability to summarize, answer questions, and provide insights, LLMs are being used to analyze large texts/documents. One of these types of large texts/documents are Internet of Things (IoT) privacy policies, which are documents specifying how smart home gadgets, health-monitoring wearables, and personal voice assistants (among others) collect and manage consumer/user data on behalf of Internet companies providing services. Even though privacy policies are important, they are difficult to comprehend due to their length and how they are written, which makes them attractive for analysis using LLMs. This study evaluates how quantized LLMs are modeling the language of privacy policies to be potentially used to transform IoT privacy policies into simpler, more usable formats, thus aiding comprehension. While the long-term goal is to achieve this usable transformation, our work focuses on evaluating quantized LLM models used for IoT privacy policy language. Particularly, we study 4-bit, 5-bit, and 8-bit quantized versions of the large language model Meta AI version 2 (Llama 2) and the base Llama 2 model (zero-shot, without fine-tuning) under different metrics and prompts to determine how well these quantized versions model the language of IoT privacy policy documents by completing and generating privacy policy text.