Smart Speakers Research Articles

Afitness: Fitness Monitoring on Smart Devices via Acoustic Motion Images

Recently, as fitness has become a popular part of people’s lives, the intention to record fitness processes and assess the standards of fitness movements has grown increasingly keen. However, the existing approaches have some limitations, for example, wearable devices can hinder users’ fitness activities; computer vision–based solutions pose the risk of privacy breach, and so on. Fortunately, we observed that smartspeaker, acoustic-based sensing is a promising method of activity monitoring. In this article, we propose Afitness, an acoustic-based sensing system that enables non-intrusive, passive, and high-precision fitness detection. Afitness has the following three innovations. (i) We utilize pulse compression to generate high-precision motion distance images on commercial devices that can be visually recognized. (ii) We propose a data augmentation algorithm, which also incorporates transfer learning to greatly reduce the pressure of data collection. (iii) We exploit incremental learning techniques that allow Afitness to improve the portability of our system and recognize new actions. Overall, Afitness achieves acoustic signal interpretability and environmental reliability detection.

An Audio - Visual Virtual Personal Assistant

Current traits in smart assistants and smart home automation are currently attracting the interests of customers and researchers. Speech enabled smart virtual assistants (named smart speakers) offer a wide sort of network orientated services and, in some instances, can connect to smart environments, accordingly improving them with new and effective user interfaces. But, such gadgets also reveal new desires and a few weaknesses. Specially, they constitute faceless and blind assistants, not able to reveal a face, and therefore an emotion, and not able to ‘see’ the person. For this reason, the interaction is impaired and, in a few cases, ineffective. One of the goals of artificial intelligence is the realization of natural talk among humans and machines. In latest years, the dialogue systems, known as interactive conversational systems are the fastest developing vicinity in AI. Many companies have used the dialogue systems technology to set up various sorts of VPAs based on their application areas, including Apple’s Siri, Amazon Alexa, etc. To triumph over such troubles, in this project we combine a number of advance techniques. The proposed Assistant is powerful and resource-efficient, interactive and customizable. We use the multi-modal dialogue structures and screen projection which process two or more combined user input modes, which includes speech, image, video, touch, manual gestures, body movements with the intention to design the NextGeneration of VPAs. The new version of VPAs can be used to increase the interaction between humans and machines by the usage of distinctive technologies, consisting of gesture recognition, image/video recognition, speech recognition, dialogue system, conversational information base, and the overall knowledge base. Furthermore, this VPA device can be utilized in different areas such as education, medical assistance, disabilities systems, home automation and security access control.

Usability Analysis of Smart Speakers from a Learnability Perspective for Novel Users

Although commercial smart speakers are becoming increasingly popular, there is still much potential for investigation into their usability. In this study, we analyzed the usability of commercial smart speakers by focusing on the learnability of young users who are not yet familiar with voice user interface (VUI) operation. In the experiment, we conducted a task in which users repeatedly operated a smart speaker 10 times under four conditions, combining two experimental factors: the presence or absence of a screen on the smart speaker and the operation method (voice control only or in conjunction with remote-control operation). The usability of the smart speaker was analyzed in terms of task-completion time, task-completion rate, number of errors, subjective evaluation, and retrospective protocol analysis. In particular, we confirmed and compared the learning curves for each condition in terms of the performance metrics. The experimental results showed that there were no substantial differences in the learning curves between the presence and absence of a screen. In addition, the “lack of feedback” and “system response error” were identified as usability problems, and it was suggested that these problems led to “distrust of the system”.

"Like another human being in the room": a community case study of smart speakers to reduce loneliness in the oldest-old.

This community case study examined the potential benefits of smart speakers to tackle loneliness in the oldest old adults living in supported accommodation. The program was established as a collaboration between the supported accommodation provider and a technology company to explore the feasibility of smart speakers to alleviate resident loneliness. Loneliness in later life often accompanies a shrinking social circle, loss of a spouse or increased disability. People aged 85 years of age and over are increasingly likely to experience these life events, leading to an increased risk of social isolation and loneliness. Five older people, mean age 90 years of age, who resided in supported accommodation, were given a smart speaker for 8 weeks to examine their experience with the voice assistant. The experiences of the five older adults are explored as case studies, with each person interviewed both before and after receiving the smart speaker. All five valued their smart speaker, recognised its potential for tackling loneliness, and wanted to keep it. The three most lonely individuals reported that their smart speaker made them feel less lonely and isolated through two mechanisms: (i) creating a presence and (ii) having some control over their situation. Although only a small study, these experiences suggest providing smart speakers for lonely and isolated oldest-old people, could be one way to help combat loneliness in community settings.

Room-scale Location Trace Tracking via Continuous Acoustic Waves

The increasing prevalence of smart devices spurs the development of emerging indoor localization technologies for supporting diverse personalized applications at home. Given marked drawbacks of popular chirp signal-based approaches, we aim at developing a novel device-free localization system via the continuous wave of the inaudible frequency. To achieve this goal, solutions are developed for fine-grained analyses, able to precisely locate moving human traces in the room-scale environment. In particular, a smart speaker is controlled to emit continuous waves at inaudible 20kHz , with a co-located microphone array to record their Doppler reflections for localization. We first develop solutions to remove potential noises and then propose a novel idea by slicing signals into a set of narrowband signals, each of which is likely to include at most one body segment’s reflection. Different from previous studies, which take original signals themselves as the baseband, our solutions employ the Doppler frequency of a narrowband signal to estimate the velocity first and apply it to get the accurate baseband frequency, which permits a precise phase measurement after I-Q (i.e., in-phase and quadrature) decomposition. A signal model is then developed, able to formulate the phase with body segment’s velocity, range, and angle. We next develop novel solutions to estimate the motion state in each narrowband signal, cluster the motion states for different body segments corresponding to the same person, and locate the moving traces while mitigating multi-path effects. Our system is implemented with commodity devices in room environments for performance evaluation. The experimental results exhibit that our system can conduct effective localization for up to three persons in a room, with the average errors of 7.49 cm for a single person, with 24.06 cm for two persons, with 51.15 cm for three persons.

Can I Trust What You Say about the Smart Speaker? Effects of AI Anxiety and Biased Assimilation on Blog Comment Credibility

ABSTRACT Advancements in Internet and communication technologies have facilitated the proliferation of product reviews and commentary pertaining to smart speakers. This study investigated the impacts of uncivil comments on blog comment credibility. Results show that the civil opinion commentary appears to be more credible than uncivil commentary. The interaction between civil opinion and low AI anxiety generates more blog review credibility than the interaction between uncivil opinion and low AI anxiety. Finally, we identify moderating effects of AI anxiety and provide deeper insights into how the interaction of comment civility and AI anxiety shape the credibility assessments of blog reviews.

Evaluation of a Smart Audio System Based on the ViP Principle and the Analytic Hierarchy Process Human–Computer Interaction Design

The current limitations of user–product interaction with smart speakers have spurred the proposal of a model to circumvent these challenges. We used the ViP design principle to redefine the user’s approach to interacting with the product. Throughout the deconstruction and design stages, we explored the structure and function of the conventional product across three layers: the product layer, interaction layer, and context layer using three models. We used the hierarchical analysis method to effectively quantify the design factors affecting user experience and identify the key design factors. This approach enabled us to contextualize the smart audio system, explore the interaction dynamics between the product and the user, and provide valuable insights on designing new products. A questionnaire method was used to survey 67 users, and a reliability test was conducted to ensure the validity of the questionnaire v (Cronbach’s coefficient α = 0.868). A pairwise comparison of factors was conducted on a 1–9 scale, with weights determined through the analytic hierarchy process (AHP). The combination of the ViP design principle and hierarchical analysis presents a novel and objective paradigm to guide designers to customize product characteristics (design attributes) to enhance user human–computer interaction experience. We validated the feasibility of the innovative design approach using the smart speaker model, offering insights for research on designing similar products.

Interactive virtual assistance for mental health promotion and self-care management in elderly with type 2 diabetes (IVAM-ED): study protocol and statistical analysis plan for a randomized controlled trial

BackgroundWith one in five individuals aged 65 or older living with type 2 diabetes worldwide, it is crucial to acknowledge and address the challenges faced by this population. In this context, our study aims to evaluate the efficacy of a behavioral intervention model delivered through a smart speaker on mental health and diabetes self-care in the elderly with diabetes.MethodsThis is a single-center, pragmatic, parallel two-arm open randomized clinical trial involving elderly patients with type 2 diabetes. We plan to enroll a total of 112 individuals who will be randomized 1:1 to receive the Smart Speaker EchoDot 3rd Gen device (Amazon Echo®) for home use (intervention arm) or to maintain usual care (control arm). The primary outcome is mental distress, assessed using the 20-item Self Reporting Questionnaire (SRQ-20) after a 12-week intervention period. Secondary outcomes include quality of life, adherence to diabetes self-care behaviors, perception of stress, glycemic control, blood pressure, and lipid profile. Analysis of covariance (ANCOVA) will be used to evaluate the effects of the intervention on the outcomes.DiscussionThis study assesses the effectiveness of an interactive virtual assistance system for enhancing mental health and glycemic control among elderly individuals with type 2 diabetes. The findings may introduce smart speakers as a valuable tool for promoting diabetes-related self-care in this population.Trial registrationClinicalTrials.gov NCT05329376. Registered on 15 April 2022. Enrollment began on 20 June 2023 and the last update of protocol was on 13 December 2023.

When the recipe is more important than the ingredients: Unveiling the complexity of consumer use of voice assistants

AbstractThe widespread use of voice assistants (VAs) creates a pressing need to understand what drives consumers to use different VAs. Existing studies have commonly focused on the net effects of antecedents that explain why consumers adopt or continue using VAs, ignoring the complexity of consumer behavior and the combinatorial effects of multiple antecedents. Our study proposes that consumer intention to continue using VAs does not depend on a single characteristic of products or consumers but on specific configurations of such characteristics. By integrating human–technology interaction and media richness theories, we suggest that consumers with distinct psychometric profiles and learning styles may evaluate humanlike and technological attributes of VAs differently. Our study shows that the complex interconnectedness between different VA attributes and consumer characteristics can provide a holistic understanding of why some consumers continue or stop using VAs. The results advance the media richness literature by offering novel insights into multimodality in consumer–technology interactions by examining consumer evaluations of single and multimodal VAs (e.g., smart speakers vs. touchscreen smart speakers). Our study provides templates for managers to effectively design VAs aligned with their segmentation and targeting strategies.

Laser-Based Command Injection Attacks on Voice-Controlled Microphone Arrays

Voice-controlled (VC) systems, such as mobile phones and smart speakers, enable users to operate smart devices through voice commands. Previous works (e.g., LightCommands) show that attackers can trigger VC systems to respond to various audio commands by injecting light signals. However, LightCommands only discusses attacks on devices with a single microphone, while new devices typically use microphone arrays with sensor fusion technology for better capturing sound from different distances. By replicating LightCommands’s experiments on the new devices, we find that simply extending the light scope (just as they do) to overlap multiple microphone apertures is inadequate to wake up the device with sensor fusion. Adapting LightCommands’s approach to microphone arrays is challenging due to their requirement for multiple sound amplifiers, and each amplifier requires an independent power driver with unique settings. The number of additional devices increases with the microphone aperture count, significantly increasing the complexity of implementing and deploying the attack equipment. With a growing number of devices adopting sensor fusion to distinguish the sound location, it is essential to propose new approaches to adapting the light injection attacks to these new devices. To address these problems, we propose a lightweight microphone array laser injection solution called LCMA (Laser Commands for Microphone Array), which can use a single laser controller to manipulate multiple laser points and simultaneously target all the apertures of a microphone array and input light waves at different frequencies. Our key design is to propose a new PWM (Pulse Width Modulation) based control signal algorithm that can be implemented on a single MCU and directly control multiple lasers via different PWM output channels. Moreover, LCMA can be remotely configured via BLE (Bluetooth Low Energy). These features allow our solution to be deployed on a drone to covertly attack the targets hidden inside the building. Using LCMA, we successfully attack 29 devices. The experiment results show that LCMA is robust on the newest devices such as the iPhone 15, and the control panel of the Tesla Model Y.

Embracing Distributed Acoustic Sensing in Car Cabin for Children Presence Detection

Contactless acoustic sensing has been actively exploited in the past few years to enable a large range of applications, ranging from fine-grained vital sign monitoring to coarse-grained human tracking. However, existing acoustic sensing systems mainly work on smartphone or smart speaker platforms. In this paper, we envision an exciting new acoustic sensing platform, i.e., car cabin which is inherently embedded with a large number of speakers and microphones. We propose the new concept of distributed acoustic sensing and develop novel designs leveraging the unique characteristics of rich multi-path in car cabin to enable fine-grained sensing even when the primary reflection is totally blocked. By using child presence detection as the application example, we show that child presence can be detected through body motions or even subtle breath (when the child is sleeping or in coma) at all locations in the cabin without any blind spots. We further show that the proposed system can robustly work in different car cabins, achieving an average detection accuracy of 97% and a false alarm rate always below 2% under different scenarios including those challenging ones such as rear-facing seat blockage. We believe the proposed distributed sensing modality in car cabin pushes acoustic sensing one big step towards real-life adoption.

Who are the (Non-)Adopters of Smart Speakers? A Cross-Sectional Survey Study of Dutch Families

Who are the (Non-)Adopters of Smart Speakers? A Cross-Sectional Survey Study of Dutch Families

Research on the Integrated Development Strategy of Future News Industry in the Smart Speaker Scene

Research on the Integrated Development Strategy of Future News Industry in the Smart Speaker Scene

Evaluating smart speakers as assistants for informal caregivers of people with dementia

Evaluating smart speakers as assistants for informal caregivers of people with dementia

Investigating the Integration and the Long-Term Use of Smart Speakers in Older Adults' Daily Practices: Qualitative Study.

As smart speakers become more popular, there have been an increasing number of studies on how they may benefit older adults or how older adults perceive them. Despite the increasing ownership rates of smart speakers among older adults, studies that examine their integration and the long-term use in older adults' daily practices are scarce. This study aims to uncover the integration of smart speakers into the daily practices of older adults over the long term, contributing to an in-depth understanding of maintained technology use among this demographic. To achieve these objectives, the study interviewed 20 older adults who had been using smart speakers for over 6 months. These semistructured interviews enabled participants to share their insights and experiences regarding the maintained use of smart speakers in the long term. We identified 4 dimensions of the long-term use of smart speakers among older adults, including functional integration, spatial integration, cognitive integration, and semantic integration. For the functional integration of smart speakers, the study reported different types of use, including entertainment, information collection, medication reminders, companionship, environment modification, and emergency calls. For the spatial integration of smart speakers, the study showed older adults' agency in defining, changing, and reshaping daily practices through the spatial organization of smart speakers. For the cognitive integration of smart speakers, the findings showed the cognitive processes involved in adapting to and incorporating smart speakers into daily habits and routines. For the semantic integration of smart speakers, the findings revealed that older adults' enjoyable user experience and strong bonds with the device contributed to their acceptance of occasional functional errors. Finally, the study proposed several suggestions for designers and developers to better design smart speakers that promote maintainable use behaviors among older adults. On the basis of the findings, this study highlighted the importance of understanding how older adults use smart speakers and the practices through which they integrate them into their daily routines. The findings suggest that smart speakers can provide significant benefits for older adults, including increased convenience and improved quality of life. However, to promote maintainable use behaviors, designers and developers should consider more about the technology use contexts and the specific needs and preferences of older adults when designing these devices.

White Elephant or Happiness Goodies? The Effect of User Personality on the Perception of Digital Companionship of Smart Speaker

White Elephant or Happiness Goodies? The Effect of User Personality on the Perception of Digital Companionship of Smart Speaker

Analisis Audio Capture untuk Pengumpulan Data pada Smart Speaker

Internet of Things or IoT is a technology that is currently trending, because it can help human in their daily routine. One of the IoT product that used in homes is smart speaker. However, every latest technological innovation does not escape from vulnerabilities and one of them is the microphone on the smart speaker. Several studies and research have found that security vulnerabilities in smart speakers, such as dolphin attacks and audio capture. To find out how this audio capture technique works, an experiment was made to understand how it works and the impact of this technique. This experiment uses a smart speaker with voice assistant Alexa, using the Alexa Skills Kit services and Flask-Ask framework to create an audio capture program. The results of this program testing are expected to be used as a benchmark to prevent smart speakers from becoming the target of this technique anymore.

Remote Multi-Person Heart Rate Monitoring with Smart Speakers: Overcoming Separation Constraint.

Heart rate is a key vital sign that can be used to understand an individual's health condition. Recently, remote sensing techniques, especially acoustic-based sensing, have received increasing attention for their ability to non-invasively detect heart rate via commercial mobile devices such as smartphones and smart speakers. However, due to signal interference, existing methods have primarily focused on monitoring a single user and required a large separation between them when monitoring multiple people. These limitations hinder many common use cases such as couples sharing the same bed or two or more people located in close proximity. In this paper, we present an approach that can minimize interference and thereby enable simultaneous heart rate monitoring of multiple individuals in close proximity using a commonly available smart speaker prototype. Our user study, conducted under various real-life scenarios, demonstrates the system's accuracy in sensing two users' heart rates when they are seated next to each other with a median error of 0.66 beats per minute (bpm). Moreover, the system can successfully monitor up to four people in close proximity.

A New mmWave-Speech Multimodal Speech System for Voice User Interface

Voice user interface (VUI) plays an essential role in intelligent scenes, e.g., smart homes. It provides a hands- and eyes-free human-machine interaction between humans and Internet of Things devices. Benefiting from the development of deep learning and natural language process, the automatic speech recognition (ASR) entitles VUI to the capacity of accurate comprehension on users' intentions. With such a convenient and flexible service, users can interact with various devices as they please. Commercial VUI products have gained in popularity over recent years, such as smart speakers (e.g., Amazon Echo and Google Home), voice assistants in smartphones (e.g., Siri), and in-vehicle voice control interactions (e.g., VUIs in Tesla Model S/X/3/Y). Analysts forecast that by 2024, the deployment of VUI-based smart speakers will reach 640 million globally.

The use of technology to address loneliness and social isolation among older adults: the role of social care providers

BackgroundAddressing loneliness and social isolation among older adults remains a public health priority. The restrictions enforced during the COVID-19 pandemic simultaneously heightened the need for services to overcome social isolation and reduce loneliness among older adults, while also limiting social care providers’ ability to deliver these. The aim of this study was to explore the experiences of social care providers in using technology to address loneliness and social isolation among older adults during the pandemic.MethodsThis was a mixed methods study involving an online survey and interviews with providers of older adult social care in Wales, UK. Invitations to participate were sent to commissioners and providers of adult social care services, including those working in the voluntary and community sectors, across all local authorities in Wales. Data was collected between September 2021 and January 2022.ResultsSixty-one service providers completed the survey, 19 of whom also took part in an interview. Addressing loneliness and isolation among older adults was reported as a key concern by nearly all survey respondents. While telephone calls were the most common means of facilitating social interaction, many service providers also tried to support older adults to make more use of devices that they already had (e.g., smartphones to hold video calls). Where funding was available, organisations purchased devices, such as tablets and smart speakers, for older adults. Analysis of interviews resulted in three themes: (1) The potential and limitations of technology; (2) Individuals’ capabilities, confidence, motivations and values; and (3) The wider system.ConclusionsTechnology was employed in a variety of ways during the pandemic to address loneliness and social isolation among older adults; many service providers continue to use technology in a hybrid system of care now that pandemic-related social restrictions have been lifted. Our findings emphasise a need for technology-assisted interventions to be designed and deployed in alignment with service users’ values, motivations and capabilities. Further, there is a need to better support service providers to assess loneliness and social isolation among older adults, and to acknowledge the important role providers play in helping older adults to adopt technology-assisted interventions.