Audio Interface Research Articles

EchoTap: Non-Verbal Sound Interaction with Knock and Tap Gestures

The growing demand for highly accessible interaction technologies to effectively interact with smart devices has led to the increasing popularity of voice user interfaces (VUIs). However, VUIs face interpretation challenges stemming from the variability of natural language input, such as speech clarity issues, linguistic variability, and speech impediments. As an alternative, non-verbal sound-based interaction techniques emerge as highly advantageous for smart device control, mitigating the inherent challenges of VUIs. In this article, we introduce EchoTap, a novel audio interface that harnesses the distinctive sound responses generated by knock and tap gestures on target objects. Employing deep neural networks, EchoTap recognizes both the type and location of these gestures based on their unique sound signatures. Through offline evaluation, EchoTap demonstrated competitive classification accuracy (88% on average) and localization precision (93% on average). Moreover, a user study involving 12 participants validated EchoTap’s practical effectiveness and user-friendliness in real-world scenarios. This study highlights EchoTap’s potential for various daily interaction contexts and discusses further design implications for leveraging auditory interfaces based on simple gestures.

Laser-sound reproduction by pulse amplitude modulation audio streams

Recently, the possibility to reproduce complex continuous acoustic signals via pulsed laser-plasma sound sources was demonstrated. This was achieved by optoacoustic transduction of dense laser pulse trains, modulated via single- or multi-bit Sigma–Delta, in the air or on solid targets. In this work, we extend the laser-sound concept to amplitude modulation techniques. Particularly, we demonstrate the possibility of transcoding audio streams directly into acoustic pulse streams by analog pulsed amplitude modulation. For this purpose, an electro-optic modulator is used to achieve pulse-to-pulse amplitude modulation of the laser radiation, similarly to the multi-level Sigma–Delta method. The modulator is directly driven by the analog input stream through an audio interface. The performance of the system is evaluated at a proof-of-principle level for the reproduction of test audio signals such as single tones, double tones and sine sweeps, within a limited frequency range of the audible spectrum. The results are supported by computational simulations of the reproduced acoustic signals using a linear convolution model that takes as input the audio signal and the laser-generated acoustic pulse profile. The study shows that amplitude modulation allows for significant relaxation of the laser repetition rate requirements compared to the Sigma–Delta-based implementation, albeit at the potential cost of increased distortion of the reproduced sound signal. The nature of the distortions is analyzed and a preliminary experimental and computational investigation for their suppression is presented.

Exploring Audio Interfaces for Vertical Guidance in Augmented Reality via Hand-Based Feedback.

This research proposes an evaluation of pitch-based sonification methods via user experiments in real-life scenarios, specifically vertical guidance, with the aim of standardizing the use of audio interfaces in AR in guidance tasks. Using literature on assistive technology for people who are blind or visually impaired, we aim to generalize their applicability to a broader population and for different use cases. We propose and test sonification methods for vertical guidance in a series of hand-navigation assessments with users without visual feedback. Including feedback from a visually impaired expert in digital accessibility, results (N=19) outlined that methods that do not rely on memorizing pitch had the most promising accuracy and self-reported workload performances. Ultimately, we argue for audio AR's ability to enhance user performance in different scenarios, from video games to finding objects in a pantry.

Enhancing the Quality of Dairy Products by Reducing the Pilferage and Increase the Efficiency of Logistics Performance by Using Automated Dairy System.

Background: Dairy farming is a vital sector in developing countries, where small farmers contribute approximately 80% of the dairy milk supply. These farmers often rely on traditional practices and are disconnected from technological advancements. This gap highlights the necessity for technology integration to meet the increasing global demand for dairy products. Objective: This research paper aims to prevent water adulteration in milk and enhance the monitoring of the milk trade through an Automated Dairy System, ensuring the provision of pure milk to the public. Methods: Our Automated Dairy System incorporates several components and interfaces, including a Sensor Interface (I/O), Internet Connection Interface, Bluetooth Interface, and Audio Interface. The system enables dairy farmers to check milk quality for impurities before dispatching to vendors and upon arrival. Additionally, it tracks the transportation of milk containers from farms using GPS technology. Results: Initial tests on the prototype demonstrate the system's effectiveness in detecting changes in milk volume, indicating potential adulteration. The GPS tracking feature successfully monitors the transport process, ensuring the milk's purity during transit. Conclusion: The implementation of this Automated Dairy System in dairy farming, especially in Pakistan, represents a significant stride towards improving milk quality and quantity. This innovation not only meets the current technological gap in traditional farming practices but also aligns with the growing global demand for high-quality dairy products. Our system is expected to have a substantial impact on the dairy industry by ensuring the delivery of pure milk to consumers and enhancing the overall efficiency of milk production and trade.

Acoustic Probe for Temperature Measurement Suitable for Audio Interfaces Having Random Input/Output Delays

Acoustic Probe for Temperature Measurement Suitable for Audio Interfaces Having Random Input/Output Delays

Acoustic probe for temperature measurement suitable for operation with audio interfaces having random input/output delays

Measuring temperature distribution is an important technique in the fields of meteorology, agriculture, and architecture. The acoustic probe emits sound waves from a speaker and measures the time it takes for the waves to reach a microphone, enabling the measurement of the average temperature between the speaker and the microphone. Additionally, by placing multiple acoustic probes around the measurement area, it becomes possible to measure the temperature distribution using fewer sensors compared to point-type sensors. However, acoustic probes can have higher costs due to the requirement for externally controllable setups or interfaces with minimal audio input/output (I/O) delays. To address this issue, we propose an acoustic probe that achieves accurate measurements even with a low-cost audio interface featuring random I/O delays. The proposed acoustic probe measures the delay time from the recording start to the playback start. This design cancels the error of sound wave propagation time caused by random delays in the audio interface. Through temperature measurement experiments using the proposed acoustic probe, it was revealed that precise measurements (the error of the proposed probe was 0.34°C, while the error of the existing probe was 4.88°C) could be achieved by canceling the random delays in the audio interface.

SyncLabeling: A Synchronized Audio Segmentation Interface for Mobile Devices

Manual audio segmentation is a time-consuming process, especially when there is more than one sound playing simultaneously that needs to be segmented and annotated (e.g., target and background sounds). In conventional audio annotation interfaces, users need to repeatedly pause and replay the audio to complete an overlap segmentation task, which is very inefficient. In this paper, we propose "SyncLabeling," a synchronized audio segmentation interface for smartphones that allows users to segment and annotate two overlapping sounds in a single audio stream at a time using a game-like labeling interface on mobile devices. We conducted a user study to compare the proposed SyncLabeling interface with a conventional audio annotation interface on four types of audio segmentation tasks. The results showed that the proposed interface is much more efficient than the conventional interface (2.4× faster) under comparable annotation accuracy in most tasks. In addition, more than half of the participants enjoyed using the proposed SyncLabeling interface and showed willingness to use it.

An End-to-End Musical Instrument System That Translates Electromyogram Biosignals to Synthesized Sound

Abstract This article presents a custom system combining hardware and software that senses physiological signals of the performer's body resulting from muscle contraction and translates them to computer-synthesized sound. Our goal was to build upon the history of research in the field to develop a complete, integrated system that could be used by nonspecialist musicians. We describe the Embodied AudioVisual Interaction Electromyogram, an end-to-end system spanning wearable sensing on the musician's body, custom microcontroller-based biosignal acquisition hardware, machine learning–based gesture-to-sound mapping middleware, and software-based granular synthesis sound output. A novel hardware design digitizes the electromyogram signals from the muscle with minimal analog preprocessing and treats it in an audio signal-processing chain as a class-compliant audio and wireless MIDI interface. The mapping layer implements an interactive machine learning workflow in a reinforcement learning configuration and can map gesture features to auditory metadata in a multidimensional information space. The system adapts existing machine learning and synthesis modules to work with the hardware, resulting in an integrated, end-to-end system. We explore its potential as a digital musical instrument through a series of public presentations and concert performances by a range of musical practitioners.

Eletric bass use as a teaching resource for the teaching of oscillations and soudwaves

Music has the potential to stimulate the brain to facilitate the memorization of information, with that there is the resourcefulness of seeking a teaching practice for teaching physics. This article proposes a didactic sequence for teaching Physics to 2nd year high school classes, using the practical context of sound waves, taking into account the difficulty of learning that is managed by abstract approaches. For the development of this project, an electric bass, audio interface and a free license software (Audacity) for audio capture were used, in addition, classes were approached to introduce the proposed content and later from the use of the electric instrument to the demonstration through the art of music with an interdisciplinary approach.

Localization of novel reduced-annoyance audio interface sounds

Foley et al. [JASA 151, 3189–3196 (2022)] have demonstrated that annoyance of auditory interface sounds can be reduced both by shortening the duration of upper harmonics and by applying percussive rather than flat amplitude envelopes. However, sounds in that study had a maximum frequency of 2400 Hz, which would likely affect localization based on spectral cues. In a setting with multiple interface devices (e.g., a multi-patient hospital ward), localization of interface sounds is a concern. We characterized normally hearing listeners' ability to localize a variety of candidate reduced-annoyance interface sounds (flat or percussive envelopes; durations between 360 and 1600 ms; all but one with uppermost harmonic limited to 2400 Hz) in quiet or at + 4-dB and −11-dB SNR in spatially diffuse multi-talker babble. Listeners stood at the center of a 360-degree loudspeaker array in a darkened anechoic chamber and used a head-pointing response to report the perceived location of each target. Decreasing SNR increased response variability and the rate of front/rear confusions. For all sounds with restricted bandwidth, front/rear confusions were frequent in the absence of head movements, but when head movements were initiated before target offset, confusions were substantially reduced. The results highlight the need to consider localizability when designing improved auditory interface sounds.

A Novel Machine Learning Based Two-Way Communication System for Deaf and Mute

Deaf and mute people are an integral part of society, and it is particularly important to provide them with a platform to be able to communicate without the need for any training or learning. These people rely on sign language, but for effective communication, it is expected that others can understand sign language. Learning sign language is a challenge for those with no impairment. Another challenge is to have a system in which hand gestures of different languages are supported. In this manuscript, a system is presented that provides communication between deaf and mute (DnM) and non-deaf and mute (NDnM). The hand gestures of DnM people are acquired and processed using deep learning, and multiple language support is achieved using supervised machine learning. The NDnM people are provided with an audio interface where the hand gestures are converted into speech and generated through the sound card interface of the computer. Speech from NDnM people is acquired using microphone input and converted into text. The system is easy to use and low cost. The system is modular and can be enhanced by adding data to support more languages in the future. A supervised machine learning dataset is defined and created that provides automated multi-language communication between the DnM and NDnM people. It is expected that this system will support DnM people in communicating effectively with others and restoring a feeling of normalcy in their daily lives. The hand gesture detection accuracy of the system is more than 90% for most, while for certain scenarios, this is between 80% and 90% due to variations in hand gestures between DnM people. The system is validated and evaluated using a series of experiments.

A Study on Features of Different Tone Quality in a Kenong Set

This work discusses how to distinguish kenong frequencies in a signal and the time-localized frequency content for each tone at a given time using an audio-based approach to tuning retrieval where the fundamental and overtone pitch is shown at all frequencies at a given time. The method of temporal localization on the dominant frequency at its unique time for each tone allows for the detection of frequencies present in the signal. Two approaches used in retrieving the harmonic, pitch, and timbre of kenong are Picoscope and Melda analyzer. The audio recording was done using an At4050 microphone and Ur22 audio interface in mono at 24-bit resolution and 48 kHz sampling rate. PicoScope produces the spectrum while the Melda analyzer produces changes in the spectra with time. Kenong D, E, G, A, and C displayed their near overtones at (2:2.8:4.0), (2:3.0:3.9), (2:2.9:3.9), (2:2.6:3.9), and (2:2.6:3.9). Kenong D had a strong fundamental peak at 295Hz. Kenong G keeps the fundamental frequency constant until t=5s. The basic peak was maintained by Kenong C. The results reveal that the kenong was properly tuned, although the tuner solely tuned it based on hearing, passed down from generation to generation. The maker’s intuition permits him to create a specific ‘signature’ through sound unique to a given kenong set.

Design and Fabrication of Drone Ambulance for Emergency Medication

The emergency medication or the first aid medication is achieved as earlier through the ariel transportation via ambulance drones it will be act as the virtual interface between the doctors in the hospital and the patient want to be treated immediately, this drone contains with the first aid tool kid which will be act as the essential emergency medication vehicle. It is also helps for the servicing in the medical field as transporting medical equipment for testing and pharmaceutical drugs delivery. This drone is specially made for reducing the time wasting for Emergency medication at the golden hour without any traffic delays made from the Regular ambulance. Camera and Monitor, Mic and Speaker will be used for Visual and audio interface between the doctor and the patient. The system is designed to have the short distance flying with more accuracy at less time which carrying a pay load of 3kg this will be helpful to achieve fastest medication at the earlier time.GPS system is used for tracking the location of the patient.

Research of Frequency Characteristics of Middle Ear Ossicles Implants

The paper considers the means to restore human hearing in cases where hearing impairment does not affect the auditory nerve. Then the restoration of hearing is carried out by tympanoplasty - by replacing (restoring) the eardrum or chain of auditory ossicles. The paper examines implants made of bioreplaceable materials that differ greatly in their mechanical properties. Therefore, for the individual selection of ossicular implants in tympanoplasty, a physical model of implants was developed. Based on it, such an important parameter of implants as the value of their resonant frequencies is determined. It was found that the first resonant frequency for a cartilage implant is 61.1 MHz, and for a titanium implant - 487.2 MHz. The paper also compares the theoretical values of the resonant frequencies of implants with the experimental results of their measurement using the original installation. The acoustic part of the experimental setup was an electroacoustic emitter placed on an acoustic vibration absorber (paralon) and a sensitive microphone placed on a counterweight to reduce the pressure on the oscillary prosthesis during measurements and fix it. Signal from the laptop was fed to amplify the input of the audio interface, and from its output signal was sent to the electroacoustic emitter. Between the electroacoustic emitter and the microphone, through the additionally developed and manufactured duralumin nozzles, which provided the introduction of sound vibrations in the prosthesis at angles of 15, 30, 45, 60, a sample of the oscillary prosthesis was recorded. With the help of the installation, the transfer coefficients of samples of ossicular prostheses made of Teflon, titanium, cartilage and bone tissues were measured and compared. It is established that for high frequencies the level of transmission coefficients decreases faster with increasing angle than for low ones, due to the increase in the acoustic size of prostheses with increasing frequency. Also due to the general decrease in the signal level with increasing angle, you can see a decrease with a center frequency of 1 kHz. It is established that at the angle of incidence of the acoustic wave 30, for all samples of prostheses, at a frequency of 4 kHz there is a rise in frequency response. Moreover, for a titanium prosthesis, the value of the transmission coefficient at an angle of incidence of 30 at this frequency exceeds the value of the direct incidence of the acoustic wave.

88 Streaming High-Fidelity ECMO Simulation from a Moving Ambulance

The Trust needed to offer extra-corporeal membrane oxygenation (ECMO) The aim of the study was to deliver high-fidelity ECMO simulation in a moving vehicle while simultaneously allowing candidates and faculty to watch the scenarios take place remotely.A custom video-over-IP system was used to run three camera feeds and a patient monitor output. A laptop captured this in a quad-view format and streamed it to a dedicated video streamer. This was then networked locally via a router with a secondary laptop which displayed the stream. The router was connected to a mobile 4G modem, allowing the secondary laptop to share this video stream via Microsoft Teams (MS Teams). In addition, a USB audio interface and microphones ensured intelligibility while the vehicle was in motion.Teaching groups were made up of 3–4 candidates from the ECMO team and 3–4 candidates from patient transport. Scenarios outlined a paediatric patient, currently on ECMO, being transferred to a specialist hospital in the region via ambulance. One or two candidates from each service would take part in the scenario and the remaining candidates would view the simulation in the centre debrief room. A technician in front of the ambulance controlled the simulator and monitored the streams. The lead ECMO specialist nurse spoke to the technician from the simulation centre via a mobile phone link. The role of a consultant was played by a member of the transport faculty on the ambulance. This allowed the faculty to oversee and prompt when necessary. Using MS Teams meant the stream could be shared with the debrief room at the centre, the control room, and with other interested parties outside of the centre. This created a unique learning experience where all candidates could see each scenario and play an active role in debrief when the ambulance returned to the centre. Successful delivery of this course will improve patient safety during potentially complex ECMO transfers. We hope to invite more remote participants via MS Teams to view the simulations and take part in the debrief, increasing learning opportunities for ECMO and transfer staff.

Developing an engaging whiteboard animation video for vitamins

Whiteboard animation videos are engaging learning media for Generation Z. This study aimed to develop and assess the feasibility of a whiteboard animation video for vitamin in the Nutrition Science subject for class X culinary vocational school. This study was research and development using a 4D approach (define, design, develop, disseminate). The define stage analyzed the curriculum, material, student character, learning media, and school facilities. The design stage included the writing and assessing the material, the writing and assessing of storyboards, and production team selection. The develop stage was making a video and assessing the feasibility. Animation images were created using Paint Tool Sai. Image conversion from JPEG to SVG was performed with Inkscape. The dubber’s voice was recorded with Universal Audio using the Apollo Twin audio interface and Avantone CV12 microphone. The audio was edited in Pro Tools 2020.9. To produce a whiteboard animation video, animation images, captions, dubber’s voice, and back sound were combined with VideoScribe. Videos used mpg format. In order to maintain video duration below 10 minutes, the video was divided into two parts. The assessment of video feasibility was carried out by one media expert and two content experts. The assessment of feasibility at the disseminate stage was carried out to 30 users. Based on the feasibility assessment, this video was very suitable to be applied as a learning medium.

Products of Interest

February 08 2023 Products of Interest Author and Article Information Online Issn: 1531-5169 Print Issn: 0148-9267 © 2022 Massachusetts Institute of Technology2022Massachusetts Institute of Technology Computer Music Journal (2023) 45 (3): 86–100. https://doi.org/10.1162/comj_r_00612 Cite Icon Cite Permissions Share Icon Share Facebook Twitter LinkedIn MailTo Views Icon Views Article contents Figures & tables Video Audio Supplementary Data Peer Review Search Site Citation Products of Interest. Computer Music Journal 2023; 45 (3): 86–100. doi: https://doi.org/10.1162/comj_r_00612 Download citation file: Ris (Zotero) Reference Manager EasyBib Bookends Mendeley Papers EndNote RefWorks BibTex toolbar search Search Dropdown Menu toolbar search search input Search input auto suggest filter your search All ContentAll JournalsComputer Music Journal Search Advanced Search The Zoom R20 is a standalone multitrack recorder, mixer, and editor in one (see Figure 1). It has six XLR inputs with 48-V phantom power available on four of them. There are also two combination microphone/line inputs, one of which is a Hi-Z instrument input. The audio connections are all located on the top panel for easy access. The R20 also has a built-in USB-C audio interface for connecting the recorder to a computer or other audio device. It supports up to eight inputs and four outputs in multitrack mode. Up to 16 tracks can be recorded, eight of them simultaneously. There are two banks of eight faders and gain knobs, as well as a master fader and transport section. The recorder also has a 4.3-in Figure 1.View largeDownload slideThe Zoom R20 multitrack recorder, mixer, and editor.Figure 1.View largeDownload slideThe Zoom R20... You do not currently have access to this content.

Assistive Navigation Using Deep Reinforcement Learning Guiding Robot With UWB/Voice Beacons and Semantic Feedbacks for Blind and Visually Impaired People.

Facilitating navigation in pedestrian environments is critical for enabling people who are blind and visually impaired (BVI) to achieve independent mobility. A deep reinforcement learning (DRL)–based assistive guiding robot with ultrawide-bandwidth (UWB) beacons that can navigate through routes with designated waypoints was designed in this study. Typically, a simultaneous localization and mapping (SLAM) framework is used to estimate the robot pose and navigational goal; however, SLAM frameworks are vulnerable in certain dynamic environments. The proposed navigation method is a learning approach based on state-of-the-art DRL and can effectively avoid obstacles. When used with UWB beacons, the proposed strategy is suitable for environments with dynamic pedestrians. We also designed a handle device with an audio interface that enables BVI users to interact with the guiding robot through intuitive feedback. The UWB beacons were installed with an audio interface to obtain environmental information. The on-handle and on-beacon verbal feedback provides points of interests and turn-by-turn information to BVI users. BVI users were recruited in this study to conduct navigation tasks in different scenarios. A route was designed in a simulated ward to represent daily activities. In real-world situations, SLAM-based state estimation might be affected by dynamic obstacles, and the visual-based trail may suffer from occlusions from pedestrians or other obstacles. The proposed system successfully navigated through environments with dynamic pedestrians, in which systems based on existing SLAM algorithms have failed.

Quantitative analysis of piano performance proficiency focusing on difference between hands

Quantitative evaluation of piano performance is of interests in many fields, including music education and computational performance rendering. Previous studies utilized features extracted from audio or musical instrument digital interface (MIDI) files but did not address the difference between hands (DBH), which might be an important aspect of high-quality performance. Therefore, we investigated DBH as an important factor determining performance proficiency. To this end, 34 experts and 34 amateurs were recruited to play two excerpts on a Yamaha Disklavier. Each performance was recorded in MIDI, and handcrafted features were extracted separately for the right hand (RH) and left hand (LH). These were conventional MIDI features representing temporal and dynamic attributes of each note and computed as absolute values (e. g., MIDI velocity) or ratios between performance and corresponding scores (e. g., ratio of duration or inter-onset interval (IOI)). These note-based features were rearranged into additional features representing DBH by simple subtraction between features of both hands. Statistical analyses showed that DBH was more significant in experts than in amateurs across features. Regarding temporal features, experts pressed keys longer and faster with the RH than did amateurs. Regarding dynamic features, RH exhibited both greater values and a smoother change along melodic intonations in experts that in amateurs. Further experiments using principal component analysis (PCA) and support vector machine (SVM) verified that hand-difference features can successfully differentiate experts from amateurs according to performance proficiency. Moreover, existing note-based raw feature values (Basic features) and DBH features were tested repeatedly via 10-fold cross-validation, suggesting that adding DBH features to Basic features improved F1 scores to 93.6% (by 3.5%) over Basic features. Our results suggest that differently controlling both hands simultaneously is an important skill for pianists; therefore, DBH features should be considered in the quantitative evaluation of piano performance.

Cognitive Audio Interfaces: Mediating Sonic Information With an Understanding of How We Hear

In a world of rich, complex, and demanding audio environments, intelligent systems can mediate our interaction with the sounds around us—both to enable meaningful, aesthetic experiences and to transition work from humans to computational agents. Drawing from several years of our research, we suggest that the design of such systems must be driven by a deep understanding of auditory cognition. In this article, we discuss two concrete approaches we take toward cognition-informed interface design—one that begins with sounds themselves to form explicit, contextualized, cognitive models, built on the foundations of large-data parsing infrastructure; and one that begins with the individual, built from intuition surrounding the influence of the cognitive state on perception. We point toward an unexplored and compelling future at their intersection.