Audio Input Research Articles

Auditory processing as perceptual, cognitive, and motoric abilities underlying successful second language acquisition: Interaction model.

A growing amount of attention has been given to examining the domain-general auditory processing of individual acoustic dimensions as a key driving force for adult L2 acquisition. Whereas auditory processing has traditionally been conceptualized as a bottom-up and encapsulated phenomenon, the interaction model (Kraus & Banai, 2007) proposes auditory processing as a set of perceptual, cognitive, and motoric abilities-the perception of acoustic details (acuity), the selection of relevant and irrelevant dimensions (attention), and the conversion of audio input into motor action (integration). To test this hypothesis, we examined the relationship between each component and the L2 outcomes of 102 adult Chinese speakers of English who varied in age, experience, and working memory background. According to the results of the statistical analyses, (a) the tests scores tapped into essentially distinct components of auditory processing (acuity, attention, and integration), and (b) these components played an equal role in explaining various aspects of L2 learning (phonology, morphosyntax) with large effects, even after biographical background and working memory were controlled for. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Informatization Integration Strategy of Modern Popular Music Teaching and Traditional Music Culture in Colleges and Universities in the Era of Artificial Intelligence

Abstract In this paper, a music teaching aid is constructed based on the automatic analysis technique of music elements. The auxiliary tool uses constant Q transform and note onset detection technology to process and recognize students’ input audio. The logical operation of music longitudinal structure and spectral structure and the logical conversion of beat and rhythmic time nodes are used to realize the integration of popular music and traditional music teaching informationization. Using two corpora of 22 kHz, the effect of teaching the fusion of musical elements of auxiliary tools was analyzed with the regular transformation of curves in the sound intensity envelope. The results showed that the peaks of the overlap between elements of popular music and traditional music appeared at 56.8 FPs, 148.9 FPs, and 343.7 FPs, respectively, corresponding to the peaks of 48.3 Hz, 59.4 Hz, and 51.6 Hz, respectively. The integration of traditional music culture into popular music teaching was successfully realized. This study effectively examined an effective method and strategy for integrating teaching, which has a positive impact on the promotion and development of traditional music culture.

AvatarStudio: Text-Driven Editing of 3D Dynamic Human Head Avatars

Capturing and editing full-head performances enables the creation of virtual characters with various applications such as extended reality and media production. The past few years witnessed a steep rise in the photorealism of human head avatars. Such avatars can be controlled through different input data modalities, including RGB, audio, depth, IMUs, and others. While these data modalities provide effective means of control, they mostly focus on editing the head movements such as the facial expressions, head pose, and/or camera viewpoint. In this paper, we propose AvatarStudio, a text-based method for editing the appearance of a dynamic full head avatar. Our approach builds on existing work to capture dynamic performances of human heads using Neural Radiance Field (NeRF) and edits this representation with a text-to-image diffusion model. Specifically, we introduce an optimization strategy for incorporating multiple keyframes representing different camera viewpoints and time stamps of a video performance into a single diffusion model. Using this personalized diffusion model, we edit the dynamic NeRF by introducing view-and-time-aware Score Distillation Sampling (VT-SDS) following a model-based guidance approach. Our method edits the full head in a canonical space and then propagates these edits to the remaining time steps via a pre-trained deformation network. We evaluate our method visually and numerically via a user study, and results show that our method outperforms existing approaches. Our experiments validate the design choices of our method and highlight that our edits are genuine, personalized, as well as 3D- and time-consistent.

Speech Enhancement using Convolutional Autoencoder Network

Abstract—We present an end-to-end deep learning approach to denoising speech signals by processing the raw waveform directly. Given input audio containing speech corrupted by an additive background signal, the system aims to produce a processed signal that contains only the speech content. Recent approaches have shown promising results using various deep network architec- tures. In this paper, we propose to train a fully-convolutional context aggregation network using a deep feature loss. That loss is based on comparing the internal feature activations in a different network, trained for acoustic environment detection and domestic audio tagging. Our approach outperforms the stateof- the-art in objective speech quality metrics and in large-scale perceptual experiments with human listeners. It also outperforms an identical network trained using traditional regression losses. The advantage of the new approach is particularly pronounced for the hardest data with the most intrusive background noise, for which denoising is most needed and most challenging. Index Terms—speech enhancement,Fully convolutional denois- ing autoencoders, single channel audio source separation, stacked convolutional autoencoders, deep convolutional neural networks, deep learning.

Speech Recognition and Sending Alert Message If Abusive Words are Detected

Abstract: This paper introduces an innovative approach to bolstering the field of speech recognition. Speech recognition, a pivotal process in facilitating computer systems' understanding and response to natural language commands, has garnered considerable importance in today's digital landscape. Our novel method aims to enable users to issue voice commands with enhanced accuracy and efficiency. This approach encompasses several critical steps: the intake of a single audio input, effective noise reduction, and the utilization of pattern matching for audio recognition. To fortify the system's robustness, we've integrated an error tolerance mechanism capable of accommodating human errors and mitigating environmental noise.What sets our method apart is its ability to circumvent the need for storing extensive audio data for matching, leading to a substantial reduction in both space and time complexity when compared to conventional systems. In our experiments, we have exclusively employed a database containing an array of predefined voice patterns, effectively eliminating the necessity for end-user training.

Vocal Visage: Crafting Lifelike 3D Talking Faces from Static Images and Sound

In the field of computer graphics and animation, the challenge of generating lifelike and expressive talking face animations has historically necessitated extensive 3D data and complex facial motion capture systems. However, this project presents an innovative approach to tackle this challenge, with the primary goal of producing realistic 3D motion coefficients for stylized talking face animations driven by a single reference image synchronized with audio input. Leveraging state-of-the-art deep learning techniques, including generative models, image-to-image translation networks, and audio processing methods, the methodology bridges the gap between static images and dynamic, emotionally rich facial animations. The ultimate aim is to synthesize talking face animations that exhibit seamless lip synchronization and natural eye blinking, thereby achieving an exceptional degree of realism and expressiveness, revolutionizing the realm of computer-generated character interactions.

May I know my EQ? Factors to automate EQ prediction using technology

Emotional intelligence (EI) involves recognizing, regulating, and understanding emotions in oneself and others, enabling effective relationship management. Emotional Intelligence acts as a differentiator in both personal and professional lives. Nevertheless, many people lack the ability of conveying and interpreting human emotions efficiently due to which they fall short in their social and professional lives. However, this can be improved if people know their Emotional Quotient (EQ) score and work towards improving it by implementing certain skills. Although traditional methods for measuring EQ exist, majority of them are manual and are built on self-efficacy based questionnaire which are often susceptible to faking. In order to overcome these lacuna, this paper proposes a digital approach that can be used for an automatic evaluation of EQ of a person via audio visual signal input. The paper devises four major factors self-awareness, self-regulation, self-motivation and social relationships with their weightages that can be utilised for automatic evaluation through the use of technology. The paper also proposes quantifiable parameters for measuring these factors. The proposed criteria were developed with the emergence of deep learning and its applicability in assessing a person's EQ in mind.

Generative Adversarial Network with Convolutional Wavelet Packet Transforms for Automated Speaker Recognition and Classification

Speech is an effective mode of communication that always conveys abundant and pertinent information, such as the gender, accent, and other distinguishing characteristics of the speaker. These distinctive characteristics allow researchers to identify human voices using artificial intelligence (AI) techniques, which are useful for forensic voice verification, security and surveillance, electronic voice eavesdropping, mobile banking, and mobile purchasing. Deep learning (DL) and other advances in hardware have piqued the interest of researchers studying automatic speaker identification (SI). In recent years, Generative Adversarial Networks (GANs) have demonstrated exceptional ability in producing synthetic data and improving the performance of several machine learning tasks. The capacity of Convolutional Wavelet Packet Transform (CWPT) and Generative Adversarial Networks are combined in this paper to propose a novel way of enhancing the accuracy and robustness of Speaker Recognition and Classification systems. Audio signals are dissected using the Convolutional Wavelet Packet Transform into a multi-resolution, time-frequency representation that faithfully preserves local and global characteristics. The improved audio features better precisely describe speech traits and handle pitch, tone, and pronunciation variations that are frequent in speaker recognition tasks. Using GANs to create synthetic speech samples, our suggested method GAN-CWPT enriches the training data and broadens the dataset's diversity. The generator and discriminator components of the GAN architecture have been tweaked to produce realistic speech samples with attributes quite similar to genuine speaker utterances. The new dataset enhances the Speaker Recognition and Classification system's robustness and generalization, even in environments with little training data. We conduct extensive tests on standard speaker recognition datasets to determine how well our method works. The findings demonstrate that, compared to conventional methods, the GAN-CWPTs combination significantly improves speaker recognition, classification accuracy, and efficiency. Additionally, the suggested model GAN-CWPT exhibits stronger generalization on unknown speakers and excels even with loud and poor audio inputs.

Reconfigurable Multiband Dynamic Range Compression-based FRM Filter for Hearing Aid

In this research, we present an innovative method for enhancing the performance of hearing aids using a Multiband Dynamic Range Compression-based Reconfigurable Frequency Response Masking (FRM) Filterbank. First, a unform16-band reconfigurable filter bank, which is reconfigurable, is designed utilizing the FRM scheme. The strategic arrangement of each sub-band within the proposed filter bank is meticulously prepared to optimize the matching performance. Based on the hearing characteristics of patients, the sub-bands can be distributed in low, medium, and high-frequency regions. Also, the gain can be adjusted per the patient's hearing profile from their audiogram for better auditory compensation.
 Further, the Multiband Dynamic Range Compression (MBDRC) technique is applied to address the specific needs of individuals with different frequency-dependent hearing impairments. It involves using dynamic range compression independently to different frequency sub-bands within a filter bank. In MBDRC, the compression parameters, such as compression threshold and ratio, can be adjusted independently for every subband. It allows for a more tailored approach to address the specific hearing needs of different frequency regions. If an individual has more severe hearing loss in high-frequency regions, higher compression ratios and lower compression thresholds can be applied to those subbands to amplify and improve audibility for high-frequency sounds. Once dynamic range compression is applied to each sub-band, the resultant sub-bands are reassembled to yield the ultimate output signal, which can subsequently be transmitted to the speaker or receiver of the hearing aid. A GUI can be helpful for better visualization and parameter control, including gain adjustment and compression parameters of this entire process. With this aim in mind, a GUI has been developed on MATLAB. Different audio files can be imported, and their frequency response can be generated and observed. Based on a person's audiogram, the control parameters can be set to low, medium, or high. Their sub-band distribution in low, medium, and high-frequency regions can be visualized. Further, the filter bank makes automatic gain adjustments, as seen in the GUI. The gain points for each band can also be manually adjusted according to users' hearing characteristics to minimize the error. Also, the compression parameters can be set separately for each subband as per the hearing requirement of the patient. Further, the processed output can be visualized in the output frequency response tab, and the input and output audio signals can be analyzed.

COVID-19 Intelligent Detection System Based on Cloud-Edge Fusion Recognition Model

Analyzing cough sounds can help with the quick detection of COVID-19. A cloud-edge deep learning fusion-based intelligent detection system for COVID-19 is proposed in this paper. In the cloud-side, a COVID-19 detection model based on ResNet18 is employed, with log-Mel-spectrum characteristics used as inputs. In the edge-side, a COVID-19 detection model based on TCNN is developed using raw audio inputs. To improve the detection accuracy, result fusion is carried out in the cloud-side after getting the recognition results from both models. On the test dataset, the fusion model attained a sensitivity of 0.8012, an AUC of 0.8251, and a specificity of 0.7255. According to comparative testing results, the fusion model outperforms the other models in classification performance and is less prone to false-positive errors. It provides a novel way to COVID-19 recognition and performs well as an auxiliary detection method.

English Pronunciation Quality Evaluation System Based on Continuous Speech Recognition Technology for Multi-Terminal

This paper presents a novel approach for evaluating the pronunciation quality of English speech using continuous speech recognition technology. The research focuses on the application of artificial intelligence in speech recognition, utilizing web browsers on various terminal devices such as computers, mobile phones, and tablets to allow users to read the provided text aloud. The web program captures audio input from the microphone, records it in MP3 format, and uploads it to the server. The server employs the Whisper model to transcribe the audio into semantic text, which is then compared with the displayed text. By calculating the semantic distance and assessing the accuracy of pronunciation, the system provides an evaluation of pronunciation quality, marking correct and incorrect words. To achieve real-time processing, the compact tiny model is employed, and further optimization is performed using Ctranslate 2, resulting in significant performance improvements.

Research on music signal feature recognition and reproduction technology based on multilayer feedforward neural network

Abstract In this paper, a multi-layer feed-forward neural network is used to construct a Meier spectrogram recognition system. By analyzing the algorithmic role of recurrent neural, the backpropagation algorithm is applied to update the weights in the neural network to obtain the mapping relationship between audio input and output. Combined with the algorithmic formula of the spectrum, the short-time Fourier transform is used to analyze the audio information. By architecting a multilayer feedforward recurrent neural network, the music signals are fused and classified. The cross-entropy loss function is applied to calculate the accuracy of micro and macro averages to improve the accuracy of music signal feature recognition. The results show that the feedforward recurrent neural network has the lowest error rate in different note recognition, and the error rate for “do” is 4%.

PENGEMBANGAN MEDIA PEMBELAJARAN MAHᾹRAH QIRᾹ’AH DENGAN MENGGUNAKAN BOOK CREATOR

<p style="text-align: justify;"><em>With the rapid development of technology today, it is an opportunity for education to continue to innovate in creating an enjoyable learning environment. Conventional learning media has often become commonplace for students. Therefore, a breakthrough is needed in the form of web-based learning media. The method used in this research uses a type of research and development using the ADDIE (Analysis Design Development Evaluation) model. So, this research aims to develop Book Creator media to increase students' interest and motivation in studying Arabic and improve language skills; both Maharah Istima' wal Kalam and Qiro'ah wal Kitabah, which on this website have specifications in the form of input files, both text, images, audio, and video, which are arranged systematically to produce interactive learning media, so that the use of learning media using Book Creator is very feasible to develop. There are 23 steps in creating media using Book Creator. Still, there are only four steps in creating it, starting with creating a concept and preparing materials, then logging in, designing, and then distributing.</em></p>

Blueprint Separable Subsampling and Aggregate Feature Conformer-Based End-to-End Neural Diarization

At present, a prevalent approach to speaker diarization is clustering based on speaker embeddings. However, this method encounters two primary issues. Firstly, it cannot directly minimize the diarization error during the training process; secondly, the majority of clustering-based methods struggle to handle speaker overlap in audio. A viable approach for addressing these issues involves adopting end-to-end speaker diarization (EEND). Nevertheless, training this EEND system generally requires lengthy audio inputs, which must be downsampled to allow efficient model processing. In this study, we develop a novel downsampling layer using blueprint separable convolution (BSConv) instead of depthwise separable convolution (DSC) as the foundational convolutional unit, which effectively preserves information from the original audio. Furthermore, we incorporate multi-scale feature aggregation (MFA) into the encoder structure to combine the features extracted by each conformer block to the output layer, consequently enhancing the expressiveness of the model’s feature extraction. Lastly, we employ the conformer as the backbone network to incorporate the proposed enhancements, resulting in an EEND system named BSAC-EEND. We assess our suggested methodology on both simulated and real datasets. The experiment indicates that our proposed EEND system reduces diarization error rate (DER) by an average of 17.3% for two-speaker datasets and 12.8% for three-speaker datasets compared to the baseline.

An underwater sound event detection algorithm based on Temporal Graph Convolutional Network

Accurate underwater sound event detection (SED) and real-time underwater situation awareness plays an important role in intelligent sonar system. However, little research on underwater SED has been done because underwater acoustics have very complex and dynamic spectral and temporal characteristics which make underwater SED difficult to solve. The goal of SED is to recognize type and time of the input audio signal and most of SED research have been focused on airborne sound so far. To address underwater SED problem, we adopt temporal graph convolutional network (T-GCN), which is originally designed for traffic prediction and is composed of GCN and gated recurrent unit (GRU) To capture spectral and temporal information simultaneously, the GCN is used to learn topological structures to capture spectral correlation and the GRU is used to learn changes of input sound to capture temporal correlation. The proposed model utilizes sound spectrogram and its annotation with temporal information as input and then the GCN and GRU of the model is employed to solve underwater SED problem. Experiments using urban SED dataset and underwater sound dataset, demonstrate that our proposed model expresses connection of spectral and temporal information effectively and shows reliable underwater SED performance accordingly.

Multipitch estimation based on the iterative detection and separation of note events from single-channel polyphonic recordings.

Multiple fundamental frequency estimation has been extensively used in applications such as melody extraction, music transcription, instrument identification, and source separation. This paper presents an approach based on the iterative detection and extraction of note events, which are considered to be harmonic sounds characterised by a continuous pitch trajectory. Note events are assumed to be associated with musical notes being played by a single instrument, and their pitch trajectories are iteratively estimated. In every iteration, the pitch contour of the predominant note event is selected from a set of pitch estimates and used to separate its spectral energy from the input mixture in order to obtain a residual signal, which is then used as input in the next iteration. This iterative process stops when the energy of the residual is below a significance threshold. The pitch trajectories of all detected note events are then revised and reassembled to form the final set of pitch estimates for the original audio input. Evaluation of performance is conducted in different scenarios to show the potential of the proposed system, both in terms of its accuracy, and also as an initial stage in other complex tasks, such as note tracking and multipitch streaming.

On the Detection of Adaptive Adversarial Attacks in Speaker Verification Systems

Speaker verification systems have been widely used in smart phones and Internet of things devices to identify legitimate users. In recent work, it has been shown that adversarial attacks, such as FAKEBOB, can work effectively against speaker verification systems. The goal of this paper is to design a detector that can distinguish an original audio from an audio contaminated by adversarial attacks. Specifically, our designed detector, called MEH-FEST, calculates the minimum energy in high frequencies from the short-time Fourier transform of an audio and uses it as a detection metric. Through both analysis and experiments, we show that our proposed detector is easy to implement, fast to process an input audio, and effective in determining whether an audio is corrupted by FAKEBOB attacks. The experimental results indicate that the detector is extremely effective: with near zero false positive and false negative rates for detecting FAKEBOB attacks in Gaussian mixture model (GMM) and i-vector speaker verification systems. Moreover, adaptive adversarial attacks against our proposed detector and their countermeasures are discussed and studied, showing the game between attackers and defenders.

EmoBot: Artificial emotion generation through an emotional chatbot during general-purpose conversations

Emotion modeling has always been intriguing to researchers, where detecting emotion is highly focused and generating emotion is much less focused to date. Therefore, in this paper, we aim to exploring emotion generation, particularly for general-purpose conversations. Based on the Cognitive Appraisal Theory and focusing on audio and textual inputs, we propose a novel method to calculate informative variables to evaluate a particular emotion-generating event and six primary emotions. Incorporating such a method of artificial emotion generation, we implement an emotional chatbot, namely EmoBot. Accordingly, EmoBot analyzes continuous audio and textual inputs, calculates the informative variables to evaluate the current situation, generates appropriate emotions, and responds accordingly. An objective evaluation indicates that EmoBot could generate more accurate emotional and semantic responses than a traditional chatbot that does not consider emotion. Additionally, a subjective evaluation of EmoBot demonstrates the appreciation of users for EmoBot over a traditional chatbot that does not consider emotion.

Optimized Binaural Enhancement via attention masking network-based speech separation framework in digital hearing aids

Earlier studies have utilized microphone signal processing for performing the target speech evaluation and separation-like feature recognition by involving a huge amount of training data along with supervised machine learning. These approaches are most appropriate for stationary noise suppression; however, it is challenging for non-stationary noise, and it does not satisfy the practical processing requirement. To overcome the former issue, a system is presented for “joint speaker separation, and noise suppression”, referred to as the Optimized Binaural Enhancement via Attention Masking Network (OBEAMNET). In this paper, a speech separation model for a hearing aid is suggested. Here, the noise speech is separated into two different speeches through the proposed OBEAMNET approach. Initially, the standard input audio signals are gathered from standard benchmark datasets. Further, the cepstral features are gathered from the input data for getting the essential information. A novel Fitness Ordered Black Widow Optimization (FO-BWO) algorithm is developed for optimizing the features to increase the PSNR as well as decrease the RMSE. The feature selection is carried out by FO-BWO to get the most noteworthy information from audio signals. The suggested FO-BWO, along with the OBEAMNET technique, is used for efficiently separating the signals. Finally, the optimal features are fed to the speech separation using the OBEAMNET approach for improving the effectiveness of the proposed model that is observed in decoding auditory attention in noisy practical environments.

Augmedix Bangladesh re-humanizing healthcare service

In 2012, Stanford University graduates “Ian Shakil” and “Pelu Tan” founded Augmedix in Silicon Valley (Augmedix Remote Documentation Services, 2019). Augmedix serves patients in the United States and three other countries. One of them is Bangladesh. This case is developed to present the operational framework of Augmedix Bangladesh based on secondary sources of information. To free up doctors’ time, this app instantly documents doctor-patient talks in the patient’s medical file. Augmedix’s platform uses NLP Technology and skilled staff to ensure correct and consistent documentation. In Bangladesh, this highly skilled group goes by the name “Scribe.” Using audio and visual input, a doctor in the United States and a scribe in Bangladesh (Augmedix BD) build a patient’s medical record in real-time. Clinicians using the Augmedix Ambient Automation Platform carry out Augmedix processes. This innovation will reduce the amount of time spent by doctors manually documenting patient information. This innovative technology might save doctors two to 3 hours per day. In this service, they provide live communication, notes, data service, etc.