Speaker Localization Research Articles

The development of an AVAS for enhancing the robustness of acoustic performance and reducing interior noise in a cabin

All the electric vehicles are required to provide warning sound by AVAS (Acoustic Vehicle Alert System) for the safety of pedestrians. AVAS sound should satisfy the regulations outside a vehicle while noise caused by the AVAS should be maintained below a level inside a cabin. Nowadays, it becomes difficult to fulfill these goals since various sensors for automotive drive are newly introduced in the space where the AVAS is traditionally located and an open area to emit sound is decreased to improve an electric efficiency of a car. Since the AVAS performance is very sensitive to a small change of the cavity, the increased complexity of the cavity makes it more challenging to develop a robust AVAS. The purpose of this study is to suggest a new AVAS for enhancing the robustness of acoustic performance as well as reducing interior noise in a cabin. Firstly, a series of simulations are carried out to closely investigate various design factors of the AVAS. Secondly, the improved performance of the new AVAS is shown through tests and simulations. Thirdly, a method to decide the optimal speaker location is newly suggested by simultaneously considering crucial performance factors obtained from multiple vehicle level simulations.

Deep beamforming for speech enhancement and speaker localization with an array response-aware loss function

Recent research advances in deep neural network (DNN)-based beamformers have shown great promise for speech enhancement under adverse acoustic conditions. Different network architectures and input features have been explored in estimating beamforming weights. In this paper, we propose a deep beamformer based on an efficient convolutional recurrent network (CRN) trained with a novel ARray RespOnse-aWare (ARROW) loss function. The ARROW loss exploits the array responses of the target and interferer by using the ground truth relative transfer functions (RTFs). The DNN-based beamforming system, trained with ARROW loss through supervised learning, is able to perform speech enhancement and speaker localization jointly. Experimental results have shown that the proposed deep beamformer, trained with the linearly weighted scale-invariant source-to-noise ratio (SI-SNR) and ARROW loss functions, achieves superior performance in speech enhancement and speaker localization compared to two baselines.

Localization and Speech-in-Noise Performance in a Virtual Reality Test Environment: A Pilot Study of Adults With Single-Sided Deafness Using a Cochlear Implant.

The purpose of this exploratory study was to (a) construct a virtual reality (VR) test environment to measure speech recognition in noise (SIN) and localization, and (b) use the VR test environment to establish degree of binaural hearing benefit among a small number of adults with single-sided deafness (SSD) using a cochlear implant (CI). This pilot study included five adults implanted for SSD. The test environment was composed of an eight-speaker array that delivered restaurant noise and Institute of Electrical and Electronics Engineers sentences. VR head-mounted display goggles delivered a video recording of a busy restaurant. Participants completed SIN and localization in two conditions: (a) normal-hearing ear and a CI on the contralateral SSD side (CI-ON) and (b) normal-hearing ear and unaided on the contralateral SSD side (CI-OFF). Overall, CI benefits for SIN and localization within the VR test environment were improved for some participants, although not all. CI benefit for SIN and localization was dependent on speaker location. VR test environments present new opportunities for studying SIN and localization abilities in participants with CIs. This pilot study shows that, within a VR test environment, degree of CI benefit among SSD participants for SIN and localization varies across speaker location and across participants.

Gays Against Groomers and the politics of digital ventriloquism

ABSTRACT Prior to its political appropriation, “groomer” characterized adults earning children's trust, so they can manipulate and sexually exploit them. Conservative activists have recently transformed “groomer” into an anti-lesbian, gay, bisexual, transgender, and queer (LGBTQ) slur. This critique of anti-“groomer” rhetoric examines an organization called Gays Against Groomers (GAG). GAG's website, which is presumably run by “gays,” depicts children as especially vulnerable to queer “indoctrination” and “mutilation.” Investigative probes recently conducted by The Advocate, the Anti-Defamation League (“Online Amplifiers,” 2023), and Media Matters revealed that far-right hate groups and anti-LGBTQ conservative media strategists are largely responsible for the group's genesis. GAG participates in a duplicitous online discourse I call digital ventriloquism. In this context, digital ventriloquy describes online communication where anti-LGBTQ cadres throw their voices into a “dummy” organization that seemingly operates from a divergent social location. GAG's homonormative cooptation of the groomer pejorative results in a dangerous paradox of speaker location, or social identity, that virtually erases child molestation from anti-“grooming” speech, and recasts transgender people and progressive gender education as child pollutants.

Localization of Speaker using Fusion Techniques and Neural Network Algorithms

ABSTACT Sound source localization especially speech and speaker is sole of the most significant techniques recently because used in various applications like smart environments, industry, robots, and audio conferences. So, the usage of these techniques needs more accuracy. In this paper, a speaker localization proposed it depends on the speech signals in closed spaces by employing fusion techniques and neural networks (NN) algorithms to get more accuracy. The proposed work included finding the classification of the speaker signals, which included three phases: the preprocessing phase, the phase of the feature extraction and classification phase. Data Fusion technique used to generate the dataset of speakers. In feature extraction phase features fusion technique was used for constructing a feature vector by using Generalized Cross Correlation (GCC) for time delay estimation, Root_MUSIC, and Minimum Variance Distortion Less (MVDR) for a direction of arrival for the signal source. In the classification stage two NN algorithms used, Restricted Boltzmann Machine (RBM), which implemented using Tensor flow library and Long Short-Term Memory (LSTM), which implemented using Keras library. The experiments results shows that the accuracy of the two methods was 99.84%, 99.15% for RBM, and LSTM respectively.

Multisensory Fusion for Unsupervised Spatiotemporal Speaker Diarization.

Speaker diarization consists of answering the question of "who spoke when" in audio recordings. In meeting scenarios, the task of labeling audio with the corresponding speaker identities can be further assisted by the exploitation of spatial features. This work proposes a framework designed to assess the effectiveness of combining speaker embeddings with Time Difference of Arrival (TDOA) values from available microphone sensor arrays in meetings. We extract speaker embeddings using two popular and robust pre-trained models, ECAPA-TDNN and X-vectors, and calculate the TDOA values via the Generalized Cross-Correlation (GCC) method with Phase Transform (PHAT) weighting. Although ECAPA-TDNN outperforms the Xvectors model, we utilize both speaker embedding models to explore the potential of employing a computationally lighter model when spatial information is exploited. Various techniques for combining the spatial-temporal information are examined in order to determine the best clustering method. The proposed framework is evaluated on two multichannel datasets: the AVLab Speaker Localization dataset and a multichannel dataset (SpeaD-M3C) enriched in the context of the present work with supplementary information from smartphone recordings. Our results strongly indicate that the integration of spatial information can significantly improve the performance of state-of-the-art deep learning diarization models, presenting a 2-3% reduction in DER compared to the baseline approach on the evaluated datasets.

Robust DOA Estimation Using Multi-Scale Fusion Network with Attention Mask

To overcome the limitations of traditional methods in reverberant and noisy environments, a robust multi-scale fusion neural network with attention mask is designed to improve direction-of-arrival (DOA) estimation accuracy for acoustic sources. It combines the benefits of deep learning and complex-valued operations to effectively deal with the interference of reverberation and noise in speech signals. The unique properties of complex-valued signals are exploited to fully capture inherent features and rich information is preserved in the complex field. An attention mask module is designed to generate distinct masks for selectively focusing and masking based on the input. After that, the multi-scale fusion block efficiently captures multi-scale spatial features by stacking complex-valued convolutional layers with small size kernels, and reduces the module complexity through special branching operations. Experimental results demonstrate that the model achieves significant improvements over other methods for speaker localization in reverberant and noisy environments. It provides a new solution for DOA estimation for acoustic sources in different scenarios, which has significant theoretical and practical implications.

Designing a smart classroom with advanced Audiovisual equipment and soft collaboration capabilities for Higher Education

This is a technology-enabled classroom design which reflects a thoughtful integration of cutting-edge audiovisual solutions to enhance the learning environment. The inclusion of automated audio-visual solutions indicates a commitment to creating a seamless and efficient learning experience through technology. The incorporation of a voice lift application ensures clear and audible communication for the instructor, addressing potential challenges in large classroom settings. The use of Dante-based communication over Ethernet signifies a high-quality and reliable audio distribution system, enabling efficient communication within the classroom. The inclusion of features catering to individuals with disabilities demonstrates a commitment to creating an inclusive and accessible learning environment. The beamforming microphone array system with multiple elements is a sophisticated technology that allows for effective tracking of the active participant. This ensures that the audio focus adapts to the speaker's location. The automatic and steerable coverage type enhances the adaptability of the microphone array system, automatically focusing on the speaker. This feature contributes to an immersive and dynamic learning experience. The support for Power over Ethernet (PoE) simplifies the installation and management of AV devices in the classroom. It reduces the need for additional power outlets and cables, streamlining the overall setup. Overall, this design prioritizes advanced audiovisual technologies to create a unique, state-of-the-art, and inclusive technology-enabled classroom. The focus on clear communication, adaptability, and ease of use demonstrates a commitment to providing an optimal learning environment for both faculty and students

CEFR-based Model of Indonesian Grammar Teaching Materials for Foreign Students

Law No. 24/2009 mandates the internationalization of the Indonesian language that is in line with the increase in the learning of the Indonesian language for foreign speakers. The number and location of Indonesian speakers are increasing and expanding in various parts of the world. However, the existing teaching materials are not optimal in supporting BIPA teaching, especially in the aspect of grammar—the most difficult language component for foreign students to learn. Thus, it is imperative to develop a model of Indonesian grammar teaching materials based on the current curriculum that refers to CEFR. This research focuses on the development of an Indonesian grammar teaching material model for A1-level BIPA students. The development design employed is an adaptation of the R&D design by Borg and Gall, which consists of several phases, such as (1) model development, (2) model evaluation, and (3) production and dissemination. This initial research constitutes a model development phase that includes (a) needs analysis, (b) needs identification based on the existing Indonesian grammar teaching materials and studies on the document of CEFR-based BIPA Graduate Competence Standard, and (c) compiling Indonesian grammar teaching materials. This preliminary research results in the mapping and drafting the initial product of the model of Indonesian grammar teaching materials for students at the A1 level.
 Keywords: teaching material model, Indonesian grammar, BIPA (Indonesian Language for Foreign Speakers), CEFR

Learning Multi-Dimensional Speaker Localization: Axis Partitioning, Unbiased Label Distribution, and Data Augmentation

Learning Multi-Dimensional Speaker Localization: Axis Partitioning, Unbiased Label Distribution, and Data Augmentation

Meta-semantic practices in social interaction

Abstract In social interaction, different kinds of word-meaning can become problematic for participants. This study analyzes two meta-semantic practices, definitions and specifications, which are used in response to clarification requests in German implemented by the format Was heißt X (‘What does X mean?’). In the data studied, definitions are used to convey generalizable lexical meanings of mostly technical terms. These terms are either unknown to requesters, or, in pedagogical contexts, requesters ask in order to check the addressee’s knowledge. Specifications, in contrast, clarify aspects of local speaker meanings of ordinary expressions (e.g., reference, participants in an event, standards applied to scalar expressions). Both definitions and specifications are recipient-designed with respect to the (presumed) knowledge of the addressee and tailored to the topical and practical relevancies of the current interaction. Both practices attest to the flexibility and situatedness of speakers’ semantic understandings and to the systematicity of using meta-semantic practices differentially for different kinds of semantic problems. Data are come from mundane and institutional interaction in German from the public corpus FOLK.

A novel audio-visual based beamformer using a 360° camera and planar microphone array system

Beamforming systems that rely only on microphone arrays may suffer from poor performance in complex acoustic environments with multiple sources and room reverberation. To overcome this limitation, we propose a novel method that combines a planar microphone array and a 360° camera to steer the beamformer. The camera provides visual information about the location and orientation of sound sources, which can be used to adjust the beamformer parameters. We will present results that compare the signal-to-noise ratio (SNR) of the proposed method with a conventional beamforming microphone-based method in a simulated room with different source positions, orientations, and noise levels to investigate the benefit of visual information for beamforming in different environments. The proposed method has the potential to improve the performance of beamforming systems in applications such as speech enhancement, speech recognition, and speaker localization.

Study of speaker localization with binaural microphone array incorporating auditory filters and lateral angle estimation

Speaker localization for binaural microphone arrays has been widely studied for applications such as speech communication, video conferencing, and robot audition. Many methods developed for this task, including the direct path dominance (DPD) test, share common stages in their processing, which include transformation using the short-time Fourier transform (STFT), and a direction of arrival (DOA) search that is based on the head related transfer function (HRTF) set. In this paper, alternatives to these processing stages, motivated by human hearing, are proposed. These include incorporating an auditory filter bank to replace the STFT, and a new DOA search based on transformed HRTF as steering vectors. A simulation study and an experimental study are conducted to validate the proposed alternatives, and both are applied to two binaural DOA estimation methods; the results show that the proposed method compares favorably with current methods.

DEVELOPING SPEAKING ENGLISH MODULE AT SURVIVAL CLASS OF LKP ALOHA EDU TEGALDLIMO IN ACADEMIC YEAR 2021/2022

The historical past of the improvement of this book (Speaking Module) media is via way of means of searching of studying media in general. In the Survival Class at LKP Aloha Edu, using media that discusses and learns talking know-how, mainly talking with local speaker. In fact, studying media may be very a good deal had to make it less complicated for instructors to switch know-how and make it less complicated for college students. Therefore, the researcher advanced the Speaking module within side the Survival elegance at LKP Aloha Edu. This examine pursuits to broaden studying media and enhance the talking exceptional of college students within side the LKP Aloha Edu, they continually have issues whilst talking in English, due to the fact they're afraid, language vocabs which might be pretty specific from English which has been standardized via way of means of the global community. This product, that's within side the shape of speaking module, has acknowledged the feasibility of the media in order that it has produced a legitimate and powerful media. The facts series method on this examine became to apply a studies and improvement questionnaire to provide suitable Speaking module primarily based totally on tests from media specialists cloth specialists, layout specialists, English instructors/ tutors and responses from college students.
 This research uses research and development (R&D) methods. The development procedure follows the Borg and Gall procedure which generally can be carried out by involving 10 main steps, namely: 1) research and information collecting, 2) planning data, 3) developing preliminary from product, 4) preliminary field testing, 5) main product revision, 6) main field testing, 7) operational product revision, 8) operational field testing, 9) final product revision, 10) Dimensions and implementation. then for research conducted in the scope of undergraduate, the research is limited to five stages, namely 1) research and information collecting, 2) planning, 3) developing preliminary from product, 4) preliminary field testing, 5) main product revision.
 The results of this assessment indicate that the English vocabulary card media has a very good feasibility according to media experts getting a percentage of 96%, material experts 100%, , teacher assessments 97,75% and student responses 91,23% which means this Learning Media is "Very Appropriate" to use.

A Coherent Wideband Acoustic Source Localization Using a Uniform Circular Array.

In modern applications such as robotics, autonomous vehicles, and speaker localization, the computational power for sound source localization applications can be limited when other functionalities get more complex. In such application fields, there is a need to maintain high localization accuracy for several sound sources while reducing computational complexity. The array manifold interpolation (AMI) method applied with the Multiple Signal Classification (MUSIC) algorithm enables sound source localization of multiple sources with high accuracy. However, the computational complexity has so far been relatively high. This paper presents a modified AMI for uniform circular array (UCA) that offers reduced computational complexity compared to the original AMI. The complexity reduction is based on the proposed UCA-specific focusing matrix which eliminates the calculation of the Bessel function. The simulation comparison is done with the existing methods of iMUSIC, the Weighted Squared Test of Orthogonality of Projected Subspaces (WS-TOPS), and the original AMI. The experiment result under different scenarios shows that the proposed algorithm outperforms the original AMI method in terms of estimation accuracy and up to a 30% reduction in computation time. An advantage offered by this proposed method is the ability to implement wideband array processing on low-end microprocessors.

Multimodal Age and Gender Estimation for Adaptive Human-Robot Interaction: A Systematic Literature Review

Identifying the gender of a person and his age by way of speaking is considered a crucial task in computer vision. It is a very important and active research topic with many areas of application, such as identifying a person, trustworthiness, demographic analysis, safety and health knowledge, visual monitoring, and aging progress. Data matching is to identify the gender of the person and his age. Thus, the study touches on a review of many research papers from 2016 to 2022. At the heart of the topic, many systematic reviews of multimodal pedagogies in Age and Gender Estimation for Adaptive were undertaken. However, no current study of the theme concerns connected to multimodal pedagogies in Age and Gender Estimation for Adaptive Learning has been published. The multimodal pedagogies in four different databases within the keywords indicate the heart of the topic. A qualitative thematic analysis based on 48 articles found during the search revealed four common themes, such as multimodal engagement and speech with the Human-Robot Interaction life world. The study touches on the presentation of many major concepts, namely Age Estimation, Gender Estimation, Speaker Recognition, Speech recognition, Speaker Localization, and Speaker Gender Identification. According to specific criteria, they were presented to all studies. The essay compares these themes to the thematic findings of other review studies on the same topic such as multimodal age, gender estimation, and dataset used. The main objective of this paper is to provide a comprehensive analysis based on the surveyed region. The study provides a platform for professors, researchers, and students alike, and proposes directions for future research.

Audio–Visual Sound Source Localization and Tracking Based on Mobile Robot for The Cocktail Party Problem

Locating the sound source is one of the most important capabilities of robot audition. In recent years, single-source localization techniques have increasingly matured. However, localizing and tracking specific sound sources in multi-source scenarios, which is known as the cocktail party problem, is still unresolved. In order to address this challenge, in this paper, we propose a system for dynamically localizing and tracking sound sources based on audio–visual information that can be deployed on a mobile robot. Our system first locates specific targets using pre-registered voiceprint and face features. Subsequently, the robot moves to track the target while keeping away from other sound sources in the surroundings instructed by the motion module, which helps the robot gather clearer audio data of the target to perform downstream tasks better. Its effectiveness has been verified via extensive real-world experiments with a 20% improvement in the success rate of specific speaker localization and a 14% reduction in word error rate in speech recognition compared to its counterparts.

“Zdes'” is not “Tut”: Some Remarks on the Deixis of Oral Mythological Stories

The article examines the functioning of two spatial deictics “zdes'” and “tut” in the system of mythological story based on the material of the Northern Russian tradition. The analysis made it possible to reveal the composition of the meanings they represent, the quantitative ratio of these units, and the modes of use. The presence of specialization of these deictic units in the oral narrative has been established: “zdes'”, in all spatial meanings, it is used exclusively as a deictic of the speech mode and it is indicating of the speaker's location in the actual situation of speech. The lexeme “tut” is used both in the speech mode and in the narrative mode – as a means images of the situation of the text relative to the character or the observer (this feature is observed only if a specific limited area of space is designated by this word), which allows us to conclude that this lexeme performs the function of a means of narrating and modeling the chronotope of the text, separated from the actual situation of speech.

A time delay estimation approach with low computational complexity for speaker localization

Time delay estimation (TDE), which aims at estimating the time difference of arrival using the signals captured by an array of microphones, plays an essential role in hands-free speech communication systems for localizing and tracking speakers. The sparse linear prediction model, which is based on the l1-norm optimization with respect to the prediction-error vector and the coefficient vector of the linear predictor, can be effectively used to prefilter the microphone signals so as to establish a time delay estimator robust to background noise and reverberation. In the course to solve this model, however, a high-dimension cross-correlation matrix has to be inverted, indicating that the corresponding TDE algorithm has huge computational load. In this paper, we propose a new solution approach to this dual l1-norm optimization model. The original prediction-error filter is decomposed by Kronecker product into two short subpredictors. Accordingly, the size of the cross-correlation matrix in the TDE algorithm is degraded, which significantly reduces the computational complexity for prewhitening microphone signals. Simulation experiments in room acoustic environments demonstrate the computational efficiency of the proposed TDE algorithm as well as its estimation precision.

Dynamic speaker localization based on a novel lightweight R–CNN model

Dynamic speaker localization based on a novel lightweight R–CNN model