Real-time Audio Signal Research Articles

Quasi-passive modulation for monolithic GaN photoelectronic circuit

Due to the overlap between the electroluminescence spectrum and spectral responsivity curve, gallium nitride (GaN)-based multi-quantum well (MQW) diodes can modulate and detect light emitted by another diode with the same MQW structure. This enables the realization of a monolithic integrated GaN optoelectronic circuit, integrating an MQW-based transmitter, waveguide, modulator, and receiver on a tiny GaN chip. It is well known that the active region of MQW absorbs high-energy photons within the plane, generating electron-hole pairs and forming photogenerated carriers. The change in free carrier concentration causes variations in the refractive index and absorption characteristics of the waveguide, thereby manipulating light propagation within the waveguide. Based on this physical phenomenon, a quasi-passive modulation scheme is proposed for a monolithic GaN photoelectronic circuit. This scheme achieves optical modulation by connecting a variable resistor between the modulator electrodes, avoiding the need for high-precision external electric fields and complex control circuits. The performance of the quasi-passive modulation was tested through on-chip data transmission and real-time audio signal transmission. The results indicate that quasi-passive modulation is highly feasible in optoelectronic systems and shows great promise as a competitive core module for future large-scale photonic integrated circuits.

Two-dimensional displacement estimation of one-dimensional laser speckle images for detection of acoustic vibration.

Detection and recovery of audio signals using optical methods is an appealing topic. Observing the movement of secondary speckle patterns is a convenient method for such a purpose. In order to have less computational cost and faster processing, one-dimensional laser speckle images are captured by an imaging device, while it sacrifices the ability to detect speckle movement along one axis. This paper proposes a laser microphone system to estimate the two-dimensional displacement from one-dimensional laser speckle images. Hence, we can regenerate audio signals in real time even as the sound source is rotating. Experimental results show that our system is capable of reconstructing audio signals under complex conditions.

Audio transmission through tunable CCT lighting system: A novel approach of VLC beyond static lighting

In this article a novel approach of visible light communication (VLC) through tunable correlated color temperature (CCT) lighting system has been demonstrated. Till date VLC has been demonstrated through either fixed CCT white LED light source or a single color LED light source. But recent research on photo biological and psychological effect of lighting on human health shows that tunable CCT lighting system is better for human health compared to fixed CCT lighting system. Modern indoor smart lighting system is also adopting the tunable CCT light sources for illumination. Hence, the concept of VLC through static lighting should also be changed. Here, to prove the concept, an audio transmission channel through tunable CCT lighting source has been designed, developed and experimentally validated. A phosphor coated warm white, single colored green and blue (WGB) LED based tunable CCT light source has been developed. The experiments have been performed for the tunable CCT range from 2800 K to 12000 K, maintaining the illuminance 200 lx on the working plane. The real time audio signal has been transmitted through it without affecting the CCT and illuminance. The maximum deviation in CCT is 137 K when set CCT is 12000 K, frequency and amplitude of the audio signal is 100 Hz and 400 mV respectively. The pulse width modulation (PWM) technique has been used for transmitting the audio signal. The communication range is 228.6 cm, which is the general distance between ceiling and the working plane.

ANAUDIO DENOISING APPROACH USING HYBRID MODIFIED FAST WAVELET TRANSFORM METHOD

Noise includes all unwelcome ambient sounds. If a sound is perceived as delightful music or annoying noise, it will be perceived as having the same decibel level. The fundamental drawback of noise in an audio signal is that it degrades the signal's quality during transmission over the communication system. Real-time audio signals are gathered for the present study using a microphone. The additive white Gaussian noise of AWGN is blended with a genuine audio source. The Median, Finite Impulse Response filter with Wavelet Transform approach is a hybrid filter for denoising audio signals that incorporates the Median Filter, FIR Filter, and Wavelet Transform techniques (MFWT). In this study Peak Signal to Noise Ratio and Mean Square Error (MSE) are used to assess MFWT performance(PSNR). In terms of SNR (82.04) and RMSE score, MFWT performs better than the median filter, FIR filter, and Wavelet transform (0.055).

Development of a Real-Time Audio Signal Processing System for Speech Enhancement

The requirements of real-time signal processing dictate that the audio signal must be completely processed before the subsequent audio segment may be received. This is done in order to achieve those requirements. This highlights how important it is to create methods of signal processing that are not just quick but also accurate. I describe many ways for processing audio signals in real time within the scope of this thesis. The publications that are being presented cover a wide range of issues, including noise dosimetry, speech analysis, and network echo cancellation, to name a few. In this article, the process of constructing a system that uses audio signal processing to improve speech in real time is broken down and examined. Speech enhancement makes speech more audible and comprehensible in noisy surroundings, which is beneficial to individuals who have hearing loss as well as speech recognition and communication systems. Signal-to-noise ratio, power spectral efficiency ratio, and spectrum transfer entropy index are the metrics that are utilized to evaluate speech quality enhancement system development strategies, abbreviated as STOI. Because studies have shown that being exposed to loud noises over extended periods of time can have severe impacts on health, it is essential to have precise methods for detecting the levels of noise. The findings of a study that measured exposure to noise while also taking into account the impact of the speaker's own voice are presented in this article.

Keeping Children Safe Online With Limited Resources: Analyzing What is Seen and Heard

It is every parent’s wish to protect their children from online pornography, cyber bullying and cyber predators. Several existing approaches analyze a limited amount of information stemming from the interactions of the child with the corresponding online party. Some restrict access to websites based on a blacklist of known forbidden URLs, others attempt to parse and analyze the exchanged multimedia content between the two parties. However, new URLs can be used to circumvent a blacklist, and images, video, and text can individually appear to be safe, but need to be judged jointly. We propose a highly modular framework of analyzing content in its final form at the user interface, or Human Computer Interaction (HCI) layer, as it appears before the child: on the screen and through the speakers. Our approach is to produce Children’s Agents for Secure and Privacy Enhanced Reaction (CASPER), which analyzes screen captures and audio signals in real time in order to make a decision based on all of the information at its disposal, with limited hardware capabilities. We employ a collection of deep learning techniques for image, audio and text processing in order to categorize visual content as pornographic or neutral, and textual content as cyberbullying or neutral. We additionally contribute a custom dataset that offers a wide spectrum of objectionable content for evaluation and training purposes. CASPER demonstrates an average accuracy of 88% and an F1 score of 0.85 when classifying text, and an accuracy of 95% when classifying pornography.

$$hf_0$$: A Hybrid Pitch Extraction Method for Multimodal Voice

Pitch or fundamental frequency ( $$f_0$$ ) estimation is a fundamental problem extensively studied for its potential speech and clinical applications. The existing $$f_0$$ estimation methods degrade in performance when applied over real-time audio signals with varying $$f_0$$ modulations and high SNR environment. In this work, a $$f_0$$ estimation method using both signal processing and deep learning approaches is developed. Specifically, we train a convolutional neural network to map the periodicity-rich input representation to pitch classes, such that the number of pitch classes is drastically reduced compared to existing deep learning approaches. Then, the accurate $$f_0$$ is estimated from the nominal pitch classes based on signal processing approaches. The observations from the experimental results showed that the proposed method generalizes to unseen modulations of speech and noisy signals (with various types of noise) for large-scale datasets. Also, the proposed hybrid model significantly reduces the learning parameters required to train the model compared to other methods. Furthermore, the evaluation measures showed that the proposed method performs significantly better than the state-of-the-art signal processing and deep learning approaches.

An Adaptive Time Delay Estimation Method for Broadcast Audio Based on Power Cepstrum

Abstract This paper proposes an adaptive time delay estimation method for broadcast audio based on power cepstrum. First, it adds two real-time broadcast audio signals with the same content but with a time delay to obtain a mixed signal, then segment the mixed signal, and obtain a power cepstrum for each mixed signal. Afterwards, the time delay of the two audio signals is determined by using the feature that the power cepstrum of the mixed signal can peak at the time delay. The experimental results show that this method can obtain more accurate time delay results. And it has higher robustness and faster than traditional methods such as generalized cross-correlation time delay estimation algorithm.

Data-driven smart manufacturing: Tool wear monitoring with audio signals and machine learning

Tool wear in machining could result in poor surface finish, excessive vibration and energy consumption. Monitoring tool wear in real-time is crucial to improve manufacturing productivity and quality. While numerous sensor-based tool wear monitoring techniques have been demonstrated in laboratory environments, few tool wear monitoring systems have been deployed in factories because it is not realistic to install some of the important sensors such as dynamometers on manufacturing machines. To address this issue, a novel audio signal processing approach is introduced. This technique does not require expensive sensors but audio sensors only. A blind source separation method is used to separate source signals from noise. An extended principal component analysis is used for dimensionality reduction. Real-time multi-channel audio signals are collected during a set of milling tests under varying cutting conditions. The experimental data are used to develop and validate a predictive model. Experimental results have shown that the predictive model is capable of classifying tool wear conditions with high accuracy.

Real-time audio signal processing using system-on-chip field programmable gate arrays

System-on-Chip (SoC) Field Programmable Gate Arrays (FPGAs) are ideal for real-time signal processing due to their low, deterministic latency and high performance. To showcase the utility of our open FPGA computational platform for real-time audio signal processing and computational modeling, several applications have been implemented. We have ported the openMHA hearing aid software [1] to our platform to show that pre-existing audio processing software can be implemented in SoC FPGAs by making external audio interfaces show up as a sound card. To highlight the ability to perform real-time computational modeling on our performance platform, we are implementing a real-time version of Laurel Carney’s auditory-nerve model [2] running in its own custom accelerator in the FPGA fabric. To illustrate the ability to develop DSP algorithms in MathWork’s Simulink and then implement them in the FPGA fabric we have taken several algorithms from Issa Panahi’s group [3] to show both frame-based processing (noise reduction) and sample-based processing (dynamic range compression). Finally, we show that the platform can be used to visualize audio signals using a real-time spectrogram where FFTs are computed in the FPGA fabric. [1] www.openmha.org. [2] JASA 126, 2390–2412. [3] www.utdallas.edu/ssprl/hearing-aid-project/.

Bearing failure prediction using audio signal analysis based on SVM algorithms

Bearings are machine elements used in a wide variety of applications including transportation. The accurate prediction of a bearing failure is important to sensitive applications to secure its safety during the service life. Bearing failure prediction is useful both in bearing testing phase as well as in case of lifetime use. Real-time Audio signal analysis and advanced algorithms are able to identify the incipient failure, caused by defects, fatigue, overload or poor maintenance. Audio signal analysis and processing remains a domain where technique and algorithm needs to be developed. In this paper is presented a proof-of-concept technique and equipment developed to predict failure of bearings in case of testing phase. For this study, acoustic emission signals were measured and analyzed during life testing of bearing while other sound source are also recorded. Correlation between the acoustic emission patterns were identified in order to identify noise signal and identify the signal associated with bearing degradation. The developed solution to isolate other sounds signals means that the technique could be used while lifetime of the bearings. The results of this study provide evidence that accurate estimation of the failure of various bearings is possible by processing the vibration signal acquired from a single point, even in case of multiple sound sources are present and introduce noise in signal processing. The SVM classifier provides at least 92% mean accuracy. The influence of model on prediction accuracy has also been discussed in the work.

Audio Effect Units in Mobile Devices for Electric Musical Instruments

With the advent of modern techniques, there has been an increase in mobile devices with powerful functions. This study aims to utilize application software instead of hardware equipment to alter the sound of electric musical instruments by developing the functions of audio effect units on a mobile device. The built-in software audio effects include dynamic effects, delay effects, mixing, and equalization. Multichannel audio signal processing is proposed to enhance sound externalization. The listening tests indicate that the sound effects together with a quadraphonic system produce superior special effects and spatial audio than conventional effect units. Additionally, a denoiser is proposed for noise reduction, which is a function especially helpful for singers performing on busy streets. The customization in the module is achieved by a personal voice preprocessing system. A dual-filter function was utilized for the denoiser to adjust to different environments. The objective performance measurements demonstrate that the proposed denoiser outperforms the state-of-the-art methods. A hardware audio interface serving as the connection between a mobile device and an electric instrument, such as an electric guitar, electric piano, or electric bass, was also built, providing impedance matching and voltage balance. The proposed interface transfers the electric instrument signal into a 3.5-mm jack in a smartphone or tablet. The developed audio interface is light and low-noise and can operate without being connected to an external power supply, thereby making it suitable for street musicians. The experiments validate the feasibility of using the proposed circuit for real-time audio signal conversion.

Stereo I/O framework for audio signal processing on android platforms

Smartphones and tablets are attractive cost-effective solutions for implementing multi-channel audio signal processing algorithms. Modern smartphones and tablets have at least two microphones and requisite processing capabilities to perform real-time audio signal processing. However, it is still challenging to integrate the dual-microphone audio capture and the audio processing pipeline in real-time for efficient operation. In this paper, we propose an efficient stereo input/output framework for real-time audio signal processing on Android smartphones/tablets. This framework enables us to perform a large variety of real-time signal-processing applications on Android platform such as Direction of Arrival (DOA) estimation, Audio Compression, and Multi-Channel Speech Enhancement. The implementations and demos of Direction of Arrival (DOA), Audio Compression, and Multi-Channel Speech Enhancement algorithms are also presented to show the effectiveness of proposed framework.

An open computational platform for low-latency real-time audio signal processing using field programmable gate arrays

Field Programmable Gate Arrays (FPGAs) provide flexible computational architectures that are ideal for digital signal processing (DSP). With support of a NIH/NIDCD SBIR grant, we are developing an open FPGA platform for the speech, hearing, and acoustics research communities. The advantage of using FPGAs in a computational platform over conventional CPU approaches is the ability to implement low-latency high-performance signal processing with deterministic latencies. The hardware portion of the platform includes an audio codec and an Intel System-on-Chip (SoC) FPGA that contains ARM CPUs alongside the computational fabric that allows custom data plane designs. Development uses Mathwork’s Simulink that allows exploration and simulation of signal processing algorithms. Once a Simulink model has been developed to implement audio processing in either the time domain or frequency domain, VHDL code can be generated that implements the desired signal processing. The VHDL code is then implemented in the FPGA computational fabric where the FPGA functions as a real-time signal processor. We are currently soliciting ideas/feedback from the speech, hearing, and acoustic communities as to what features they would like to see in the next iteration of the open FPGA-based computational platform. [NIH/NIDCD R44DC015443: www.openspeechtools.com]

Open community platform for hearing aid algorithm research

The project “Open community platform for hearing aid algorithm research” funded by the National Institutes of Health (NIH Grant 1R01DC015429-01) aims at sustainable, focused research toward improvement and new types of assistive hearing systems. To this end, an open-source software platform for real-time audio signal processing will be developed and made available to the research community including a standard set of reference algorithms. Furthermore, novel algorithms for dynamic and frequency compression, auditory-scene-analysis based noise suppression and speech enhancement, and feedback management will be investigated. For a realistic assessment of the benefits of hearing aid algorithms and combinations thereof, instrumental measures of performance in virtual acoustic environments of varying complexity will be included in the algorithm design and optimization. With such a quasi-standard set of benchmarks and the means to develop and integrate own signal-processing methods and measures in the same framework, the platform enables reproducible, comparative studies and collaborative research efforts. Beyond an implementation for PC hardware the system will also be made usable for ARM-processor based hardware to allow pre-development of wearable audio devices—so-called “hearables.” This contribution will present underlying previous work and the goals and plans of the project that has started midyear 2016. www.openMHA.org.

Developing an audio network system for teleconferencing with evaluation of audio quality

A teleconference system was developed to transmit speech signals for audio communication with remote users. The hardware system was composed of 3-D microphone arrays to capture directional sound and a wireless headset to provide freedom of movement for users. An application programming interface (API) was used for interaction between the computer and the audio devices, including the microphone, headphone, and ADDA converter, transferring input/output audio signals in real time through the network. For the evaluation of the system, a latency test was performed with a simultaneous recording system in capturing and reproduction positions. The latency from the network procedure was calculated with different buffer sizes, and the lower-delay and smaller-deviation buffer was selected. In addition, the transmitted audio signals were compared with the input signals in terms of signal patterns and frequency responses. The resulting audio qualities in the time and frequency domain were suitable for a teleconferencing system.

Toolkits for Real Time Digital Audio Signal Processing Teaching Laboratory

This paper describes an audio digital signal-processing toolkit that the authors develop to supplement a lecture course on digital signal processing (DSP) taught at the department of Electrical and Electronics Engineering at the University of Rwanda. In engineering education, laboratory work is a very important component for a holistic learning experience. However, even though today there is an increasing availability of programmable DSP hardware that students can largely benefit from, many poorly endowed universities cannot afford a costly full-fledged DSP laboratory. To help remedy this problem, the authors have developed C#.NET toolkits, which can be used for real-time digital audio signal processing laboratory. These toolkits can be used with any managed languages, like Visual Basic, C#, F# and managed C++. They provide frequently used modules for digital audio processing such as filtering, equalization, spectrum analysis, audio playback, and sound effects. It is anticipated that by creating a flexible and reusable components, students will not only learn fundamentals of DSP but also get an insight into the practicability of what they have learned in the classroom.

UNIVERSAL EMBEDDED RECONFIGURABLE HARDWARE PLATFORM FOR MULTIMEDIA APPLICATIONS IN REAL-TIME

This paper deals with reconfigurable hardware platform for different purposes real-time speech and audio signal processing. A design conception and turnkey solution are described. Much attention is paid to reconfigurable peripheral processor meant for external interface realization, pre- and post- data processing as well as digital signal processing algorithms implementation with the object of the DSP unloads. Moreover, three applications implemented on the considered platform are demonstrated.

Real-time audio signal capture and processing using matlab object oriented programming

In MATLAB programming language the real-time audio processing functions are usually simulated in non-real-time due to a lack of real-time audio programming support. As a result the real-time audio signal capture and processing functionalities are usually implemented in other programming languages and cannot utilize the extensive signal processing functionalities provided by MATLAB. In this paper we introduce a MATLAB real-time signal processing framework based on MATLAB timer object and audiorecorder object. The proposed processing framework serves as an alternative solution for real-time programming implementation and demonstration. In our proposed processing framework the timer object is implemented to handle the looping of processing cycle, schedule the signal processing tasks, and handle the error processing. The audio capturing/processing functionality is implemented in the timer cycle by using two audiorecorder objects that read the audio streaming data and feed a segment of data to signal processing alternatively. The proposed framework achieves satisfactory real-time performance with no missing audio frames when a short audio delay setting of 10ms is applied. Several application examples of our proposed framework are also demonstrated.

Performance Following: Real-Time Prediction of Musical Sequences Without a Score

This paper introduces a technique for predicting harmonic sequences in a musical performance for which no score is available, using real-time audio signals. Recent short-term information is aligned with longer term information, contextualizing the present within the past, allowing predictions about the future of the performance to be made. Using a mid-level representation in the form of beat-synchronous harmonic sequences, we reduce the size of the information needed to represent the performance. This allows the implementation of real-time performance following in live performance situations. We conduct an objective evaluation on a database of rock, pop, and folk music. Our results show that we are able to predict a large majority of repeated harmonic content with no prior knowledge in the form of a score.