Audio Processing Applications Research Articles

Study on the optimization process and application of FIR digital filter

As one of the components of digital signal processing technology, the FIR digital filter has been widely used in real life and played an increasingly important role due to its advantages of system stability and ease of linear phase. This paper will summarize and analyze the research results of some past scientific research teams on FIR digital filters, and show the optimization process of this technology and the application examples in real life. This paper focuses on the optimization process of noise reduction performance, delay and size, and energy consumption optimization of FIR digital filters. This study summarizes three examples of its applications in medical, audio processing, and radar. In the medical field, the processing of electrocardiogram (ECG) signals, the audio processing fields are the frequency shaping, noise removal, and signal enhancement of audio signals, and the precise filtering and processing of received signals in the radar field. At the end of this paper, the FIR digital filter has an irreplaceable role in all fields in the world and gradually develops in the direction of miniaturization and low power consumption in the future.

Deep learning restores speech intelligibility in multi-talker interference for cochlear implant users

Cochlear implants (CIs) do not offer the same level of effectiveness in noisy environments as in quiet settings. Current single-microphone noise reduction algorithms in hearing aids and CIs only remove predictable, stationary noise, and are ineffective against realistic, non-stationary noise such as multi-talker interference. Recent developments in deep neural network (DNN) algorithms have achieved noteworthy performance in speech enhancement and separation, especially in removing speech noise. However, more work is needed to investigate the potential of DNN algorithms in removing speech noise when tested with listeners fitted with CIs. Here, we implemented two DNN algorithms that are well suited for applications in speech audio processing: (1) recurrent neural network (RNN) and (2) SepFormer. The algorithms were trained with a customized dataset (∼\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\sim$$\\end{document} 30 h), and then tested with thirteen CI listeners. Both RNN and SepFormer algorithms significantly improved CI listener’s speech intelligibility in noise without compromising the perceived quality of speech overall. These algorithms not only increased the intelligibility in stationary non-speech noise, but also introduced a substantial improvement in non-stationary noise, where conventional signal processing strategies fall short with little benefits. These results show the promise of using DNN algorithms as a solution for listening challenges in multi-talker noise interference.

Advancements in Filter Technology: Evolution and Applications

This paper explores the historical trajectory and contemporary applications of signal processing filters. From the early days of analog filters to the current digital era, the paper traces their evolution, emphasizing key developments in active RC filters, monolithic integrated op amps, and the transition to switched-capacitor and continuous-time filters. Fundamentals of analog and digital filters are outlined, distinguishing their processing of continuous-time and discrete-time signals. The classification of filters, including low-pass, high-pass, band-pass, and band-stop, is discussed with insights into their specific applications. The paper delves into crucial applications in audio processing, video processing, communication systems, power management, and sensor signal processing, showcasing filters' pivotal role in enhancing signal quality across diverse fields. Addressing emerging technologies, the paper highlights adaptive and deep learning filters, projecting the future trends in filter technology. It anticipates developments in analog integrated filters, emphasizing the growing significance of integrated circuits technology. This comprehensive review aims to deepen readers' understanding and inspire continued innovation in signal processing.

Development of a high-performance arithmetic and logic unit for efficient digital signal processing based on reversible logic and quantum dots

Digital Signal Processing (DSP) finds a wide range of applications in various fields, including telecommunications, audio and video processing, biomedical engineering, radar systems, and image processing. Previous DSP designs faced limitations in available processing power and computational resources. Insufficient processing power could result in slower execution times, an inability to handle complex algorithms, or limited capacity to process high-speed or large-scale signals. As the demand for minimal power consumption in DSP circuits continues to grow, reversible logic and quantum-dot cellular automata (QCA) have emerged as promising technologies due to their inherent ability to reduce energy loss. Within this landscape, the arithmetic and logic unit (ALU) plays a vital role in complex circuitry, serving as a key component in digital signal processing applications. However, challenges persist, including high quantum cost and the need to limit the number of cells in the ALU design. To address these challenges, our research aims to develop an efficient ALU by integrating reversible logic and QCA technology. Our focus will be on generating essential components, such as Feynman gates, Fredkin gates, and full adder circuits, which serve as foundational building blocks for reversible logic and QCA designs. These components will be combined to construct a comprehensive ALU capable of performing 20 different operations. Our implementation efforts will be centered around QCADesigner, with a specific emphasis on digital signal processing systems that prioritize energy efficiency and optimal utilization of occupied areas.

DAEM: A Data- and Application-Aware Error Analysis Methodology for Approximate Adders

Approximate adders are some of the fundamental arithmetic operators that are being employed in error-resilient applications, to achieve performance/energy/area gains. This improvement usually comes at the cost of some accuracy and, therefore, requires prior error analysis, to select an approximate adder variant that provides acceptable accuracy. Most of the state-of-the-art error analysis techniques for approximate adders assume input bits and operands to be independent of one another, while some also assume the operands to be uniformly distributed. In this paper, we analyze the impact of these assumptions on the accuracy of error estimation techniques, and we highlight the need to address these assumptions, to achieve better and more realistic quality estimates. Based on our analysis, we propose DAEM, a data- and application-aware error analysis methodology for approximate adders. Unlike existing error analysis models, we neither assume the adder operands to be uniformly distributed nor assume them to be independent. Specifically, we use 2D joint input probability mass functions (PMFs), populated using sample data, in order to incorporate the data and application knowledge in the analysis. These 2D joint input PMFs, along with 2D error maps of approximate adders, are used to estimate the error PMF of an adder network. The error PMF is then utilized to compute different error measures, such as the mean squared error (MSE) and mean error distance (MED). We evaluate the proposed error analysis methodology on audio and video processing applications, and we demonstrate that our methodology provides error estimates having a better correlation with the simulation results, as compared to the state-of-the-art techniques.

Deep learning methods and integrated digital image processing techniques for detecting and evaluating wheat stripe rust disease

In recent years, signal processing and deep learning convergence has sparked transformative synergies across various domains, including image and speech recognition, natural language processing, autonomous systems, and healthcare diagnostics. This fusion capitalizes on the strength of signal processing in extracting meaningful features from raw data and the prowess of deep learning in unraveling intricate patterns, driving innovation and research into uncharted territories. This paper explores literature spanning the past three years to illuminate the dynamic landscape of scholarly endeavors that leverage the integration of signal processing techniques within deep learning architectures. The resulting paradigm shift magnifies the precision and efficiency of applications in computer vision, speech and audio processing, natural language comprehension, and interdisciplinary domains like healthcare. Notable advances include synergizing wavelet transformations with convolutional neural networks (CNNs) for enhanced image classification accuracy, integrating spectrogram-based features with deep learning architectures for improved speech-to-text accuracy, and pioneering the fusion of wavelet packet decomposition into recurrent architectures for sentiment analysis. Moreover, the paper delves into developing and evaluating a U-Net neural network model for image segmentation, investigating its performance under varying training conditions using metrics such as confusion matrices, heat maps, and precision-recall curves. The comprehensive survey identifies research gaps, notably within the context of wheat rust detection, and emphasizes the need for tailored innovations to enhance accuracy and efficiency. Overall, the synthesis of signal processing techniques with deep learning architectures propels innovation, poised to address complex challenges across diverse domains

Characterization of Deep Learning-Based Speech-Enhancement Techniques in Online Audio Processing Applications.

Deep learning-based speech-enhancement techniques have recently been an area of growing interest, since their impressive performance can potentially benefit a wide variety of digital voice communication systems. However, such performance has been evaluated mostly in offline audio-processing scenarios (i.e., feeding the model, in one go, a complete audio recording, which may extend several seconds). It is of significant interest to evaluate and characterize the current state-of-the-art in applications that process audio online (i.e., feeding the model a sequence of segments of audio data, concatenating the results at the output end). Although evaluations and comparisons between speech-enhancement techniques have been carried out before, as far as the author knows, the work presented here is the first that evaluates the performance of such techniques in relation to their online applicability. This means that this work measures how the output signal-to-interference ratio (as a separation metric), the response time, and memory usage (as online metrics) are impacted by the input length (the size of audio segments), in addition to the amount of noise, amount and number of interferences, and amount of reverberation. Three popular models were evaluated, given their availability on public repositories and online viability, MetricGAN+, Spectral Feature Mapping with Mimic Loss, and Demucs-Denoiser. The characterization was carried out using a systematic evaluation protocol based on the Speechbrain framework. Several intuitions are presented and discussed, and some recommendations for future work are proposed.

Design of Generalized Enhanced Static Segment Multiplier with Minimum Mean Square Error for Uniform and Nonuniform Input Distributions

In this paper, we analyze the performances of an Enhanced Static Segment Multiplier (ESSM) when the inputs have both uniform and non-uniform distribution. The enhanced segmentation divides the multiplicands into a lower, a middle, and an upper segment. While the middle segment is placed at the center of the inputs in other implementations, we seek the optimal position able to minimize the approximation error. To this aim, two design parameters are exploited: m, defining the size and the accuracy of the multiplier, and q, defining the position of the middle segment for further accuracy tuning. A hardware implementation is proposed for our generalized ESSM (gESSM), and an analytical model is described, able to find m and q which minimize the mean square approximation error. With uniform inputs, the error slightly improves by increasing q, whereas a large error decrease is observed by properly choosing q when the inputs are half-normal (with a NoEB up to 18.5 bits for a 16-bit multiplier). Implementation results in 28 nm CMOS technology are also satisfactory, with area and power reductions up to 71% and 83%. We report image and audio processing applications, showing that gESSM is a suitable candidate in applications with non-uniform inputs.

A Survey of Optimization Methods for Independent Vector Analysis in Audio Source Separation

With the advent of the era of big data information, artificial intelligence (AI) methods have become extremely promising and attractive. It has become extremely important to extract useful signals by decomposing various mixed signals through blind source separation (BSS). BSS has been proven to have prominent applications in multichannel audio processing. For multichannel speech signals, independent component analysis (ICA) requires a certain statistical independence of source signals and other conditions to allow blind separation. independent vector analysis (IVA) is an extension of ICA for the simultaneous separation of multiple parallel mixed signals. IVA solves the problem of arrangement ambiguity caused by independent component analysis by exploiting the dependencies between source signal components and plays a crucial role in dealing with the problem of convolutional blind signal separation. So far, many researchers have made great contributions to the improvement of this algorithm by adopting different methods to optimize the update rules of the algorithm, accelerate the convergence speed of the algorithm, enhance the separation performance of the algorithm, and adapt to different application scenarios. This meaningful and attractive research work prompted us to conduct a comprehensive survey of this field. This paper briefly reviews the basic principles of the BSS problem, ICA, and IVA and focuses on the existing IVA-based optimization update rule techniques. Additionally, the experimental results show that the AuxIVA-IPA method has the best performance in the deterministic environment, followed by AuxIVA-IP2, and the OverIVA-IP2 has the best performance in the overdetermined environment. The performance of the IVA-NG method is not very optimistic in all environments.

Frequency, Time, Representation and Modeling Aspects for Major Speech and Audio Processing Applications.

There are many speech and audio processing applications and their number is growing. They may cover a wide range of tasks, each having different requirements on the processed speech or audio signals and, therefore, indirectly, on the audio sensors as well. This article reports on tests and evaluation of the effect of basic physical properties of speech and audio signals on the recognition accuracy of major speech/audio processing applications, i.e., speech recognition, speaker recognition, speech emotion recognition, and audio event recognition. A particular focus is on frequency ranges, time intervals, a precision of representation (quantization), and complexities of models suitable for each class of applications. Using domain-specific datasets, eligible feature extraction methods and complex neural network models, it was possible to test and evaluate the effect of basic speech and audio signal properties on the achieved accuracies for each group of applications. The tests confirmed that the basic parameters do affect the overall performance and, moreover, this effect is domain-dependent. Therefore, accurate knowledge of the extent of these effects can be valuable for system designers when selecting appropriate hardware, sensors, architecture, and software for a particular application, especially in the case of limited resources.

Memristor based spiking neural network accelerator architecture

Spiking neural network (SNN) as the third-generation artificial neural network, has higher computational efficiency, lower resource overhead and higher biological rationality. It shows greater potential applications in audio and image processing. With the traditional method, the adder is used to add the membrane potential, which has low efficiency, high resource overhead and low level of integration. In this work, we propose a spiking neural network inference accelerator with higher integration and computational efficiency. Resistive random access memory (RRAM or memristor) is an emerging storage technology, in which resistance varies with voltage. It can be used to build a crossbar architecture to simulate matrix computing, and it has been widely used in processing in memory (PIM), neural network computing, and other fields. In this work, we design a weight storage matrix and peripheral circuit to simulate the leaky integrate and fire (LIF) neuron based on the memristor array. And we propose an SNN hardware inference accelerator, which integrates 24k neurons and 192M synapses with 0.75k memristor. We deploy a three-layer fully connected network on the accelerator and use it to execute the inference task of the MNIST dataset. The result shows that the accelerator can achieve 148.2 frames/s and 96.4% accuracy at a frequency of 50 MHz.

Machine-Learning-Based Self-Tunable Design of Approximate Computing

Approximate computing (AC) is an emerging computing paradigm suitable for intrinsic error-tolerant applications to reduce energy consumption and execution time. Different approximate techniques and designs, at both hardware and software levels, have been proposed and demonstrated the effectiveness of relaxing the average output quality constraint. However, the output quality of AC is highly input-dependent, i.e., for some input data, the output errors may reach unacceptable levels. Therefore, there is a dire need for an input-dependent tunable approximate design. With this motivation, in this article, we propose a lightweight and efficient machine-learning-based approach to build an input-aware design selector, i.e., quality controller, to adapt the approximate design in order to meet the target output quality (TOQ). For illustration purposes, we use a library of 8-bit and 16-bit energy-efficient approximate array multipliers with 20 different settings, which are commonly used in image and audio processing applications. The simulation results, based on two sets of images, including an 8 Scene Categories Dataset, which is a benchmark of images data set, demonstrate the effectiveness of the lightweight selector where the proposed tunable design achieves a significant reduction in quality loss with relatively low overhead.

Time-Frequency Phase Retrieval for Audio—The Effect of Transform Parameters

In audio processing applications, phase retrieval (PR) is often performed from the magnitude of short-time Fourier transform (STFT) coefficients. Although PR performance has been observed to depend on the considered STFT parameters and audio data, the extent of this dependence has not been systematically evaluated yet. To address this, we studied the performance of three PR algorithms for various types of audio content and various STFT parameters such as redundancy, time-frequency ratio, and the type of window. The quality of PR was studied in terms of objective difference grade and signal-to-noise ratio of the STFT magnitude, to provide auditory- and signal-based quality assessments. Our results show that PR quality improved with increasing redundancy, with a strong relevance of the time-frequency ratio. The effect of the audio content was smaller but still observable. The effect of the window was only significant for one of the PR algorithms. Interestingly, for a good PR quality, each of the three algorithms required a different set of parameters, demonstrating the relevance of individual parameter sets for a fair comparison across PR algorithms. Based on these results, we developed guidelines for optimizing STFT parameters for a given application.

Array configuration optimization of first-order steerable differential arrays with minimum number of microphones.

The first-order steerable differential arrays (FOSDAs), which have found a variety of applications in speech and audio processing, are usually designed by construction of two orthogonal dipoles using four microphones. Actually, however, three microphones are enough to construct a FOSDA by forming two dipoles with a shared microphone, which is the scheme with the minimum number of microphones and, hence, the most cost-effective. This paper studies the design and analysis of such three-element FOSDAs by using a least-squares method. In particular, the effect of the array configuration on the three-element FOSDA is studied, and optimum array configurations under different steering angle ranges are derived in terms of both beampattern fitting quality and white noise gain. It also reveals that care should be taken to avoid a conservative setting of the steering angle range in order to achieve superior design of the three-element FOSDA.

Dirichlet Latent Variable Model: A Dynamic Model Based on Dirichlet Prior for Audio Processing

We propose a dynamic latent variable model for learning latent bases from time varying, non-negative data. We take a probabilistic approach to modeling the temporal dependence in data by introducing a dynamic Dirichlet prior—a Dirichlet distribution with dynamic parameters. This new distribution allows us to assure non-negativity and avoid intractability when sequential updates are performed otherwise encountered in using Dirichlet prior. We refer to the proposed model as the Dirichlet latent variable model DLVM. We develop an expectation maximization algorithm for the proposed model, and also derive a maximum a posteriori estimate of the parameters. Furthermore, we connect the proposed DLVM to two popular latent basis learning methods—probabilistic latent component analysis PLCA and non-negative matrix factorization NMF. We show that 1 PLCA is a special case of our DLVM, and 2 DLVM can be interpreted as a dynamic version of NMF. The usefulness of DLVM is demonstrated for three audio processing applications—speaker source separation, denoising, and bandwidth expansion. To this end, a new algorithm for source separation is also proposed. Through extensive experiments on benchmark databases, we show that the proposed model outperforms several relevant existing methods in all three applications.

Energy-efficient low-latency audio on android

Counting more than two billion devices, Android is nowadays one of the most popular open-source general-purpose operating systems, based on Linux. Because of the diversity of applications that can be installed, it manages a number of different workloads, many of them requiring performance/QoS guarantees. When running audio processing applications, the user would like an uninterrupted, glitch-free, output stream that reacts to the user input, typically with a delay not bigger than 4−10 ms, while keeping the energy consumption of the mobile device as low as possible.This work focuses on improvements to the real-time audio processing performance on Android. Such improvements are achieved by using a deadline based scheduler and an adaptive scheduling strategy that dynamically and proactively modulates the allocated runtime. The proposed strategy is evaluated through an extensive set of experiments, showing that (1) compared to the existing way to ensure low-latency audio processing, the proposed mechanism provides an energy saving of almost 40%, and (2) compared to the existing way to achieve a good balance between power consumption and latency in a glitch-free audio processing experience, the proposed solution reduces audio latency from 26.67 ms to 2.67 ms, at the expense of a limited power consumption increase of 6.25%.

Sparse coding based classifier ensembles in supervised and active learning scenarios for data classification

Sparse coding and dictionary learning has recently gained great interest in signal, image and audio processing applications through representing each problem instance by a sparse set of atoms. This also allows us to obtain different representations of feature sets in machine learning problems. Thus, different feature views for classifier ensembles can be obtained using sparse coding. On the other hand, nowadays unlabelled data is abundant and active learning methods with single and classifier ensembles received great interest. In this study, Random Subspace Dictionary Learning (RDL) and Bagging Dictionary Learning (BDL) algorithms are examined by learning ensembles of dictionaries through feature/instance subspaces. Besides, ensembles of dictionaries are evaluated under active learning framework as promising models and they are named as Active Random Subspace Dictionary Learning (ARDL) and Active Bagging Dictionary Learning (ABDL) algorithms. Active learning methods are compared with their Support Vector Machines counterparts. The experiments on eleven datasets from UCI and OpenML repositories has shown that selecting instance and feature subspaces for dictionary learning model increases the number of correctly classified instances for the most of the data sets while SVM has superiority over all of the applied models. Furthermore, using an active learner generally increases the chance of improved classification performance as the number of iterations is increased.

Phase retrieval for wavelet transforms

This paper describes a new algorithm that solves a particular phase retrieval problem, with important applications in audio processing: the reconstruction of a function from its scalogram, that is, from the modulus of its wavelet transform. It is a multiscale iterative algorithm that reconstructs the signal from low-to-high frequencies. It relies on a new reformulation of the phase retrieval problem that involves the holomorphic extension of the wavelet transform. This reformulation allows to propagate phase information from low-to-high frequencies. Numerical results, on audio and non-audio signals, show that reconstruction is precise and stable to noise. The complexity of the algorithm is linear in the size of the signal, up to logarithmic factors. It can thus be applied to large signals.

Conditions on optimal support recovery in unmixing problems by means of multi-penalty regularization

Inspired by several real-life applications in audio processing and medical image analysis, where the quantity of interest is generated by several sources to be accurately modeled and separated, as well as by recent advances in regularization theory and optimization, we study the conditions on optimal support recovery in inverse problems of unmixing type by means of multi-penalty regularization. We consider and analyze a regularization functional composed of a data-fidelity term, where signal and noise are additively mixed, a non-smooth, convex, sparsity promoting term, and a quadratic penalty term to model the noise. We prove not only that the well-established theory for sparse recovery in the single parameter case can be translated to the multi-penalty settings, but we also demonstrate the enhanced properties of multi-penalty regularization in terms of support identification compared to sole -minimization. We additionally confirm and support the theoretical results by extensive numerical simulations, which give a statistics of robustness of the multi-penalty regularization scheme with respect to the single-parameter counterpart. Eventually, we confirm a significant improvement in performance compared to standard -regularization for compressive sensing problems considered in our experiments.

Detection and Classification of OFDM Waveforms Using Cepstral Analysis

Cepstral analysis has been widely used in audio and speech processing applications because of its ability to reveal periodicities in a signal. The presence of cyclic prefix (CP) in orthogonal frequency division multiplexing (OFDM) signals induces periodicities. Motivated by this, the paper focuses on cepstral analysis of OFDM signal. The distributions of cepstral coefficients are derived for two scenarios of noise only and OFDM signal in noise. It is shown that the OFDM cepstrum is significantly different from the additive white Gaussian noise (AWGN) cepstrum and can be used to detect OFDM waveforms. It is also shown that the cepstrum of OFDM is rich in features and can be used to estimate OFDM parameters such as number of subcarriers and length of the CP in an OFDM symbol. These OFDM waveform parameters can be used to automatically recognize or classify different OFDM waveforms, which are important for cognitive radios, coexistence of heterogeneous networks and signal intelligence. Two schemes are proposed to detect OFDM based primary user (PU) signals in cognitive radio systems. The distributions of the test statistics under the two hypotheses are established. Neyman-Pearson detection strategy is employed. Algorithms for estimating the number of subcarriers and the length of the CP are proposed and their performances studied through simulations. Later the proposed schemes are extended to cooperative sensing scenario with multiple secondary users (SUs) and it is shown that the collaboration between them significantly improve the performance of the proposed cepstrum based detection and estimation schemes.