The wavelet transform-domain LMS adaptive filter employing dynamic selection of subband-coefficients

Adaptive filters play an important role in the fields related to digital signal processing and communication, such as system identification, noise cancellation, channel equalization, and beamforming. In practical applications, the computational complexity of an adaptive filter is an important consideration. The Least Mean Square (LMS) algorithm is widely used because of its low computational complexity ($O(N)$) and simplicity in implementation. The least squares algorithms, such as Recursive Least Squares (RLS), Conjugate Gradient (CG), and Euclidean Direction Search (EDS), can converge faster and have lower steady-state mean square error (MSE) than LMS. However, their high computational complexity ($O(N^2)$) makes them unsuitable for many real-time applications. A well-known approach to controlling computational complexity is applying partial update (PU) method to adaptive filters. A partial update method can reduce the adaptive algorithm complexity by updating part of the weight vector instead of the entire vector or by updating part of the time. In the literature, there are only a few analyses of these partial update adaptive filter algorithms. Most analyses are based on partial update LMS and its variants. Only a few papers have addressed partial update RLS and Affine Projection (AP). Therefore, analyses for PU least-squares adaptive filter algorithms are necessary and meaningful. This monograph mostly focuses on the analyses of the partial update least-squares adaptive filter algorithms. Basic partial update methods are applied to adaptive filter algorithms including Least Squares CMA (LSCMA), EDS, and CG. The PU methods are also applied to CMA1-2 and NCMA to compare with the performance of the LSCMA. Mathematical derivation and performance analysis are provided including convergence condition, steady-state mean and mean-square performance for a time-invariant system. The steady-state mean and mean-square performance are also presented for a time-varying system. Computational complexity is calculated for each adaptive filter algorithm. Numerical examples are shown to compare the computational complexity of the PU adaptive filters with the full-update filters. Computer simulation examples, including system identification and channel equalization, are used to demonstrate the mathematical analysis and show the performance of PU adaptive filter algorithms. They also show the convergence performance of PU adaptive filters. The performance is compared between the original adaptive filter algorithms and different partial-update methods. The performance is also compared among similar PU least-squares adaptive filter algorithms, such as PU RLS, PU CG, and PU EDS. In addition to the generic applications of system identification and channel equalization, two special applications of using partial update adaptive filters are also presented. One application uses PU adaptive filters to detect Global System for Mobile Communication (GSM) signals in a local GSM system using the Open Base Transceiver Station (OpenBTS) and Asterisk Private Branch Exchange (PBX). The other application uses PU adaptive filters to do image compression in a system combining hyperspectral image compression and classification.

ᅟIn this paper, the signed regressor normalized subband adaptive filter (SR-NSAF) algorithm is proposed. This algorithm is optimized by L1-norm minimization criteria. The SR-NSAF has a fast convergence speed and a low steady-state error similar to the conventional NSAF. In addition, the proposed algorithm has lower computational complexity than NSAF due to the signed regressor of the input signal at each subband. The theoretical mean-square performance analysis of the proposed algorithm in the stationary and nonstationary environments is studied based on the energy conservation relation and the steady-state, the transient, and the stability bounds of the SR-NSAF are predicated by the closed form expressions. The good performance of SR-NSAF is demonstrated through several simulation results in system identification, acoustic echo cancelation (AEC) and line EC (LEC) applications. The theoretical relations are also verified by presenting various experimental results.

This paper presents a new variable step-size normalized subband adaptive filter (VSS-NSAF) algorithm with dynamic selection (DS) of subband filters. The selection strategy is performed to achieve the largest decrease between the successive mean square deviations at every iteration. The proposed VSS-DS-NSAF has fast convergence speed and low steady-state mean square error (MSE). Also due to DS feature, the VSS-DS-NSAF has lower computational complexity than VSS-NSAF. We demonstrate the good performance of the proposed algorithm through several simulations in system identification scenario.

In single-channel hands-free telephony, the acoustic coupling between the loudspeaker and the microphone can be strong and this generates echoes that can degrade user experience. Therefore, effective acoustic echo cancellation (AEC) is necessary to maintain a stable system and hence improve the perceived voice quality of a call. Traditionally, adaptive filters have been deployed in acoustic echo cancellers to estimate the acoustic impulse responses (AIRs) using adaptive algorithms. The performances of a range of well-known algorithms are studied in the context of both AEC and network echo cancellation (NEC). It presents insights into their tracking performances under both time-invariant and time-varying system conditions. In the context of AEC, the level of sparseness in AIRs can vary greatly in a mobile environment. When the response is strongly sparse, convergence of conventional approaches is poor. Drawing on techniques originally developed for NEC, a class of time-domain and a frequency-domain AEC algorithms are proposed that can not only work well in both sparse and dispersive circumstances, but also adapt dynamically to the level of sparseness using a new sparseness-controlled approach. As it will be shown later that the early part of the acoustic echo path is sparse while the late reverberant part of the acoustic path is dispersive, a novel approach to an adaptive filter structure that consists of two time-domain partition blocks is proposed such that different adaptive algorithms can be used for each part. By properly controlling the mixing parameter for the partitioned blocks separately, where the block lengths are controlled adaptively, the proposed partitioned block algorithm works well in both sparse and dispersive time-varying circumstances. A new insight into an analysis on the tracking performance of improved proportionate NLMS (IPNLMS) is presented by deriving the expression for the mean-square error. By employing the framework for both sparse and dispersive time-varying echo paths, this work validates the analytic results in practical simulations for AEC. The time-domain second-order statistic based blind SIMO identification algorithms, which exploit the cross relation method, are investigated and then a technique with proportionate step-size control for both sparse and dispersive system identification is also developed.

In this brief, an improved multiband-structured subband adaptive filter (IMSAF) algorithm is applied in diffusion networks. The introduced diffusion IMSAF (DIMSAF) utilizes the input regressors at each subband to increase the convergence speed of the diffusion normalized subband adaptive filter. This advantage is achieved at the cost of increased computational complexity and steady-state error. Therefore, the DIMSAF with dynamic selection of nodes (DIMSAF-DSN) is proposed. In DIMSAF-DSN, the nodes are dynamically selected according to the largest decrease in mean-square deviation at each iteration. The DIMSAF-DSN has low computational complexity, fast convergence speed, and low steady-state error. Simulation results demonstrate the good performance of the proposed algorithms.

The task of the identification of the multichannel linear system with long channel impulse responses is considered in this paper. The multichannel acoustic echo cancellation in the closed rooms is an example of this task. Dependently on the room size, its decoration and the signal sampling frequency value the duration of the identified impulse responses may take a few thousand samples. To solve this task a multichannel adaptive filter with the corresponding large number of weights is required. The implementation of such filters is a difficult task as the arithmetic complexity of the adaptive filters depends on the number of the filter weights. Due to this reason the effective time-domain Recursive Least Squares (RLS) adaptive filters even in their fast (computationally efficient) forms are rarely used in the acoustic echo cancellers. The computationally simple but less effective time-domain Least Means Squares (LMS) or the Normalized LMS (NLMS) adaptive filters are usually used for this purpose. However, such filters have a number of well-known drawbacks. The slow convergence is one of them when the correlated or nonstationary signals are processed. Speech is an example of such signals. The frequency-domain LMS/NLMS adaptive filters de-correlate the signals. These filters have a smaller computational complexity comparing to their time-domain prototypes. However, as the signals are processed using the blocks of the samples, the frequency-domain adaptive filters introduce the output signal delay and demonstrate a slower convergence because their weights are updated only once per block. This decreases the tracking ability of the frequency-domain adaptive filters. The ability is an important feature of the adaptive filters in the acoustic echo cancellation task because the acoustic medium is usually a not time-invariant one. The usage of the Fast Affine Projection (FAP) adaptive filter is the tradeoff between the acoustic echo canceller efficiency and its complexity. Such adaptive filter is widely used in the single-channel acoustic echo cancellers. This paper considers the adaptive filter application in the multichannel echo cancellers. The computational procedure of a multichannel FAP adaptive filter is presented and an example of the two-channel echo canceller is considered in this paper. The simulation demonstrates about the same steady-state performance of the FAP adaptive filter as that of the time-domain RLS adaptive filter or the time/frequency domain NLMS adaptive filter. However the convergence of the FAP adaptive filter comparing to the NLMS one is a few times faster. Because of this feature the FAP adaptive filter can efficiently track the changes in the processed signal statistics and/or the changes in the acoustic medium.

Adaptive filters are frequently employed in many applications in which the statistics of the underlying signals are either unknown a priori or slowly time-varying. Adaptive filtering algorithms are usually expected to have fast convergence speed, low computational complexity and high robustness to numerical problems and outlier interference. Many researchers have invested enormous efforts in deriving new algorithms with the above properties and analyzing their convergence behaviors. The latter is even more complicated due to the mathematical manipulations involved. Following the same guideline, in this dissertation we study a set of efficient adaptive transversal filtering algorithms and their convergence performance analysis. The development of the new algorithms and the establishment of the effective analytical framework are based on three important modeling approaches. (1) Noise modeling approach. By modeling the outliner impulsive noise as contaminated Gaussian distributed, we study the normalized least mean M-estimate (NLMM), transform domain NLMM (TD-NLMM) and partial update NLMM (PU-NLMM) algorithms which are more robust to impulsive noise than their conventional normalized least mean square (NLMS)-based counterparts. Complete convergence analyses of these algorithms are provided to interpret the underlying principles behind their performances. (2) Input modeling approach. By modeling the input signal as a low-order autoregressive process, the fast LMS/Newton algorithm can reduce the computational complexity of the traditional Newton-type algorithm while retaining its improved convergence speed. We propose two improved fast LMS/Newton algorithms. One is the block exact fast LMS/Newton algorithm which is mathematically equivalent to the original algorithm but has a significantly reduced complexity. The other is the robust fast LMM/Newton algorithm which is derived through the noise modeling approach used in (1). Moreover, we also develop a Newton-type algorithm with a uniform structure. It can realize flexible performance-complexity tradeoff and has the potential to be incorporated with the certain input modeling approach to achieve fast convergence performance with low complexity. (3) Channel modeling approach. By exploiting the sparse feature of the system channel encountered in vast applications, the generalized proportionate NLMS (GP-NLMS) algorithm possesses a faster initial convergence and tracking speed. Our proposed generalized proportionate stepsize (GPS)-fast LMS/Newton algorithm combines the advantages of the GP-NLMS and the fast LMS/Newton algorithms and exhibits a superior overall convergence and tracking performance. In addition, based on the GP-NLMS algorithm, another variable forgetting factor QR decomposition-based recursive least M-estimate (RLM) (VFF QR-RLM) algorithm is proposed. It has both an improved numerical stability and faster overall convergence and tracking speed than the conventional RLM algorithm using constant forgetting factor. All the proposed algorithms and the corresponding convergence analysis were tested through extensive computer simulations and good agreements between the theoretical predictions and the experimental results were observed. In general, the algorithms proposed in this dissertation have addressed some of the key problems in adaptive filtering algorithm design with the aid of three modeling approaches. They should form an algorithm set whose constituting components are intrinsically connected together and can potentially be utilized for various applications either individually or in a combination.

Presents the concept of transform-domain (TD) adaptive filtering based on the discrete nonlinear Wiener model for a 2nd-order Volterra system identification application with a colored Gaussian input signal. In earlier work (1999), we presented the 2nd- and 3rd-order nonlinear discrete Wiener adaptive algorithm, and its performance analysis focused on the Gaussian white input case. In this paper, we present new results for the colored Gaussian input environment. From the analysis, we realize that the nonlinear Wiener model has many advantages over other models, such as the Volterra model. The main advantage is that, for both white and colored Gaussian input, it can have a reasonably fast convergence speed and low computational complexity. This is because the nonlinear Wiener model performs a complete orthogonalization procedure to the truncated Volterra series which allows us to use linear adaptive filtering algorithms to calculate all the coefficients efficiently. For a Gaussian colored input signal, the TD nonlinear Wiener model is introduced to decouple the colored effect. Computer simulation results of discrete cosine transform (DCT) domain nonlinear Wiener adaptive filtering with 1st-order autoregressive colored Gaussian input are presented to verify the theoretical analysis.

In some applications of adaptive filtering such as active noise reduction, and network and acoustic echo cancellation, the adaptive filter may be required to have a large number of coefficients in order to model the unknown physical medium with sufficient accuracy. The computational complexity of adaptation algorithms is proportional to the number of filter coefficients. This implies that, for long adaptive filters, the adaptation task can become prohibitively expensive, ruling out cost-effective implementation on digital signal processors. The purpose of partial coefficient updates is to reduce the computational complexity of an adaptive filter by adapting a block of the filter coefficients rather than the entire filter at every iteration. In this paper, we develop a selective-partial-update normalized least-mean-square (NLMS) algorithm, and analyze its stability using the traditional independence assumptions and error-energy bounds. Selective partial updating is also extended to the affine projection (AP) algorithm by introducing multiple constraints. The new algorithms appear to have good convergence performance as attested to by computer simulations with real speech signals.

We propose an acoustic echo and noise canceller (AENC) using a shared-error normalized least mean square (SENLMS) algorithm. The AENC consists of an acoustic echo canceller (AEC) and a noise suppressor (NS) to reduce the acoustic echo and background noise. One of the structures of the AENC is the AEC based on a frequency-domain block normalized least mean square (FBNLMS) algorithm and the NS based on a Wiener filter (WF). However, FBNLMS and WF are different from each other in terms of the optimization problem. Hence, it is difficult to optimize both the AEC and the NS at the same time. In this paper, the SENLMS algorithm is proposed and used for the AENC. The proposed AENC utilizes time-domain adaptive digital filters (ADFs) for both the AEC and the adaptive noise canceller (ANC), and two ADFs are optimized by the NLMS algorithm with the shared error. Simulation results show that the proposed AENC can improve the ability of acoustic echo reduction while maintaining the ability of noise reduction under the high and low signal-to-noise ratio conditions.

Today, with increase in the demand for higher quality communication, a kind of long adaptive filter is frequently encountered in practical application, such as the acoustic echo cancellation. Increase of adaptive filter length from decades to hundreds or thousands causes the conventional adaptive algorithms encounter new challenges. Therefore, a new variable step-size normalized least-mean-square algorithm combined with Partial update is proposed and its performances are investigated through simulations. The proposed step size method takes into account the instantaneous value of the output error and provides a trade-off between the convergence rate and the steady-state coefficient error. In order to deal with this obstacle that the large number of filter coefficients diminishes the usefulness of the adaptive filtering algorithm owing to increased complexity, the new algorithm employing tap-selection partial update schemes only updates subset of the filter coefficients that correspond to the largest magnitude elements of the regression vector. Simulation results of such applications in acoustic echo cancellation verify that the proposed algorithm achieves higher rate of convergence and brings significant computation savings compared to the NLMS algorithm.

Adaptive filters, as part of digital signal systems, have been widely used, as well as in applications such as adaptive noise cancellation, adaptive beam forming, channel equalization, and system identification. However, its implementation takes a great deal and becomes a very important field in digital system world. When FPGA (Field Programmable Logic Array) grows in area and provides a lot of facilities to the designers, it becomes an important competitor in the signal processing market. In general FIR structure has been used more successfully than IIR structure in adaptive filters. However, when the adaptive FIR filter was made this required appropriate algorithm to update the filter’s coefficients. The algorithm used to update the filter coefficient is the Least Mean Square (LMS) algorithm which is known for its simplification, low computational complexity, and better performance in different running environments. When compared to other algorithms used for implementing adaptive filters the LMS algorithm is seen to perform very well in terms of the number of iterations required for convergence. This phenomenon can be achieved by a sufficient choice of bit length to represent the filter’s coefficients. This paper presents a lowcost and high performance programmable digital finite impulse response (FIR) filter. It follows the adaptive algorithm used for the development of the system. The architecture employs the computation sharing algorithm to reduce the computation complexity.

We present a novel affine projection algorithm (APA) which dynamically selects input vectors in order to improve convergence performance. The optimum selection of the input vectors is derived by the largest decrease of the mean-square deviation. The experimental results show that the proposed algorithm has a fast convergence speed and a small steady-state error compared to the conventional APA. In addition, the proposed algorithm retains a low overall computational complexity.

To improve the convergence rate, multiple input vectors are used to update the filter coefficients in the affine projection algorithm (APA) family, however, different input vector affects the convergence differently. The ideal selection criterion of the input vectors is derived by the largest decrease of the mean-square deviation. This selection condition fits for different algorithms of the APA family. Based on the ideal selection criterion, a family of APAs which dynamically select input vectors is presented. The experimental results show that the proposed algorithms have fast convergence speed, small steady-state error compared to the conventional APAs. In addition, the proposed algorithms retain low overall computational complexity.

In the traditional decorrelation normalized least-mean-square (D-NLMS) algorithm, high computational complexity is mainly caused by finding the decorrelated-vector. To address this issue, this article proposes a low-complexity implementation approach, which cleverly utilizes the periodic update of the decorrelation parameters and delay characteristics of the decorrelated-vector. We firstly develop two low-complexity decorrelation algorithms, (i) fast D-NLMS (FD-NLMS) and (ii) approximate FD-NLMS (AFD-NLMS) which is an approximate version of the first algorithm with even smaller computational requirement. Theoretical performance of the FD-NLMS scheme is also derived. To further obtain low steady-state error in the acoustic echo cancellation (AEC) application, separated-decorrelation AEC structure and robust step-size schemes are designed, resulting in two improved algorithms, namely, fast separated-decorrelation NLMS (FSD-NLMS) and approximate FSD-NLMS (AFSD-NLMS). Finally, extensive simulation study on system identification and AEC is undertaken to verify the efficiency of the proposed methods.