Dichotomous Coordinate Descent Iterations Research Articles

Nonlinear acoustic echo cancellation using low-complexity low-rank recursive least-squares algorithms

Adaptive exponential functional link networks (AEFLN) are a type of linear-in-the-parameter nonlinear filters, which have shown enhanced modelling capability for nonlinear systems. However, the convergence speed of traditional AEFLN is generally slow for long impulse responses, hence, AEFLN trained using recursive least-squares becomes an attractive choice to achieve faster convergence. Moreover, huge computational burden of RLS makes it unsuitable in practical applications like echo cancellation. While low complexity versions of RLS are widely available in literature, they still suffer from low convergence speed. To address this issue, we propose a nonlinear acoustic echo cancellation (NAEC) system using AEFLN-RLS, based on the nearest Kronecker product (NKP) decomposition and low-rank approximation technique, which not only reduces computational complexity but also achieves improved convergence speed (especially tracking). To further improve the echo cancellation performance in non-stationary conditions, a variable regularization approach based NKP-AEFLN-RLS system is also proposed. To also reduce the computational complexity further, dichotomous coordinate descent (DCD) updates are incorporated into the proposed NKP-AEFLN-RLS NAEC system and its variable regularization version. Experimental results show the effectiveness of the proposed algorithms, with the variable regularized version of the algorithm using DCD iterations showing the best compromise between convergence capability and lower computational complexity.

BEM Adaptive filtering for SI cancellation in full-duplex underwater acoustic systems

Self-interference (SI) cancellation (SIC) is the key technology for achieving full-duplex (FD) communications in underwater acoustic systems. In practice, SI channels are often fast-varying, e.g., due to reflections from surface waves. Classical adaptive filters, such as the recursive least squares (RLS) algorithm, predict the channel impulse response when used for channel estimation. If a tracking delay is acceptable, interpolating estimators capable of providing more accurate estimates of time-varying impulse responses can be used. Interpolating estimators with good tracking performance are normally of high complexity. In this paper, we propose low-complexity interpolating adaptive filters which combine the basis expansion model (BEM) approach with the sliding-window RLS (SRLS) algorithm. Specifically, we use the Legendre polynomials as the basis functions and solve the system of equations using dichotomous coordinate descent (DCD) iterations, thus the name the SRLS-L-DCD adaptive filter. A sparse algorithm (HSRLS-L-DCD) based on homotopy iterations is then proposed to exploit the sparsity in the expansion coefficients. The identification performance of the adaptive filters is investigated by a simulation which mimics an FD lake experiment. Both the simulation and lake experiments show that significant improvement in the SIC performance is achieved with the proposed low-complexity algorithms compared to the classical SRLS algorithm.

Diffusion recursive total least square algorithm over adaptive networks and performance analysis

In most existing distributed estimation algorithms, many scholars adopted the assumption that the output signal of the system is noisy and the input signal is sufficiently accurate. However, in real applications, the filter input vector usually contains noise. In such situations, it has been proved that the adaptive filtering algorithm using total least squares (TLS) method has shown better performance than classical least squares (LS) method. So in this paper, we propose a diffusion recursive total least squares (DRTLS) algorithm by using the single inverse power iterations in the distributed network. Then, we analyze the mean and mean square behaviors of the proposed DRTLS algorithm. In addition, to reduce the computational complexity of the DRTLS algorithm,an improved algorithm called DCD-DRTLS is also proposed by using the dichotomous coordinate descent (DCD) iterations. At last, simulation results illustrate the superiority of the proposed algorithms and the validity of the theoretical analysis results.

Improved Convergence Speed of a DCD-Based Algorithm for Sparse Solutions

To solve a system of equations that needs few updates, such as sparse systems, the leading dichotomous coordinate descent (DCD) algorithm is better than the cyclic DCD algorithm because of its fast speed of convergence. In the case of sparse systems requiring a large number of updates, the cyclic DCD algorithm converges faster and has a lower error level than the leading DCD algorithm. However, the leading DCD algorithm has a faster convergence speed in the initial updates. In this paper, we propose a combination of leading and cyclic DCD iterations, the leading-cyclic DCD algorithm, to improve the convergence speed of the cyclic DCD algorithm. The proposed algorithm involves two steps. First, by properly selecting the number of updates of the solution vector used in the leading DCD algorithm, a solution is obtained from the leading DCD algorithm. Second, taking the output of the leading DCD algorithm as the initial values, an improved soft output is generated by the cyclic DCD algorithm with a large number of iterations. Numerical results demonstrate that when the solution sparsity γ is in the interval [ 1 / 8 , 6 / 8 ] , the proposed leading-cyclic DCD algorithm outperforms both the existing cyclic and leading DCD algorithms for all iterations.

Design of AWC core using DCD iterations for MVDR beamformer

This paper presents a hardware architecture using Dichotomous Coordinate Descent (DCD) iterations for Adaptive Weight Computation (AWC) in Minimum Variance Distortionless Response (MVDR) Beamformer. The objective of the proposed work is to achieve low latency and reduced area architecture for the AWC stage in MVDR Beamformer. The work investigates the computation of adaptive weight for 4,8,16 and 32 channels array beamformer. In order to improve the weight updating rate, the existing complex valued cyclic DCD hardware implementation is optimized to 2 clock cycles per iteration. Moreover, DCD algorithm implementation does not require any multiplication and division. The proposed work is implemented on FPGA platform, and the results are compared with state-of-art literature and conclude that the proposed architecture is suitable for MVDR Beamformer employed in high sampling rate applications like medical ultrasound imaging due to its occupancy of a moderate amount of resources and improved processing speed.

Recursive Least-Squares Algorithms for the Identification of Low-Rank Systems

The recursive least-squares (RLS) adaptive filter is an appealing choice in many system identification problems. The main reason behind its popularity is its fast convergence rate. However, this algorithm is computationally very complex, which may make it useless for the identification of long length impulse responses, like in echo cancellation. Computationally efficient versions of the RLS algorithm, like those based on the dichotomous coordinate descent (DCD) iterations or QR decomposition techniques, reduce the complexity, but still have to face the challenges related to long length adaptive filters (e.g., convergence/tracking capabilities). In this paper, we focus on a different approach to improve the efficiency of the RLS algorithm. The basic idea is to exploit the impulse response decomposition based on the nearest Kronecker product and low-rank approximation. In other words, a high-dimension system identification problem is reformulated in terms of low-dimension problems, which are combined together. This approach was recently addressed in terms of the Wiener filter, showing appealing features for the identification of low-rank systems, like real-world echo paths. In this paper, besides the development of the RLS algorithm based on this approach, we also propose a variable regularized version of this algorithm (using the DCD method to reduce the complexity), with improved robustness to double-talk. Simulations are performed in the context of echo cancellation and the results indicate the good performance of these algorithms.

MMP-DCD-CV based sparse channel estimation algorithm for underwater acoustic transform domain communication system

In this paper, we propose a computationally efficient multipath matching pursuit (MMP) channel estimation algorithm for underwater acoustic (UWA) transform domain communication systems (TDCSs). The algorithm, referred to as the MMP-DCD-CV algorithm, is based on the dichotomous coordinate descent (DCD) iterations and cross validation (CV). The MMP-DCD-CV sparse channel estimator in each iteration searches for multiple promising path candidates most relevant to a residual vector and chooses the best candidate. The DCD iterations are used to solve the corresponding least squares problem with low complexity and numerical stability. The CV provides a stopping criterion of the algorithm without a priori information on the channel sparsity and noise level and examines whether the algorithm overfits its data, thus improving the estimation accuracy. The performance of the proposed algorithm is evaluated under simulated sparse UWA channels. The numerical results show that the algorithm achieves better performance than the original MMP algorithm, has lower complexity, and does not require prior knowledge on the channel sparsity and noise level. We also propose an UWA TDCS with sparse channel estimation based on the proposed MMP-DCD-CV algorithm. The proposed UWA communication system is tested by the Waymark simulator, providing the virtual signal transmission in the UWA channel, with a measured Sound Speed Profile and bathymetry. Numerical results demonstrate that the UWA TDCS with the proposed sparse channel estimator offers considerable improvement in system performance compared to other TDCS schemes.

Digital Self-Interference Cancellation for Full-Duplex Underwater Acoustic Systems

Underwater acoustic (UWA) communication suffers from the limited available bandwidth for data transmission. Full-duplex (FD) communication has demonstrated the ability of achieving high spectral efficiency in terrestrial radio communications. There is a significant potential in adopting the benefits of FD in UWA systems. The major obstacle in FD communications is the severe self-interference (SI) introduced by the near-end transmitted signal. For FD UWA communications, the low signal frequency allows high-resolution analogue-to-digital converter (ADCs) to be used. With higher performance ADCs, it might be possible to achieve higher digital SI cancellation performance than that in FD radio systems. In this brief, we present experimental results of digital SI cancellation in FD UWA system, based on the use of the low-complexity recursive least-squares (RLS) adaptive filter with dichotomous coordinate descent iterations. The experimental results demonstrate that up to 46 dB of SI is cancelled when we use the transmitted digital data as the regressor in the adaptive filter. To improve the SI cancellation performance without introducing high-complexity operation, we use the digitalized power amplifier (PA) output as the regressor to deal with the non-linear distortions caused by the PA in the transmitted chain. With this technique, as high a level as 69 dB of digital SI cancellation is achieved.

A Delay Relaxed RLS-DCD Algorithm for Real-Time Implementation

The recursive least squares algorithm (RLS) using dichotomous coordinate descent (DCD) iterations, namely, RLS-DCD, is regarded to be well suited for hardware implementation because of its small computational complexity compared to the classical RLS algorithm. While this is true, yet another important aspect that ultimately determines its applicability for real-time applications with high sample rates, is its iteration bound. In this brief, we discuss this issue and propose a modified RLS-DCD algorithm based on delay relaxation whose iteration bound can be reduced arbitrarily. The degradation in convergence speed is shown to be tolerable, which results in still much faster convergence compared to the normalized least mean square algorithm.

VFF ‐norm penalised WL‐RLS algorithm using DCD iterations for underwater acoustic communication

Based on dichotomous coordinate descent (DCD) iterations and with the use of the variable forgetting factor (VFF), a widely linear (WL) l1-norm recursive least squares (RLS) adaptive filtering algorithm is proposed for sparse underwater acoustic channelequalization. In the proposed l1-norm WL-RLS algorithm with VFF, the WL model is employed to exploit the second order statistics of the non-circular signals and the VFF is employed to improve the tracking ability of the RLS algorithms. DCD iterations are incorporated in the proposed l1-norm WL-RLS-DCD algorithm with VFF to reduce the computing complexity. Moreover, the proposed algorithms are employed by the direct adaptive decision feedback equalizer (DA-DFE). Numerical results indicate that compared with the conventional RLS, l1-norm RLS, WL-RLS with VFF, l1-norm WL-RLS-DCD algorithms, the proposed algorithms achieve a better performance in terms of the convergence rate, mean square errorand symbol error ratein the DA-DFE receiver. Experimental results also show that the proposed algorithms can promote the DA-DFE receiver to obtain a better performance in the sparse time-varying underwater acoustic communication system. Even though the transmitted signals are circular quadrature phase-shift keying (QPSK) through the underwater acoustic channel, the proposed adaptive RLS algorithms can still obtain a better performance.

Low-Complexity DCD-Based Sparse Recovery Algorithms

Sparse recovery techniques find applications in many areas. Real-time implementation of such techniques has been recently an important area for research. In this paper, we propose computationally efficient techniques based on dichotomous coordinate descent (DCD) iterations for recovery of sparse complex-valued signals. We first consider $\ell _{2} \ell _{1}$ optimization that can incorporate a priori information on the solution in the form of a weight vector. We propose a DCD-based algorithm for $\ell _{2} \ell _{1}$ optimization with a fixed $\ell _{1}$ regularization, and then efficiently incorporate it in reweighting iterations using a warm start at each iteration. We then exploit homotopy by sampling the regularization parameter and arrive at an algorithm that, in each homotopy iteration, performs the $\ell _{2} \ell _{1}$ optimization on the current support with a fixed regularization parameter and then updates the support by adding/removing elements. We propose efficient rules for adding and removing the elements. The performance of the homotopy algorithm is further improved with the reweighting. We then propose an algorithm for $\ell _{2} \ell _{0}$ optimization that exploits homotopy for the $\ell _{0}$ regularization; it alternates between the least-squares (LSs) optimization on the support and the support update, for which we also propose an efficient rule. The algorithm complexity is reduced when DCD iterations with a warm start are used for the LS optimization, and, as most of the DCD operations are additions and bit-shifts, it is especially suited to real-time implementation. The proposed algorithms are investigated in channel estimation scenarios and compared with known sparse recovery techniques, such as the matching pursuit (MP) and YALL1 algorithms. The numerical examples show that the proposed techniques achieve a mean-squared error smaller than that of the YALL1 algorithm and complexity comparable to that of the MP algorithm.

A block parallel ℓ0-norm penalized shrinkage and widely linear affine projection algorithm for adaptive filter

To improve the identification capability of AP algorithm in time-varying sparse system, we propose a block parallel l 0 -SWL-DCD-AP algorithm in this paper. In the proposed algorithm, we first introduce the l 0 -norm constraint to promote its application for sparse system. Second, we use the shrinkage denoising method to improve its track ability. Third, we adopt the widely linear processing to take advantage of the non-circular properties of communication signals. Last, to reduce the high computational complexity and make it easy to implemented, we utilize the dichotomous coordinate descent (DCD) iterations and the parallel processing to deal with the tap-weight update in the proposed algorithm. To verify the convergence condition of the proposed algorithm, we also analyze its steady-state behavior. Several simulation are done and results show that the proposed algorithm can achieve a faster convergence speed and a lower steady-state misalignment than similar APA-type algorithm. When apply the proposed algorithm in the decision feedback equalizer (DFE), the bite error rate (BER) decreases obviously.

Low-Complexity $$l_0$$ l 0 -Norm Penalized Shrinkage Linear and Widely Linear Affine Projection Algorithms

In this paper, we propose an $$l_0$$l0-norm penalized shrinkage linear affine projection ($$l_0$$l0-SL-AP) algorithm and an $$l_0$$l0-norm penalized shrinkage widely linear affine projection ($$l_0$$l0-SWL-AP) algorithm. The proposed algorithms provide variable step-size by minimizing the noise-free a posteriori error at each iteration and introduce an $$l_0$$l0-norm constraint to the cost function. The $$l_0$$l0-SWL-AP algorithm also exploits noncircular properties of the input signal. In contrast with conventional AP algorithms, the proposed algorithms increase the estimation accuracy for time-varying sparse system identification. A quantitative analysis of the convergence behavior for the $$l_0$$l0-SWL-AP algorithm verifies the capabilities of the proposed algorithms. To reduce the complexity, we also introduce dichotomous coordinate descent (DCD) iterations to the proposed algorithms ($$l_0$$l0-SL-DCD-AP and $$l_0$$l0-SWL-DCD-AP) in this paper. Simulations indicate that the $$l_0$$l0-SL-AP and $$l_0$$l0-SWL-AP algorithms provide faster convergence speed and lower steady-state misalignment than the previous APA-type algorithms. The $$l_0$$l0-SL-DCD-AP and $$l_0$$l0-SWL-DCD-AP algorithms perform similarly to their counterparts but with reduced complexity.

Sliding-Window RLS Low-Cost Implementation of Proportionate Affine Projection Algorithms

This paper addresses adaptive filtering for sparse identification. Proportionate affine projection algorithms (PAPAs) are known to be efficient techniques for this purpose. We show that the PAPA performance may improve with an increase in the projection order M (for example, such as M = 512), which, however, also results in an increased complexity; the complexity is in general O(M <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> N) or at least O(MN) operations per sample, where N is the filter length. We show that PAPAs are equivalent to specific sliding-window recursive least squares (SRLS) adaptive algorithms with time-varying and tap-varying diagonal loading (SRLS-VDLs). We then propose an approximation to the SRLS-VDLs based on dichotomous coordinate descent (DCD) iterations with a complexity of O(N <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">u</sub> N), which does not depend on M; it depends on the number of DCD iterations N <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">u</sub> , which as we show can be significantly smaller than M, thus allowing a low-complexity implementation of PAPA adaptive filters.

Recursive Total Least-Squares Algorithm Based on Inverse Power Method and Dichotomous Coordinate-Descent Iterations

We develop a recursive total least-squares (RTLS) algorithm for errors-in-variables system identification utilizing the inverse power method and the dichotomous coordinate-descent (DCD) iterations. The proposed algorithm, called DCD-RTLS, outperforms the previously proposed RTLS algorithms, which are based on the line-search method, with reduced computational complexity. We perform a comprehensive analysis of the DCD-RTLS algorithm and show that it is asymptotically unbiased as well as being stable in the mean. We also find a lower bound for the forgetting factor that ensures mean-square stability of the algorithm and calculate the theoretical steady-state mean-square deviation (MSD). We verify the effectiveness of the proposed algorithm and the accuracy of the predicted steady-state MSD via simulations.

Dynamic RLS-DCD for Sparse System Identification

In this passage we propose a computationally efficient adaptive filtering algorithm for sparse system identification.The algorithm is based on dichotomous coordinate descent iterations, reweighting iterations,iterative support detection.In order to reduce the complexity we try to discuss in the support.we suppose the support is partial,and partly erroneous.Then we can use the iterative support detection to solve the problem.Numerical examples show that the proposed method achieves an identification performance better than that of advanced sparse adaptive filters (l1-RLS,l0-RLS) and its performance is close to the oracle performance.

A Fast and Accurate Sparse Continuous Signal Reconstruction by Homotopy DCD with Non-Convex Regularization

In recent years, various applications regarding sparse continuous signal recovery such as source localization, radar imaging, communication channel estimation, etc., have been addressed from the perspective of compressive sensing (CS) theory. However, there are two major defects that need to be tackled when considering any practical utilization. The first issue is off-grid problem caused by the basis mismatch between arbitrary located unknowns and the pre-specified dictionary, which would make conventional CS reconstruction methods degrade considerably. The second important issue is the urgent demand for low-complexity algorithms, especially when faced with the requirement of real-time implementation. In this paper, to deal with these two problems, we have presented three fast and accurate sparse reconstruction algorithms, termed as HR-DCD, Hlog-DCD and Hlp-DCD, which are based on homotopy, dichotomous coordinate descent (DCD) iterations and non-convex regularizations, by combining with the grid refinement technique. Experimental results are provided to demonstrate the effectiveness of the proposed algorithms and related analysis.

DCD-RLS Adaptive Filters With Penalties for Sparse Identification

In this paper, we propose a family of low-complexity adaptive filtering algorithms based on dichotomous coordinate descent (DCD) iterations for identification of sparse systems. The proposed algorithms are appealing for practical designs as they operate at the bit level, resulting in stable hardware implementations. We introduce a general approach for developing adaptive filters with different penalties and specify it for exponential and sliding window RLS. We then propose low-complexity DCD-based RLS adaptive filters with the lasso, ridge-regression, elastic net, and penalties that attract sparsity. We also propose a simple recursive reweighting of the penalties and incorporate the reweighting into the proposed adaptive algorithms to further improve the performance. For general regressors, the proposed algorithms have a complexity of operations per sample, where is the filter length. For transversal adaptive filters, the algorithms require only operations per sample. A unique feature of the proposed algorithms is that they are well suited for implementation in finite precision, e.g., on FPGAs. We demonstrate by simulation that the proposed algorithms have performance close to the oracle RLS performance.

Orthogonal matching pursuit with DCD iterations

Greedy algorithms are computationally efficient optimisation techniques for solving sparse recovery problems. Matching pursuit (MP) and orthogonal MP (OMP) are popular greedy algorithms; MP possesses the lowest complexity whereas OMP provides better performance. In this reported work, OMP is modified using dichotomous coordinate descent (DCD) iterations and an algorithm is arrived at that has performance close to that of OMP and complexity even lower than that of MP.

Reduced-Complexity Constrained Recursive Least-Squares Adaptive Filtering Algorithm

A linearly-constrained recursive least-squares adaptive filtering algorithm based on the method of weighting and the dichotomous coordinate descent (DCD) iterations is proposed. The method of weighting is employed to incorporate the linear constraints into the least-squares problem. The normal equations of the resultant unconstrained least-squares problem are then solved using the DCD iterations. The proposed algorithm has a significantly smaller computational complexity than the previously proposed constrained recursive least square (CRLS) algorithm while delivering convergence performance on par with CRLS. The effectiveness of the proposed algorithm is demonstrated by simulation examples.