Analytical and Simple Form of Shrinkage Functions for Non-Convex Penalty Functions in Fused Lasso Algorithm

Abstract

In some circumstances, the performance of machine learning (ML) tasks are based on the quality of signal (data) that is processed in these tasks. Therefore, the pre-processing techniques, such as reconstruction and denoising methods, are important techniques in ML tasks. In reconstructed (estimated) method, the fused lasso algorithm with non-convex penalty function is an efficient method when the signal corrupted by additive white Gaussian noise (AWGN) is considered. Therefore, this paper proposes new shrinkage functions for non-convex penalty functions, modified arctangent and exponential models, in fused lasso formulation. A lot of works present the shrinkage function for arctangent penalty function. Unfortunately, there is no closed-form solution. The numerical solution is required for shrinkage function of this penalty function. However, the analytical solution is derived in this paper. Moreover, the shrinkage function of modified exponential penalty function is proposed. This shrinkage function obtains from simple iterative method, fixed-point algorithm. We demonstrate the proposed methods through simulations with standard one-dimensional signals contaminated by AWGN. The proposed techniques are compared with traditional estimation methods, such as total variation (TV) and wavelet denoising methods. In experimental results, our proposed methods outperform several exiting methods both visual quality and in terms of root mean square error (RMSE). In fact, the proposed methods can better preserve the feature of noise-free signal than the compared methods. The denoised signals produced by the proposed methods are less smooth than the denoised signals produced by the compared methods.