Fused Lasso Signal Approximator Research Articles

Combining variational mode decomposition with regularisation techniques to denoise MRI data

In this paper, we propose a novel method for removing noise from MRI data by exploiting regularisation techniques combined with variational mode decomposition. Variational mode decomposition is a new decomposition technique for sparse decomposition of a 1D or 2D signal into a set of modes. In turn, regularisation is a method that can translate the ill-posed problem (e.g., image denoising) into a well-posed problem.The proposed method aims to remove the noise from the image in two steps. In the first step, the MR imaging data are decomposed by the 2D variational mode decomposition algorithm. In the second step, for effective suppression of Rician noise from MRI data, we used the fused lasso signal approximator with all modes acquired from the medical scan.The performance of the proposed approach was compared with state-of-the-art reference methods based on different metrics, that is, the peak signal-to-noise ratio, the structural similarity index metrics, the high-frequency error norm, the quality index based on local variance, and the sharpness index. The experiments were performed on the basis of both simulated and real images. The presented results prove the high denoising performance of the proposed algorithm; particularly under heavy noise conditions.

Path algorithms for fused lasso signal approximator with application to COVID-19 spread in Korea.

The fused lasso signal approximator (FLSA) is a smoothing procedure for noisy observations that uses fused lasso penalty on unobserved mean levels to find sparse signal blocks. Several path algorithms have been developed to obtain the whole solution path of the FLSA. However, it is known that the FLSA has model selection inconsistency when the underlying signals have a stair-case block, where three consecutive signal blocks are either strictly increasing or decreasing. Modified path algorithms for the FLSA have been proposed to guarantee model selection consistency regardless of the stair-case block. In this paper, we provide a comprehensive review of the path algorithms for the FLSA and prove the properties of the recently modified path algorithms' hitting times. Specifically, we reinterpret the modified path algorithm as the path algorithm for local FLSA problems and reveal the condition that the hitting time for the fusion of the modified path algorithm is not monotone in a tuning parameter. To recover the monotonicity of the solution path, we propose a pathwise adaptive FLSA having monotonicity with similar performance as the modified solution path algorithm. Finally, we apply the proposed method to the number of daily-confirmed cases of COVID-19 in Korea to identify the change points of its spread.

Sparse Reconstruction for Enhancement of the Empirical Mode Decomposition-Based Signal Denoising

Effective signal denoising methods are essential for science and engineering. In general, denoising algorithms may be either linear or non-linear. Most of the linear ones are unable to remove the noise from the real-world measurements. More suitable methods are usually based on non-linear approaches. One of the possible algorithms to signal denoising is based on empirical mode decomposition. The typical approach to the empirical mode decomposition-based signal denoising is the partial reconstruction. More recently, a new concept inspired by the wavelet thresholding principle was proposed. The method is named the interval thresholding. In this article, we further extend the concept by the application of the sparse reconstruction to the empirical mode decomposition-based signal denoising algorithm. To this end, we state and then solve the problem of signal denoising as a regularization problem. In the article, we consider three cases, that is, three types of penalty functions. The first algorithm is combining total variation denoising with empirical mode decomposition approach. In the second one, we applied the fused LASSO Signal Approximator to design the empirical mode decomposition-based signal denoising algorithm. The third approach solves the denoising problem by applying a non-convex sparse regularization. The proposed algorithms were validated on synthetic and real-world signals. We found that the proposed methods have the ability to improve the accuracy of the signal denoising in comparison to the reference methods. Significant improvements from both the synthetic and the real-world signals were obtained for the algorithm based on non-convex sparse regularization. The presented results show that the proposed approach to signal denoising based on empirical mode decomposition algorithm and sparse regularization gives a great improvement of accuracy, and it is the promising direction of future research.

A Projection Neural Network for the Generalized Lasso.

The generalized lasso (GLasso) is an extension of the lasso regression in which there is an l1 penalty term (or regularization) of the linearly transformed coefficient vector. Finding the optimal solution of GLasso is not straightforward since the penalty term is not differentiable. This brief presents a novel one-layer neural network to solve the generalized lasso for a wide range of penalty transformation matrices. The proposed neural network is proven to be stable in the sense of Lyapunov and converges globally to the optimal solution of the GLasso. It is also shown that the proposed neural solution can solve many optimization problems, including sparse and weighted sparse representations, (weighted) total variation denoising, fused lasso signal approximator, and trend filtering. Disparate experiments on the above problems illustrate and confirm the excellent performance of the proposed neural network in comparison to other competing techniques.

A Compact Neural Network for Fused Lasso Signal Approximator.

The fused lasso signal approximator (FLSA) is a vital optimization problem with extensive applications in signal processing and biomedical engineering. However, the optimization problem is difficult to solve since it is both nonsmooth and nonseparable. The existing numerical solutions implicate the use of several auxiliary variables in order to deal with the nondifferentiable penalty. Thus, the resulting algorithms are both time- and memory-inefficient. This paper proposes a compact neural network to solve the FLSA. The neural network has a one-layer structure with the number of neurons proportionate to the dimension of the given signal, thanks to the utilization of consecutive projections. The proposed neural network is stable in the Lyapunov sense and is guaranteed to converge globally to the optimal solution of the FLSA. Experiments on several applications from signal processing and biomedical engineering confirm the reasonable performance of the proposed neural network.

Modified path algorithm of fused Lasso signal approximator for consistent recovery of change points

The path algorithm of the fused lasso signal approximator is known to fail in finding change points when monotonically increasing or decreasing blocks exist in the mean vector. In this paper, we first understand why the standard path algorithm by Hoefling (2010) fails in the primal optimization problem. We then propose a modified path algorithm for the consistent recovery of the change points and study its properties theoretically and numerically.

Fused-MCP With Application to Signal Processing

ABSTRACTFriedman et al. proposed the fused lasso signal approximator (FLSA) to denoise piecewise constant signals by penalizing the ℓ1 differences between adjacent signal points. In this article, we propose a new method, referred to as the fused-MCP, by combining the minimax concave penalty (MCP) with the fusion penalty. The fused-MCP performs better than the FLSA in maintaining the profile of the original signal and preserving the edge structure. We show that, with a high probability, the fused-MCP selects the right change-points and has the oracle property, unlike the FLSA. We further show that the fused-MCP achieves the same l2 error rate as the FLSA. We develop algorithms to solve fused-MCP problems, either by transforming them into MCP regression problems or by using an adjusted majorization-minimization algorithm. Simulation and experimental results show the effectiveness of our method. Supplementary material for this article is available online.

A sequential solution for anisotropic total variation image denoising with interval constraints

We show that two problems involving the anisotropic total variation (TV) and interval constraints on the unknown variables admit, under some conditions, a simple sequential solution. Problem 1 is a constrained TV penalized image denoising problem; problem 2 is a constrained fused lasso signal approximator. The sequential solution entails finding first the solution to the unconstrained problem, and then applying a thresholding to satisfy the constraints. If the interval constraints are uniform, this sequential solution solves problem 1. If the interval constraints furthermore contain zero, the sequential solution solves problem 2. Here uniform interval constraints refer to all unknowns being constrained to the same interval. A typical example of application is image denoising in x-ray CT, where the image intensities are non-negative as they physically represent linear attenuation coefficient in the patient body. Our results are simple yet seem unknown; we establish them using the Karush–Kuhn–Tucker conditions for constrained convex optimization.

On stepwise pattern recovery of the fused Lasso

We study the property of the Fused Lasso Signal Approximator (FLSA) for estimating a blocky signal sequence with additive noise. We transform the FLSA to an ordinary Lasso problem, and find that in general the resulting design matrix does not satisfy the irrepresentable condition that is known as an almost necessary and sufficient condition for exact pattern recovery. We give necessary and sufficient conditions on the expected signal pattern such that the irrepresentable condition holds in the transformed Lasso problem. However, these conditions turn out to be very restrictive. We apply the newly developed preconditioning method — Puffer Transformation (Jia and Rohe, 2015) to the transformed Lasso and call the new procedure the preconditioned fused Lasso. We give non-asymptotic results for this method, showing that as long as the signal-to-noise ratio is not too small, our preconditioned fused Lasso estimator always recovers the correct pattern with high probability. Theoretical results give insight into what controls the ability of recovering the pattern — it is the noise level instead of the length of the signal sequence. Simulations further confirm our theorems and visualize the significant improvement of the preconditioned fused Lasso estimator over the vanilla FLSA in exact pattern recovery.

Adaptive basis expansion via $$\ell _1$$ ℓ 1 trend filtering

We propose a new approach for nonlinear regression modeling by employing basis expansion for the case where the underlying regression function has inhomogeneous smoothness. In this case, conventional nonlinear regression models tend to be over- or underfitting, where the function is more or less smoother, respectively. First, the underlying regression function is roughly approximated with a locally linear function using an $$\ell _1$$ penalized method, where this procedure is executed by extending an algorithm for the fused lasso signal approximator. We then extend the fused lasso signal approximator and develop an algorithm. Next, the residuals between the locally linear function and the data are used to adaptively prepare the basis functions. Finally, we construct a nonlinear regression model with these basis functions along with the technique of a regularization method. To select the optimal values of the tuning parameters for the regularization method, we provide an explicit form of the generalized information criterion. The validity of our proposed method is then demonstrated through several numerical examples.

A Dynamic Programming Algorithm for the Fused Lasso and L 0-Segmentation

We propose a dynamic programming algorithm for the one-dimensional Fused Lasso Signal Approximator (FLSA). The proposed algorithm has a linear running time in the worst case. A similar approach is developed for the task of least squares segmentation, and simulations indicate substantial performance improvement over existing algorithms. Examples of R and C implementations are provided in the online Supplementary materials, posted on the journal web site.

Creating facial animation of characters via MoCap data

We consider the problem of generating 3D facial animation of characters. An efficient procedure is realized by using the motion capture data (MoCap data), which is obtained by tracking the facial markers from an actor/actress. In some cases of artistic animation, the MoCap actor/actress and the 3D character facial animation show different expressions. For example, from the original facial MoCap data of speaking, a user would like to create the character facial animation of speaking with a smirk. In this paper, we propose a new easy-to-use system for making character facial animation via MoCap data. Our system is based on the interpolation: once the character facial expressions of the starting and the ending frames are given, the intermediate frames are automatically generated by information from the MoCap data. The interpolation procedure consists of three stages. First, the time axis of animation is divided into several intervals by the fused lasso signal approximator. In the second stage, we use the kernel k-means clustering to obtain control points. Finally, the interpolation is realized by using the control points. The user can easily create a wide variety of 3D character facial expressions by changing the control points.

A simple and efficient algorithm for fused lasso signal approximator with convex loss function

We consider the augmented Lagrangian method (ALM) as a solver for the fused lasso signal approximator (FLSA) problem. The ALM is a dual method in which squares of the constraint functions are added as penalties to the Lagrangian. In order to apply this method to FLSA, two types of auxiliary variables are introduced to transform the original unconstrained minimization problem into a linearly constrained minimization problem. Each updating in this iterative algorithm consists of just a simple one-dimensional convex programming problem, with closed form solution in many cases. While the existing literature mostly focused on the quadratic loss function, our algorithm can be easily implemented for general convex loss. We also provide some convergence analysis of the algorithm. Finally, the method is illustrated with some simulation datasets.

A Path Algorithm for the Fused Lasso Signal Approximator

The Lasso is a very well-known penalized regression model, which adds an L1 penalty with parameter λ1 on the coefficients to the squared error loss function. The Fused Lasso extends this model by also putting an L1 penalty with parameter λ2 on the difference of neighboring coefficients, assuming there is a natural ordering. In this article, we develop a path algorithm for solving the Fused Lasso Signal Approximator that computes the solutions for all values of λ1 and λ2. We also present an approximate algorithm that has considerable speed advantages for a moderate trade-off in accuracy. In the Online Supplement for this article, we provide proofs and further details for the methods developed in the article.