L1/2 regularization: a thresholding representation theory and a fast solver.

The log-sum function as a penalty has always been drawing widespread attention in the field of sparse problems. However, it brings a non-convex, non-smooth and non-Lipschitz optimization problem that is difficult to tackle. To overcome the problem, an iterative threshold algorithm for the sparse optimization problems with log-sum function is proposed in this paper. For brevity, the sparse optimization problems with log-sum function are named log-sum regularization. Firstly, by introducing an intermediate function to construct another new function, a property theorem about solution for log-sum regularization is established. Secondly, based on the above theorem, the optimal setting rules of the compromising parameters are elaborated, and an iterative log-sum threshold algorithm is proposed. Thirdly, under the situation that the compromising parameters of log-sum regularization are relatively small, it can be proven that the proposed algorithm converges to a local minimizer of log-sum regularization. Finally, a series of simulations are implemented to examine performance of the proposed algorithm, and the results exhibit that the proposed algorithm outperforms the state-of-the-art algorithms.

Regularization algorithms have been investigated extensively to solve the ill-posed inverse problem of electrical tomography. Sparse regularization algorithms with sparsity constrains have become popular in recent years. The iterative shrinkage thresholding algorithms have been applied to deal with the sparse regularization due to their simplicity and low calculation cost. However, the performance of the reconstructed images varies with the thresholding parameter and initial parameters of the iterative thresholding algorithm, which are selected manually. Inspired by the iterative varied thresholding operator, a fast iterative updated thresholding algorithm is proposed for electrical resistance tomography (ERT) and further a new scheme for updating the thresholding parameter adaptively during the iteration process is designed. More penalty is implemented with a larger thresholding parameter when the sparsity is reduced, and less penalty is implemented with a smaller thresholding parameter when the sparsity is increased. In addition, a speedup step is exploited in order to accelerate the progress. This proposed method is verified quantitatively in numerical simulation as well as in experiment test on a practical ERT system. Moreover, the impacts of different initial parameters are discussed in detailed, the simulation results demonstrate that the proposed method is almost unaffected by different initial parameters. The advantage of this method is that a higher spatial resolution image with a faster solving speed can be reconstructed with less iterations. The results indicate that the quality of images reconstructed by this proposed method outperforms that of traditional methods whether in size or location of the inclusion. It also has a stronger ability in preserving edges and noise immunity. Furthermore, the proposed method can be applied to image reconstruction in other kinds of tomography.

Sparse coding, which aims at finding appropriate sparse representations of data with an overcomplete dictionary set, has become a mature class of methods with good efficiency in various areas, but it faces limitations in immediate processing such as real-time video denoising. Unsupervised deep neural network structured sparse coding (DNN-SC) algorithms can enhance the efficiency of iterative sparse coding algorithms to achieve the goal. In this paper, we first propose a sparse coding algorithm by adding the idea “weighted" in the iterative shrinkage thresholding algorithm (ISTA), named WISTA, which can enjoy the benefit of the l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> norm (0 <; p <; 1) sparsity constraint. Then, we propose two novel DNN-SC algorithms by combining deep learning with WISTA and the iterative half thresholding algorithm (IHTA), which is the l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0.5</sub> norm sparse coding algorithm. Furthermore, we present that by changing the loss function, the DNN can be learned supervisedly and unsupervisedly. Unsupervised learning is the key to ensure the DNN to be learned online during processing, which enables the use of the DNN-SC algorithms in applications lacking labels for signals. Synthetic data experiments show that WISTA can outperform ISTA and IHTA. Moreover, the DNNstructured WISTA can successfully achieve converged results of WISTA. In real-world data experiments, the procedure of utilizing DNN-SC algorithms in image denoising is first presented. All DNN-SC algorithms can accelerate at least 45 times while maintaining PSNR results compared with their corresponding sparse coding algorithms. Finally, the strategy of utilizing DNN-SC algorithms in real-time video denoising is presented. The video-denoising experiments show that the DNN-structured ISTA and WISTA can conduct real-time video denoising for 25 frames/s 360 × 480 pixels gray-scaled videos.

Finding the sparset solution of an underdetermined system of linear equations y=Ax has attracted considerable attention in recent years. Among a large number of algorithms, iterative thresholding algorithms are recognized as one of the most efficient and important classes of algorithms. This is mainly due to their low computational complexities, especially for large scale applications. The aim of this paper is to provide guarantees on the global convergence of a wide class of iterative thresholding algorithms. Since the thresholds of the considered algorithms are set adaptively at each iteration, we call them adaptively iterative thresholding (AIT) algorithms. As the main result, we show that as long as A satisfies a certain coherence property, AIT algorithms can find the correct support set within finite iterations, and then converge to the original sparse solution exponentially fast once the correct support set has been identified. Meanwhile, we also demonstrate that AIT algorithms are robust to the algorithmic parameters. In addition, it should be pointed out that most of the existing iterative thresholding algorithms such as hard, soft, half and smoothly clipped absolute deviation (SCAD) algorithms are included in the class of AIT algorithms studied in this paper.

Multilevel minimum cross entropy thresholding: A comparative study

Seismic inversion is a process of imaging or predicting the spatial and physical properties of underground strata. The most commonly used one is sparse-spike seismic inversion with sparse regularization. There are many effective methods to solve sparse regularization, such as L0-norm, L1-norm, weighted L1-norm, etc. This paper studies the sparse-spike inversion with L0-norm. It is usually solved by the iterative hard thresholding algorithm (IHTA) or its faster variants. However, hard thresholding algorithms often lead to a sharp increase or numerical oscillation of the residual, which will affect the inversion results. In order to deal with this issue, this paper attempts the idea of the relaxed optimal thresholding algorithm (ROTA). In the solution process, due to the particularity of the sparse constraints in this convex relaxation model, this model can be considered as a L1-norm problem when dealt with the location of non-zero elements. We use a modified iterative soft thresholding algorithm (MISTA) to solve it. Hence, it forms a new algorithm called the iterative hybrid thresholding algorithm (IHyTA), which combines IHTA and MISTA. The synthetic and real seismic data tests show that, compared with IHTA, the results of IHyTA are more accurate with the same SNR. IHyTA improves the noise resistance.

In this paper, we propose a Barzilai---Borwein-like iterative half thresholding algorithm for the $$L_{1/2}$$L1/2 regularized problem. The algorithm is closely related to the iterative reweighted minimization algorithm and the iterative half thresholding algorithm. Under mild conditions, we verify that any accumulation point of the sequence of iterates generated by the algorithm is a first-order stationary point of the $$L_{1/2}$$L1/2 regularized problem. We also prove that any accumulation point is a local minimizer of the $$L_{1/2}$$L1/2 regularized problem when additional conditions are satisfied. Furthermore, we show that the worst-case iteration complexity for finding an $$\varepsilon $$? scaled first-order stationary point is $$O(\varepsilon ^{-2})$$O(?-2). Preliminary numerical results show that the proposed algorithm is practically effective.

With the continuous expansion of the market of device-free localization in smart cities, the requirements of device-free localization technology are becoming higher and higher. The large amount of high-dimensional data generated by the existing device-free localization technology will improve the positioning accuracy as well as increase the positioning time and complexity. The positions required from single target to multi-targets become a further increasing difficulty for device-free localization. In order to satisfy the practical localizing application in smart city, an efficient multi-target device-free localization method is proposed based on a sparse coding model. To accelerate the positioning as well as improve the localization accuracy, a sparse coding-based iterative shrinkage threshold algorithm (SC-IA) is proposed and a subspace sparse coding-based iterative shrinkage threshold algorithm (SSC-IA) is presented for different practical application requirements. Experiments with practical dataset are performed for single-target and multi-targets localization, respectively. Compared with three typical machine learning algorithms: deep learning based on auto encoder, K-nearest neighbor, and orthogonal matching pursuit, experimental results show that the proposed sparse coding-based iterative shrinkage threshold algorithm and subspace sparse coding-based iterative shrinkage threshold algorithm can achieve high localization accuracy and low time cost simultaneously, so as to be more practical and applicable for the development of smart city.

We propose iterative thresholding algorithms based on theiterated Tikhonov method for image deblurring problems.Our method is similar in idea to the modified linearized Bregman algorithm (MLBA) sois easy to implement. In order to obtain good restorations, MLBA requires an accurate estimate of the regularization parameter $\alpha$ whichis hard to get in real applications.Based on previous results in iterated Tikhonov method, we designtwo nonstationary iterative thresholding algorithms which give nearoptimal results without estimating $\alpha$. One of them is basedon the iterative soft thresholding algorithm and the other is basedon MLBA. We show that the nonstationary methods,if converge, will converge to the same minimizers ofthe stationary variants.Numerical resultsshow that the accuracy and convergence of our nonstationary methods are very robustwith respect to the changes in the parameters and the restorationresults are comparable to those of MLBA with optimal $\alpha$.

We present a discrepancy-based parameter choice and stopping rule for iterative algorithms performing approximate Tikhonov-functional minimization which adapts the regularization parameter value during the optimization procedure. The suggested parameter choice and stopping rule can be applied to a wide class of penalty terms and iterative algorithms which aim at Tikhonov regularization with a fixed parameter value. It leads, in particular, to computable guaranteed estimates for the regularized exact discrepancy in terms of numerical approximations. Based on these estimates, convergence to a solution is shown. As an example, the developed theory and the algorithm is applied to the case of sparse regularization. We prove order optimal convergence rates in the case of sparse regularization, i.e. weighted ℓp norms, which turn out to be the same as for the a priori parameter choice rule already obtained in the literature as well as for Morozov’s principle applied to exact regularized solutions. Finally, numerical results for two different minimization techniques, iterative soft thresholding algorithm and monotone fast iterative soft thresholding algorithm, are presented, confirming, in particular, the results from the theory.

Terahertz sparse deconvolution based on an iterative shrinkage and thresholding algorithm (ISTA) has been used to characterize multilayered structures with resolution equivalent to or finer than the sampling period of the measurement. However, this method was only studied on thin samples to separate the overlapped echos that can't be distinguished by other deconvolution algorithms. Besides, ISTA heavily depends on the convolution matrix consisting of delayed incident pulse, which is difficult to precisely extricate from the reference signal, and thereby fluctuations caused by noise are occasionally treated as echos. In this work, a terahertz sparse deconvolution based on a learned iterative shrinkage and thresholding algorithm (LISTA) is proposed. The method enclosed the matrix multiplication and soft thresholding in a block and cascaded multiple blocks together to form a deep network. The convolution matrices of the network were updated by stochastic gradient descent to minimize the distance between the output sparse vector and the optimal sparse representation of the signal, and subsequently the trained network made more precise estimation of the echos than ISTA. Additionally, LISTA is notably faster than ISTA, which is important for real-time tomographic-image processing. The algorithm was evaluated on terahertz tomographic imaging of a high-density poly ethylene (HDPE) sample, revealing obvious improvements in detecting defects of different sizes and depths. This technique has potential usage in nondestructive testings of thick samples, where echos reflected by minor defects are not discernible by existed deconvolution algorithms.

Performance analysis for a class of iterative image thresholding algorithms

In recent years, the problem of recovering a low-rank matrix from partial entries, known as low-rank matrix completion problem, has attracted much attention in many applications. However, it is a NP-hard problem due to the nonconvexity nature of the matrix rank function. In this paper, a rank Laplace function is studied to recover the low-rank matrices. Firstly, we propose an iterative Laplace thresholding algorithm to solve the regularized Laplace low-rank matrix completion problem. Secondly, some other iterative thresholding algorithms are designed to recover the low-rank matrices. Finally, we provide a series of numerical simulations to test the proposed algorithms on some low-rank matrix completion and image inpainting problems, and the results show that our algorithms perform better than some state-of-art methods in recovering the low-rank matrices.

Step adaptive fast iterative shrinkage thresholding algorithm for compressively sampled MR imaging reconstruction

In this paper, we consider the problem of finding the sparsest solution to underdetermined linear systems. Unlike the literatures which use the ℓ1 regularization to approximate the original problem, we consider the ℓ2/3 regularization which leads to a better approximation but a nonconvex, nonsmooth, and non-Lipschitz optimization problem. Through developing a fixed point representation theory associated with the two thirds thresholding operator for ℓ2/3 regularization solutions, we propose a fixed point iterative thresholding algorithm based on two thirds norm for solving the k-sparsity problems. Relying on the restricted isometry property, we provide subsequentional convergence guarantee for this fixed point iterative thresholding algorithm on recovering a sparse signal. By discussing the preferred regularization parameters and studying the phase diagram, we get an adequate and efficient algorithm for the high-dimensional sparse signal recovery. Finally, comparing with the existing algorithms, such as the standard ℓ1 minimization, the iterative reweighted ℓ2 minimization, the iterative reweighted ℓ1 minimization, and iterative Half thresholding algorithm, we display the results of the experiment which indicate that the two thirds norm fixed point iterative thresholding algorithm applied to sparse signal recovery and large scale imageries from noisy measurements can be accepted as an effective solver for ℓ2/3 regularization.