Soft-thresholding Operator Research Articles

Higher Cheeger ratios of features in Laplace-Beltrami eigenfunctions

This paper investigates links between the eigenvalues and eigenfunctions of the Laplace-Beltrami operator, and the higher Cheeger constants of smooth Riemannian manifolds, possibly weighted and/or with boundary. The higher Cheeger constants give a loose description of the major geometric features of a manifold. We give a constructive upper bound on the higher Cheeger constants, in terms of the eigenvalue of any eigenfunction with the corresponding number of nodal domains. Specifically, we show that for each such eigenfunction, a positive-measure collection of its superlevel sets have their Cheeger ratios bounded above in terms of the corresponding eigenvalue.Some manifolds have their major features entwined across several eigenfunctions, and no single eigenfunction contains all the major features. In this case, there may exist carefully chosen linear combinations of the eigenfunctions, each with large values on a single feature, and small values elsewhere. We can then apply a soft-thresholding operator to these linear combinations to obtain new functions, each supported on a single feature. We show that the Cheeger ratios of the level sets of these functions also give an upper bound on the Laplace-Beltrami eigenvalues. We extend these level set results to nonautonomous dynamical systems, and show that the dynamic Laplacian eigenfunctions reveal sets with small dynamic Cheeger ratios.

Multi-Domain Based Dynamic Graph Representation Learning for EEG Emotion Recognition.

Graph neural networks (GNNs) have demonstrated efficient processing of graph-structured data, making them a promising method for electroencephalogram (EEG) emotion recognition. However, due to dynamic functional connectivity and nonlinear relationships between brain regions, representing EEG as graph data remains a great challenge. To solve this problem, we proposed a multi-domain based graph representation learning (MD 2GRL) framework to model EEG signals as graph data. Specifically, MD 2GRL leverages gated recurrent units (GRU) and power spectral density (PSD) to construct node features of two subgraphs. Subsequently, the self-attention mechanism is adopted to learn the similarity matrix between nodes and fuse it with the intrinsic spatial matrix of EEG to compute the corresponding adjacency matrix. In addition, we introduced a learnable soft thresholding operator to sparsify the adjacency matrix to reduce noise in the graph structure. In the downstream task, we designed a dual-branch GNN and incorporated spatial asymmetry for graph coarsening. We conducted experiments using the publicly available datasets SEED and DEAP, separately for subject-dependent and subject-independent, to evaluate the performance of our model in emotion classification. Experimental results demonstrated that our method achieved state-of-the-art (SOTA) classification performance in both subject-dependent and subject-independent experiments. Furthermore, the visualization analysis of the learned graph structure reveals EEG channel connections that are significantly related to emotion and suppress irrelevant noise. These findings are consistent with established neuroscience research and demonstrate the potential of our approach in comprehending the neural underpinnings of emotion.

Incomplete label distribution learning via label correlation decomposition

Label distribution learning (LDL) has garnered increased attention in recent studies on label ambiguity. However, collecting complete annotations for LDL tasks is often time-consuming and labor-intensive compared to traditional learning paradigms. Therefore, designing effective incomplete LDL algorithms is crucial to broaden LDL’s application scope. In this paper, we propose a novel LDL algorithm, called Incomplete Label Distribution Learning via Label Correlation Decomposition (IncomLDL-LCD), which simultaneously learns label distributions and recovers missing description degrees of labels through label correlations. Specifically, we decompose the label correlation into sparse local label correlation and low-rank global label correlation using a soft-thresholding operator and a singular value thresholding operator, respectively. The former is utilized to capture the related label subsets necessary for reconstructing each possible label, while the latter focuses on extracting the coarse-grained semantic concepts from all labels and exploring the groupings of labels. Additionally, we develop an alternating solution with the accelerated proximal gradient descent method for optimization. Extensive experiments on 16 real-world data sets with varying degrees of missing annotations validate that our algorithm effectively handles incomplete LDL tasks and outperforms state-of-the-art algorithms.

LIASM-NRID: Constructing an atmospheric scattering model for low-light conditions and dehazing nighttime road images

Dehazing of nighttime road images holds significant practical relevance for autonomous driving and intelligent transportation systems operating in nocturnal conditions. To address the limitations of traditional atmospheric scattering models in accurately describing the imaging process in low illumination environments such as nighttime, this study constructs a low illumination environment atmospheric scattering model. Based on this newly constructed model, we propose an algorithm specifically designed for dehazing road images at nighttime. The proposed model incorporates an incident light attenuation factor, an additive noise factor, and a global compensation factor related to the scene's depth. The proposed nighttime road image dehazing algorithm firstly suppresses the interference of additive noise in the original image using wavelet decomposition and soft thresholding operation. Then, a transmittance-solving method containing the primary structural information of the image is designed based on the color attenuation linear model, and the solved transmittance is refined based on the interval gradient image texture structure method. Consequently, the global light map estimation process is formulated as an optimization problem to ensure the accuracy of the atmospheric light value calculation. In the final stages, a novel RGB color equalization method is introduced to address the issue of non-uniform incident light color bias in the dehazed images. Additionally, to eliminate the halo problem in the dehazed images, the adaptive histogram equalization algorithm is modified by incorporating a normalized gamma correction function. The experimental results show that the proposed nighttime road image dehazing algorithm exhibits robust performance and generality across various nighttime scenes. Compared with the representative algorithms for nighttime image dehazing, it can get more excellent dehazing results and has more outstanding performance in all numerical evaluation indexes.

A Designed Thresholding Operator for Low-Rank Matrix Completion

In this paper, a new thresholding operator, namely, designed thresholding operator, is designed to recover the low-rank matrices. With the change of parameter in designed thresholding operator, the designed thresholding operator can apply less bias to the larger singular values of a matrix compared with the classical soft thresholding operator. Based on the designed thresholding operator, an iterative thresholding algorithm for recovering the low-rank matrices is proposed. Numerical experiments on some image inpainting problems show that the proposed thresholding algorithm performs effectively in recovering the low-rank matrices.

Lp quasi-norm minimization: algorithm and applications

Sparsity finds applications in diverse areas such as statistics, machine learning, and signal processing. Computations over sparse structures are less complex compared to their dense counterparts and need less storage. This paper proposes a heuristic method for retrieving sparse approximate solutions of optimization problems via minimizing the ℓp\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _{p}$$\\end{document} quasi-norm, where 0<p<1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$0<p<1$$\\end{document}. An iterative two-block algorithm for minimizing the ℓp\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _{p}$$\\end{document} quasi-norm subject to convex constraints is proposed. The proposed algorithm requires solving for the roots of a scalar degree polynomial as opposed to applying a soft thresholding operator in the case of ℓ1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _{1}$$\\end{document} norm minimization. The algorithm’s merit relies on its ability to solve the ℓp\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _{p}$$\\end{document} quasi-norm minimization subject to any convex constraints set. For the specific case of constraints defined by differentiable functions with Lipschitz continuous gradient, a second, faster algorithm is proposed. Using a proximal gradient step, we mitigate the convex projection step and hence enhance the algorithm’s speed while proving its convergence. We present various applications where the proposed algorithm excels, namely, sparse signal reconstruction, system identification, and matrix completion. The results demonstrate the significant gains obtained by the proposed algorithm compared to other ℓp\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _{p}$$\\end{document} quasi-norm based methods presented in previous literature.

Circuit Implementation of Proximal Projection Neural Networks for Composite Optimization Problems

Existing circuits for composite optimization problems tend to ignore the structure of nonsmooth objectives and lead to less practicability. To this end, with a proximal projection neural network (PPNN) and an inertial proximal projection neural network (IPPNN), this paper presents two novel analog circuit frameworks in presence of a nonsmooth term, where the actions of neuron are simulated via a feedback loop formed by integrator, gradient estimate, proximal operator and other basic operation models. In our circiuts, it shows that the stable output voltages are mapped to optimal solutions. Considering the nonsmooth term as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${{l}_{1}}$</tex-math></inline-formula> -norm, we further design an effective analog circuit of soft thresholding operator. Theoretically, it is proved that both PPNN-based and IPPNN-based analog circuit frameworks have stable output voltages in the sense that the value of electronic components meets a certain mild proportion. Finally, two examples are simulated on Multisim 14.0 platform to validate the effectiveness and practicability of the proposed analog circuits.

High-Similarity-Pass Attention for Single Image Super-Resolution.

Recent developments in the field of non-local attention (NLA) have led to a renewed interest in self-similarity-based single image super-resolution (SISR). Researchers usually use the NLA to explore non-local self-similarity (NSS) in SISR and achieve satisfactory reconstruction results. However, a surprising phenomenon that the reconstruction performance of the standard NLA is similar to that of the NLA with randomly selected regions prompted us to revisit NLA. In this paper, we first analyzed the attention map of the standard NLA from different perspectives and discovered that the resulting probability distribution always has full support for every local feature, which implies a statistical waste of assigning values to irrelevant non-local features, especially for SISR which needs to model long-range dependence with a large number of redundant non-local features. Based on these findings, we introduced a concise yet effective soft thresholding operation to obtain high-similarity-pass attention (HSPA), which is beneficial for generating a more compact and interpretable distribution. Furthermore, we derived some key properties of the soft thresholding operation that enable training our HSPA in an end-to-end manner. The HSPA can be integrated into existing deep SISR models as an efficient general building block. In addition, to demonstrate the effectiveness of the HSPA, we constructed a deep high-similarity-pass attention network (HSPAN) by integrating a few HSPAs in a simple backbone. Extensive experimental results demonstrate that HSPAN outperforms state-of-the-art approaches on both quantitative and qualitative evaluations. Our code and a pre-trained model were uploaded to GitHub (https://github.com/laoyangui/HSPAN) for validation.

ADMM-inspired image reconstruction forterahertz off-axis digital holography.

Image reconstruction in off-axis terahertz digital holography is complicated due to the harsh recording conditions and the non-convexity form of the problem. In this paper, we propose an inverse problem-based reconstruction technique that jointly reconstructs the object field and the amplitude of the reference field. Regularization in the wavelet domain promotes a sparse object solution. A single objective function combining the data-fidelity and regularization terms is optimized with a dedicated algorithm based on an alternating direction method of multipliers framework. Each iteration alternates between two consecutive optimizations using projections operating on each solution and one soft thresholding operator applying to the object solution. The method is preceded by a windowing process to alleviate artifacts due to the mismatch between camera frame truncation and periodic boundary conditions assumed to implement convolution operators. Experiments demonstrate the effectiveness of the proposed method, in particular, improvements of reconstruction quality, compared to two other methods.

Nonparametric quantile scalar-on-image regression

A quantile scalar-on-image regression model is developed to comprehensively study the relationship between cognitive decline and various clinical covariates and imaging factors. As a motivating example, the high-dimensional brain imaging data from the research on Alzheimer's disease are considered predictors of patients' cognitive decline. A Bayesian nonparametric model is proposed to handle the complex spatially distributed imaging data, where the coefficient function is assumed to be a latent Gaussian process. A soft-thresholding operator is introduced to capture the sparse structure of the regression coefficients. Utilizing kernel basis functions to approximate the latent Gaussian process facilitates easy-to-implement computation and consistent estimation. Inference is performed within the Bayesian framework, using an efficient Markov chain Monte Carlo algorithm. The proposed method is compared with the functional principal component analysis method in simulations and applied to a study of Alzheimer's disease.

L1 norm based sparse reconstruction approach to compressed ultrafast photography of velocity interferometer system for any reflector.

The one-dimensional velocity interferometer system for any reflector (VISAR) is widely used in inertial confinement fusion (ICF). Compressed ultrafast photography (CUP) is recently combined with VISAR to enable two-dimensional imaging, in which, the time-varied two-dimensional fringe images reconstruction from one image is a challenging problem. In this Letter, a new l1 norm based sparse reconstruction approach is presented, to the best of our knowledge, in which: (1) fringe images are sparsely represented with a few coefficients over the discrete cosine transformation (DCT) basis; (2) the image reconstruction is formulated as an l1 norm based sparse coefficients optimization problem; and (3) the two-step iterative shrinkage/threshold algorithm combined with a soft-thresholding operator is proposed to efficiently solve such a problem. Finally, the results show that, compared with the usual method, reconstructed fringe images with clear boundaries and good continuity are obtained. Additionally, the maximum relative error of the velocity is reduced from 14% to 8%, which is a reduction of approximately half.

Beyond ℓ1 sparse coding in V1.

Growing evidence indicates that only a sparse subset from a pool of sensory neurons is active for the encoding of visual stimuli at any instant in time. Traditionally, to replicate such biological sparsity, generative models have been using the ℓ1 norm as a penalty due to its convexity, which makes it amenable to fast and simple algorithmic solvers. In this work, we use biological vision as a test-bed and show that the soft thresholding operation associated to the use of the ℓ1 norm is highly suboptimal compared to other functions suited to approximating ℓp with 0 ≤ p < 1 (including recently proposed continuous exact relaxations), in terms of performance. We show that ℓ1 sparsity employs a pool with more neurons, i.e. has a higher degree of overcompleteness, in order to maintain the same reconstruction error as the other methods considered. More specifically, at the same sparsity level, the thresholding algorithm using the ℓ1 norm as a penalty requires a dictionary of ten times more units compared to the proposed approach, where a non-convex continuous relaxation of the ℓ0 pseudo-norm is used, to reconstruct the external stimulus equally well. At a fixed sparsity level, both ℓ0- and ℓ1-based regularization develop units with receptive field (RF) shapes similar to biological neurons in V1 (and a subset of neurons in V2), but ℓ0-based regularization shows approximately five times better reconstruction of the stimulus. Our results in conjunction with recent metabolic findings indicate that for V1 to operate efficiently it should follow a coding regime which uses a regularization that is closer to the ℓ0 pseudo-norm rather than the ℓ1 one, and suggests a similar mode of operation for the sensory cortex in general.

Improved sparse low-rank model via periodic overlapping group shrinkage and truncated nuclear norm for rolling bearing fault diagnosis

The early faults of rolling bearings are the common causes of rotating machinery failures. Rolling bearings with local faults usually generate periodic shocks during operation, but the pulse information is easily masked by a large number of random shocks and noise. To effectively diagnose the early fault information of rolling bearings, a dual-dimensional sparse low-rank (DDSLR) model is proposed in this paper, which can simultaneously extract the sparsity within and across groups and periodic self-similarity of fault signal. In the DDSLR model, a newly developed dimension transformation operator is used to transform the fault signal between one-dimensional vector and low-rank matrix, and the periodic overlapping group shrinkage and truncated nuclear norm are used to improve the traditional sparse low-rank model. In addition, the setting rules of periodic prior and parameters in the DDSLR model are discussed, so that the DDSLR model has certain adaptive ability. Finally, the DDSLR model is proved to be a multi-convex optimization problem, and its solution algorithm is derived by using soft threshold operator and majorization-minimization algorithm under the framework of block coordinate descent method. The results of simulation analysis and experiments show that the proposed DDSLR model has higher fault signal estimation accuracy and better fault feature extraction performance than some classical sparse noise reduction models.

Automated Tire visual inspection based on low rank matrix recovery

Visual inspection is a challenging and widely employed process in industries. In this work, an automated tire visual inspection system is proposed based on low rank matrix recovery. Deep Network is employed to perform texture segmentation which benefits low rank decomposition in both quality and computational efficiency. We propose a dual optimization method to improve convergence speed and matrix sparsity by incorporating the improvement of the soft-threshold shrinkage operator by the weight matrix M. We investigated how incremental multiplier affects the decomposition accuracy and the convergence speed of the algorithm. On this basis, image blocks were decomposed into low-rank matrix and sparse matrix in which defects were separated. Comparative experiments have been performed on our dataset. Experimental results validate the theoretical analysis. The method is promising in false alarm, robustness and running time based on multi-core processor distributed computing. It can be extended to other real-time industrial applications.

Variational mode decomposition for surface and intramuscular EMG signal denoising

Electromyographic signals contaminated with noise during the acquisition process affect the results of follow-up applications such as disease diagnosis, motion recognition, gesture recognition, and human–computer interaction. This paper proposes a denoising technique based on the variational mode decomposition (VMD) for both surface electromyography signals (sEMG) and intramuscular electromyography signals (iEMG). sEMG and iEMG obtained from 5 healthy subjects were first decomposed using VMD into respective variational mode functions (VMFs), then thresholds were set to remove the noise, and finally, the denoised signal was reconstructed. The denoising efficacy of interval thresholding (IT) and iterative interval thresholding (IIT) techniques in combination with SOFT, HARD, and smoothly clipped absolute deviation (SCAD) thresholding operators was quantitatively evaluated by using Signal to Noise Ratio (SNR) and further statistically validated by Friedman test. The results demonstrated that IIT provides better SNR values than IT at all noise levels (P-value < 0.05) for sEMG signals. For iEMG, IIT outperformed IT at 0db and 5db noise levels, but at a noise level of 10db and 15db, IT outperformed IIT. However, the results for the 10db noise level were statistically insignificant. The SOFT thresholding operator outperforms HARD and SCAD at all noise levels for sEMG, as well as iEMG (P-value < 0.05). The study demonstrates that the combination of the IIT thresholding technique with the VMD-based SOFT thresholding operator yields the best denoising results while retaining the original signal characteristics. The proposed method can be used in the fields of disease diagnosis, pattern recognition, and movement classification.

A wonderful triangle in compressed sensing

In order to explore some concrete metric relationship between the ‖x‖1/‖x‖∞ and ‖x‖0, we introduce a wonderful triangle whose sides are composed of ‖x‖0,‖x‖1 and ‖x‖∞ for any non-zero vector x∈Rn by delving into the iterative soft-thresholding operator in this paper. Based on the angle β of the triangle corresponding to the side whose length is ‖x‖∞-‖x‖1/‖x‖0, we demonstrate that the sparsity of signal is generally meaningful within a certain exact interval [1,50] without regard to its dimension. Finally, we construct a feasible algorithm for solving ℓ1/ℓ∞ minimization, which further illustrates that it is effective for sparse recovery.

Multiview Clustering of Adaptive Sparse Representation Based on Coupled P Systems

A multiview clustering (MVC) has been a significant technique to dispose data mining issues. Most of the existing studies on this topic adopt a fixed number of neighbors when constructing the similarity matrix of each view, like single-view clustering. However, this may reduce the clustering effect due to the diversity of multiview data sources. Moreover, most MVC utilizes iterative optimization to obtain clustering results, which consumes a significant amount of time. Therefore, this paper proposes a multiview clustering of adaptive sparse representation based on coupled P system (MVCS-CP) without iteration. The whole algorithm flow runs in the coupled P system. Firstly, the natural neighbor search algorithm without parameters automatically determines the number of neighbors of each view. In turn, manifold learning and sparse representation are employed to construct the similarity matrix, which preserves the internal geometry of the views. Next, a soft thresholding operator is introduced to form the unified graph to gain the clustering results. The experimental results on nine real datasets indicate that the MVCS-CP outperforms other state-of-the-art comparison algorithms.

Varying-coefficient hidden Markov models with zero-effect regions

In psychological, social, behavioral, and medical studies, hidden Markov models (HMMs) have been extensively applied to the simultaneous modeling of longitudinal observations and the underlying dynamic transition process. However, the existing HMMs mainly focus on constant-coefficient HMMs. This study considers a varying-coefficient HMM, which enables simultaneous investigation of the dynamic covariate effects and between-state transitions. Moreover, a soft-thresholding operator is introduced to detect zero-effect regions of the coefficient functions. A full Bayesian approach with a hybird Markov chain Monte Carlo algorithm that combines B-spline approximation and penalization technique is developed for statistical inference. The empirical performance of the propose method is evaluated through simulation studies. An application to a study on the Alzheimer's Disease Neuroimaging Initiative dataset is presented.

Learning soft threshold for sparse reparameterization using gradual projection operators

Deep neural networks (DNNs) have achieved great success in the field of computer vision in recent years. While being high-precision, the characteristic of over-parameterization impedes DNNs from being applied to lightweight devices. To obtain parameter-efficient networks, a large body of work based on uniform sparsity or heuristic non-uniform sparsity techniques has been explored. However, these sparsity techniques offer limited improvement in inference speed (FLOPs) and prediction accuracy. To make further progress, we propose a novel gradual projection operators (GPO) to learn the soft threshold for sparse reparameterization. GPO approaches the soft-threshold operator with a family of projection operators, which progressively reduces the gradient of weights to be pruned during training, to gently learn the pruning thresholds. Experiments on ImageNet show that ResNet-50 with the proposed training algorithm achieves 76.52% top-1 validation accuracy at the sparsity 81.62%, which has merely a 0.5% accuracy gap to its dense counterpart. Additionally, the non-uniform budgets learned by GPO can reduce the FLOPs by up to 10% compared to the state-of-the-arts, which is superior to the popular heuristics methods, thus yielding an effective mechanism for sparse reparameterization.11Our code is publicly available athttps://github.com/RiyaoDong/GPO.

Enhancing underwater image via adaptive color and contrast enhancement, and denoising

Images captured under water are often characterized by low contrast, color distortion, and noise, hindering some visual tasks carried out on it. Despite remarkable breakthrough has been made in recent years, effective and robust enhancement of degraded image remains a challenging problem. To improve the quality of underwater images, we propose a novel scheme by constructing an adaptive color and contrast enhancement, and denoising (ACCE-D) framework. In the proposed framework, Difference of Gaussian (DoG) filter and bilateral filter are respectively employed to decompose the high-frequency and low-frequency components. Benefited from this separation, we utilize soft-thresholding operation to suppress the noise in the high-frequency component. Specially, the low-frequency component is enhanced by using an adaptive color and contrast enhancement (ACCE) strategy. Moreover, we derive a numerical solution for ACCE, and adopt a pyramid-based strategy to accelerate the solving procedure. Both qualitative and quantitative experiments demonstrate that our strategy is effective in color correction, contrast enhancement, and detail revealing. In the quantitative evaluations, by performing on the 890 real-world underwater images from UIEBD, the proposed method obtains 0.65 UCIQE, 1.59 UIQM, 0.81 FDUM, 1.34 PCQI, 0.62 CBPD, and 7.75 entropy scores, achieving average increase of 5% comparing with several state-of-the-art methods. Furthermore, we have verified the utility of our proposed ACCE-D for enhancing other types of degraded scenes, including foggy scene, sandstorm scene and low-light scene.