P-norm Regularization Research Articles

Optimal Error Bounds in the Absence of Constraint Qualifications with Applications to p-Cones and Beyond

We prove tight Hölderian error bounds for all p-cones. Surprisingly, the exponents differ in several ways from those that have been previously conjectured. Moreover, they illuminate p-cones as a curious example of a class of objects that possess properties in three dimensions that they do not in four or more. Using our error bounds, we analyse least squares problems with p-norm regularization, where our results enable us to compute the corresponding Kurdyka–Łojasiewicz exponents for previously inaccessible values of p. Another application is a (relatively) simple proof that most p-cones are neither self-dual nor homogeneous. Our error bounds are obtained under the framework of facial residual functions, and we expand it by establishing for general cones an optimality criterion under which the resulting error bound must be tight. Funding: The second author was partly supported by the Japan Society for the Promotion of Science Grant-in-Aid for Early-Career Scientists [Grants 19K20217, 23K16844] and Grant-in-Aid for Scientific Research [Grant (B)21H03398]. The third author was partly supported by the Hong Kong Research Grants Council [Grant PolyU153000/20p].

Spinet-QSM: model-based deep learning with schatten p-norm regularization for improved quantitative susceptibility mapping.

Quantitative susceptibility mapping (QSM) provides an estimate of the magnetic susceptibility of tissue using magnetic resonance (MR) phase measurements. The tissue magnetic susceptibility (source) from the measured magnetic field distribution/local tissue field (effect) inherent in the MR phase images is estimated by numerically solving the inverse source-effect problem. This study aims to develop an effective model-based deep-learning framework to solve the inverse problem of QSM. This work proposes a Schatten -norm-driven model-based deep learning framework for QSM with a learnable norm parameter to adapt to the data. In contrast to other model-based architectures that enforce the l -norm or l -norm for the denoiser, the proposed approach can enforce any -norm ( ) on a trainable regulariser. The proposed method was compared with deep learning-based approaches, such as QSMnet, and model-based deep learning approaches, such as learned proximal convolutional neural network (LPCNN). Reconstructions performed using 77 imaging volumes with different acquisition protocols and clinical conditions, such as hemorrhage and multiple sclerosis, showed that the proposed approach outperformed existing state-of-the-art methods by a significant margin in terms of quantitative merits. The proposed SpiNet-QSM showed a consistent improvement of at least 5% in terms of the high-frequency error norm (HFEN) and normalized root mean squared error (NRMSE) over other QSM reconstruction methods with limited training data.

Tensorized Label Learning on Anchor Graph

Graph-based multimedia data clustering has attracted much attention due to the impressive clustering performance for arbitrarily shaped multimedia data. However, existing graph-based clustering methods need post-processing to get labels for multimedia data with high computational complexity. Moreover, it is sub-optimal for label learning due to the fact that they exploit the complementary information embedded in data with different types pixel by pixel. To handle these problems, we present a novel label learning model with good interpretability for clustering. To be specific, our model decomposes anchor graph into the products of two matrices with orthogonal non-negative constraint to directly get soft label without any post-processing, which remarkably reduces the computational complexity. To well exploit the complementary information embedded in multimedia data, we introduce tensor Schatten p-norm regularization on the label tensor which is composed of soft labels of multimedia data. The solution can be obtained by iteratively optimizing four decoupled sub-problems, which can be solved more efficiently with good convergence. Experimental results on various datasets demonstrate the efficiency of our model.

Optimal correction of infeasible equations system as Ax + B|x|= b using ℓ p-norm regularization

Optimal correction of an infeasible equations system as Ax + B|x|= b leads into a non-convex fractional problem. In this paper, a regularization method(ℓp-norm, 0 < p < 1), is presented to solve mentioned fractional problem. In this method, the obtained problem can be formulated as a non-convex and nonsmooth optimization problem which is not Lipschitz. The objective function of this problem can be decomposed as a difference of convex functions (DC). For this reason, we use a special smoothing technique based on DC programming. The numerical results obtained for generated problem show high performance and the effectiveness of the proposed method.

Multiview Spectral Clustering With Bipartite Graph.

Multi-view spectral clustering has become appealing due to its good performance in capturing the correlations among all views. However, on one hand, many existing methods usually require a quadratic or cubic complexity for graph construction or eigenvalue decomposition of Laplacian matrix; on the other hand, they are inefficient and unbearable burden to be applied to large scale data sets, which can be easily obtained in the era of big data. Moreover, the existing methods cannot encode the complementary information between adjacency matrices, i.e., similarity graphs of views and the low-rank spatial structure of adjacency matrix of each view. To address these limitations, we develop a novel multi-view spectral clustering model. Our model well encodes the complementary information by Schatten p -norm regularization on the third tensor whose lateral slices are composed of the adjacency matrices of the corresponding views. To further improve the computational efficiency, we leverage anchor graphs of views instead of full adjacency matrices of the corresponding views, and then present a fast model that encodes the complementary information embedded in anchor graphs of views by Schatten p -norm regularization on the tensor bipartite graph. Finally, an efficient alternating algorithm is derived to optimize our model. The constructed sequence was proved to converge to the stationary KKT point. Extensive experimental results indicate that our method has good performance.

Dynamic and reliable subtask tracker with general schatten [formula omitted]-norm regularization

Some multi-task trackers adopt an inaccurate shrink strategy to treat different rank components equally. Thus, their flexibility is vulnerable to some tracking challenges. To resolve this problem, we propose a spatial-aware reliable multi-subtask tracker via weighted Schatten p-norm regularization (SLRT-W), which dynamically chooses the suitable and reliable subset of the whole subtasks for tracking. Its major merits not only assign the flexible weights to different subtask rank components depending on their tracking contribution, but also preserve consistent spatial layout structure and correspondence of layered multi-subtask. Specifically, multiple layered subtasks correspond to different target subregions, they are cooperative and complement. A weighted Schatten p-norm is introduced to adaptively shrink different multi-subtask rank components, and emphasize important components as reliable ones. Then, a structured hyper-graph regularized term simultaneously exploits the intrinsic geometry correspondence among multiple layers of subtasks, and spatial layout structure inside each layer. We devise an alternatively generalized iterated shrinkage method to optimize the multi-subtask Schatten p-norm minimization. Finally, a robust decision-evaluation strategy is developed to choose the reliable multi-subtask tracking combination. Encouraging results on some challenging benchmarks demonstrate the proposed tracker performs favorably in robustness and accuracy, against some state-of-the-art trackers.

SpiNet: A deep neural network for Schatten p-norm regularized medical image reconstruction.

To propose a generic deep learning based medical image reconstruction model (named as SpiNet) that can enforce any Schatten p-norm regularization with 0<p≤2, where the p can be learnt (or fixed) based on the problem at hand. Model-based deep learning architecture for solving inverse problems consists of two parts, a deep learning based denoiser and an iterative data consistency solver. The former has either L2 norm or L1 norm enforced on it, which are convex and can be easily minimized. This work proposes a method to enforce any p norm on the noise prior where 0<p≤2. This is achieved by using Majorization-Minimization algorithm, which upper bounds the cost function with a convex function, thus can be easily minimized. The proposed SpiNet has the capability to work for a fixed p or it can learn p based on the data. The network was tested for solving the inverse problem of reconstructing magnetic resonance (MR) images from undersampled k space data and the results were compared with a popular model-based deep learning architecture MoDL which enforces L2 norm along with other compressive sensing-based algorithms. This comparison between MoDL and proposed SpiNet was performed for undersampling rates (R) of 2×, 4×, 6×, 8×, 12×, 16×, and 20×. Multiple figures of merit such as PSNR, SSIM, and NRMSE were utilized in this comparison. A two-tailed t test was performed for all undersampling rates and for all metrices for proving the superior performance of proposed SpiNet compared to MoDL. For training and testing, the same dataset that was utilized in MoDL implementation was deployed. The results indicate that for all undersampling rates, the proposed SpiNet shows higher PSNR and SSIM and lower NRMSE than MoDL. However, for low undersampling rates of 2× and 4×, there is no significant difference in performance of proposed SpiNet and MoDL in terms of PSNR and NRMSE. This can be expected as the learnt p value is close to 2 (norm enforced by MoDL). For higher undersampling rates ≥6×, SpiNet significantly outperforms MoDL in all metrices with improvement as high as 4dB in PSNR and 0.5 points in SSIM. As deep learning based medical image reconstruction methods are gaining popularity, the proposed SpiNet provides a generic framework to incorporate Schatten p-norm regularization with 0<p≤2 with an added advantage of providing superior performance compared to its counterparts. The proposed SpiNet also has useful addition of Schatten p-norm value in regularization term being automatically chosen based on the available training data.

3D MR image denoising using a modified adaptive high order singular value decomposition method.

Magnetic resonance (MR) images are generally degraded by random noise governed by Rician distributions. In this study, we developed a modified adaptive high order singular value decomposition (HOSVD) method, taking consideration of the nonlocal self-similarity and weighted Schatten p-norm. We extracted 3D cubes from noise images and classified the similar cubes by the Euclidean distance between cubes to construction a fourth-order tensor. Each rank of unfolding matrices was adaptively determined by weighted Schatten p-norm regularization. The latent noise-free 3D MR images can be obtained by an adaptive HOSVD. Denoising experiments were tested on both synthetic and clinical 3D MR images, and the results showed the proposed method outperformed several existing methods for Rician noise removal in 3D MR images.

Image denoising in impulsive noise via weighted Schatten p-norm regularization

Low-rank methods have been widely exploited in image denoising and have shown admirable denoising performance, of which weighted Schatten p-norm minimization (WSNM) is particularly effective. However, the WSNM method which applies Frobenius-norm loss model cannot obtain a satisfactory denoising performance when images corrupted by impulse noise. An optimization strategy based on the alternating direction method of multipliers framework is used to solve the proposed model efficiently. Experimental results show that the proposed method outperforms some state-of-the-art denoising methods both quantitatively and qualitatively under various impulsive noise models.

Robust principal component analysis via optimal mean by joint ℓ2,1 and Schatten p-norms minimization

Abstract Since principal component analysis (PCA) is sensitive to corrupted variables or observations that affect its performance and applicability in real scenarios, some convex robust PCA methods have been developed to enhance the robustness of PCA. However, most of them neglect the optimal mean calculation problem. They center the data with the mean calculated by the l2-norm, which is incorrect because the l1-norm objective function is used in the following steps. In this paper, we consider a novel robust PCA method that can pursue and remove outliers, exactly recover a low-rank matrix and calculate the optimal mean. Specifically, we propose an optimization model constituted by a l2,1-norm based loss function and a Schatten p-norm regularization term. The l2,1-norm used in loss function aims to pursue and remove outliers, the Schatten p-norm can suppress the singular values of reconstructed data at smaller p (0

Robust feature selection via simultaneous sapped norm and sparse regularizer minimization

Abstract High dimension is one of the key characters of big data. Feature selection, as a framework to identify a small subset of illustrative and discriminative features, has been proved as a basic solution in dealing with high-dimensional data. In previous literatures, l2, p-norm regularization was studied by many researches as an effective approach to select features across data sets with sparsity. However, l2, p-norm loss function is just robust to noise but not considering the influence of outliers. In this paper, we propose a new robust and efficient feature selection method with emphasizing Simultaneous Capped l2-norm loss and l2, p-norm regularizer Minimization (SCM). The capped l2-norm based loss function can effectively eliminate the influence of noise and outliers in regression and the l2, p-norm regularization is used to select features across data sets with joint sparsity. An efficient approach is then introduced with proved convergence. Extensive experimental studies on synthetic and real-world datasets demonstrate the effectiveness of our method in comparison with other popular feature selection methods.

Multi-label feature selection with missing labels

The consistently increasing of the feature dimension brings about great time complexity and storage burden for multi-label learning. Numerous multi-label feature selection techniques are developed to alleviate the effect of high-dimensionality. The existing multi-label feature selection algorithms assume that the labels of the training data are complete. However, this assumption does not always hold true for labeling data is costly and there is ambiguity among classes. Hence, in real-world applications, the data available usually have an incomplete set of labels. In this paper, we present a novel multi-label feature selection model under the circumstance of missing labels. With the proposed algorithm, the most discriminative features are selected and missing labels are recovered simultaneously. To remove the irrelevant and noisy features, the effective l2, p-norm (0 < p ≤ 1) regularization item is imposed on the feature selection matrix. To solve the optimization problem, we developed an iterative reweighted least squares (IRLS) algorithm with guaranteed convergence. Experimental results on benchmark datasets show that the proposed method outperforms the state-of-the-art multi-label feature selection algorithms.

Non-integer norm regularization SVM via Legendre–Fenchel duality

Support vector machine is an effective classification and regression method that uses VC theory of large margin to maximize the predictive accuracy while avoiding over-fitting of data. L2-norm regularization has been commonly used. If the training data set contains many noise features, L1-norm regularization SVM will provide a better performance. However, both L1-norm and L2-norm are not the optimal regularization method when handling a large number of redundant features and only a small amount of data points are useful for machine learning. We have therefore proposed an adaptive learning algorithm using the p-norm regularization SVM for 0<p≤2. Leveraging on the theory of Legendre–Fenchel duality, we derive a variational quadratic upper bound of non-differentiable non-convex Lp-norm regularized term when 0<p≤1. Generalization error bounds for non-integer norm regularization SVM were provided. Five cancer data sets from public data banks were used for the evaluation. All five evaluations empirically showed that the new adaptive algorithm was able to achieve the optimal prediction error using a less than L1 norm. On the seven different data sets having different sizes and different application domains, our approach was evaluated and compared to current state-of-the-art L1-norm and L2-norm SVM, repeatedly demonstrating that proposed method substantially improved performance. Moreover, we observed that the proposed p-norm penalty is more robust to noise features than the L1-norm and L2-norm penalties.