Norm Minimization Problem Research Articles

A priori error analysis of high-order LL* (FOSLL*) finite element methods

A number of non-standard finite element methods have been proposed in recent years, each of which derives from a specific class of PDE-constrained norm minimization problems. The most notable examples are LL⁎ methods. In this work, we argue that all high-order methods in this class should be expected to deliver substandard uniform h-refinement convergence rates. In fact, one may not even see rates proportional to the polynomial order p>1 when the exact solution is a constant function. We show that the convergence rate is limited by the regularity of an extraneous Lagrange multiplier variable which naturally appears via a saddle-point analysis. In turn, limited convergence rates appear because the regularity of this Lagrange multiplier is determined, in part, by the geometry of the domain. Numerical experiments support our conclusions.

Weighted truncated nuclear norm regularization for low-rank quaternion matrix completion

In recent years, quaternion matrix completion (QMC) based on low-rank regularization has been gradually used in image processing. Unlike low-rank matrix completion (LRMC) which handles RGB images by recovering each color channel separately, QMC models retain the connection of three channels and process them as a whole. Most of the existing quaternion-based methods formulate low-rank QMC (LRQMC) as a quaternion nuclear norm (a convex relaxation of the rank) minimization problem. The main limitation of these approaches is that they minimize the singular values simultaneously such that cannot approximate low-rank attributes efficiently. To achieve a more accurate low-rank approximation, we introduce a quaternion truncated nuclear norm (QTNN) for LRQMC and utilize the alternating direction method of multipliers (ADMM) to get the optimization in this paper. Further, we propose weights to the residual error quaternion matrix during the update process for accelerating the convergence of the QTNN method with admissible performance. The weighted method utilizes a concise gradient descent strategy which has a theoretical guarantee in optimization. The effectiveness of our method is illustrated by experiments on real visual data sets.

DOA estimation based on sum\u2013difference coarray with virtual array interpolation concept

The sum–difference coarray is the union of difference coarray and the sum coarray, which is capable to obtain a higher number of degrees of freedom (DOF) than the difference coarray. However, this method fails to use all information provided by the coprime array because of the existence of holes. In this paper, we introduce the virtual array interpolation into the sum–difference coarray domain. After interpolating the virtual array, we estimate the DOA by reconstructing the covariance matrix to resolve an atomic norm minimization problem in a gridless way. The proposed method is gridless and can effectively utilize the DOF of a larger virtual array. Numerical simulation results verify the effectiveness and the superior performance of the proposed algorithm.

Inverse Synthetic Aperture Radar Sparse Imaging Exploiting the Group Dictionary Learning

Sparse imaging relies on sparse representations of the target scenes to be imaged. Predefined dictionaries have long been used to transform radar target scenes into sparse domains, but the performance is limited by the artificially designed or existing transforms, e.g., Fourier transform and wavelet transform, which are not optimal for the target scenes to be sparsified. The dictionary learning (DL) technique has been exploited to obtain sparse transforms optimized jointly with the radar imaging problem. Nevertheless, the DL technique is usually implemented in a manner of patch processing, which ignores the relationship between patches, leading to the omission of some feature information during the learning of the sparse transforms. To capture the feature information of the target scenes more accurately, we adopt image patch group (IPG) instead of patch in DL. The IPG is constructed by the patches with similar structures. DL is performed with respect to each IPG, which is termed as group dictionary learning (GDL). The group oriented sparse representation (GOSR) and target image reconstruction are then jointly optimized by solving a l1 norm minimization problem exploiting GOSR, during which a generalized Gaussian distribution hypothesis of radar image reconstruction error is introduced to make the imaging problem tractable. The imaging results using the real ISAR data show that the GDL-based imaging method outperforms the original DL-based imaging method in both imaging quality and computational speed.

Geometric Rescaling Algorithms for Submodular Function Minimization

We present a new class of polynomial-time algorithms for submodular function minimization (SFM) as well as a unified framework to obtain strongly polynomial SFM algorithms. Our algorithms are based on simple iterative methods for the minimum-norm problem, such as the conditional gradient and Fujishige–Wolfe algorithms. We exhibit two techniques to turn simple iterative methods into polynomial-time algorithms. First, we adapt the geometric rescaling technique, which has recently gained attention in linear programming, to SFM and obtain a weakly polynomial bound [Formula: see text]. Second, we exhibit a general combinatorial black box approach to turn [Formula: see text]-approximate SFM oracles into strongly polynomial exact SFM algorithms. This framework can be applied to a wide range of combinatorial and continuous algorithms, including pseudo-polynomial ones. In particular, we can obtain strongly polynomial algorithms by a repeated application of the conditional gradient or of the Fujishige–Wolfe algorithm. Combined with the geometric rescaling technique, the black box approach provides an [Formula: see text] algorithm. Finally, we show that one of the techniques we develop in the paper can also be combined with the cutting-plane method of Lee et al., yielding a simplified variant of their [Formula: see text] algorithm.

Medical Image Denoising Via Matrix Norm Minimization Problems

This paper presents the matrix completion problem for image denoising. Three problems based on matrix norm are performing: Spectral norm minimization problem (SNP), Nuclear norm minimization problem (NNP), and Weighted nuclear norm minimization problem (WNNP). In general, images representing by a matrix this matrix contains the information of the image, some information is irrelevant or unfavorable, so to overcome this unwanted information in the image matrix, information completion is used to comperes the matrix and remove this unwanted information. The unwanted information is handled by defining {0,1}-operator under some threshold. Applying this operator on a given matrix keeps the important information in the image and removing the unwanted information by solving the matrix completion problem that is defined by P. The quadratic programming use to solve the given three norm-based minimization problems. To improve the optimal solution a weighted exponential is used to compute the weighted vector of spectral that use to improve the threshold of optimal low rank that getting from solving the nuclear norm and spectral norm problems. The result of applying the proposed method on different types of images is given by adopting some metrics. The results showed the ability of the given methods.

Fast Universal Low Rank Representation

As well known, low rank representation method (LRR) has obtained promising performance for subspace clustering, and many LRR variants have been developed, which mainly solve the three problems existing in LRR: 1) Problem of mean calculation; 2) Problem of deviating from the real low rank solution; 3) Problem of high computation cost on the large-scale data. In this paper, we first propose a universal LRR method referring to the first two problems. More specifically, we introduce the ability of removing the optimal mean automatically into LRR and extend it to a Schatten <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> -norm minimization problem. Then, referring to the third problem, we reformulate the universal LRR version as an equivalent fast optimization version by removing the null space of data. More importantly, the effective theory proof is proposed to guarantee that the fast optimization method can dramatically improve the algorithmic computation efficiency on the large-scale data but without any loss of information. Finally, experimental results demonstrate the effectiveness and efficiency of the proposed method.

Filter pruning-based two-step feature map reconstruction

In deep neural network compression, channel/filter pruning is widely used for compressing the pre-trained network by judging the redundant channels/filters. In this paper, we propose a two-step filter pruning method to judge the redundant channels/filters layer by layer. The first step is to design a filter selection scheme based on ell _{2,1}-norm by reconstructing the feature map of current layer. More specifically, the filter selection scheme aims to solve a joint ell _{2,1}-norm minimization problem, i.e., both the regularization term and feature map reconstruction error term are constrained by ell _{2,1}-norm. The ell _{2,1}-norm regularization plays a role in the channel/filter selection, while the ell _{2,1}-norm feature map reconstruction error term plays a role in the robust reconstruction. In this way, the proposed filter selection scheme can learn a column-sparse coefficient representation matrix that can indicate the redundancy of filters. Since pruning the redundant filters in current layer might dramatically influence the output feature map of the following layer, the second step needs to update the filters of the following layer to assure output of feature map approximates to that of baseline. Experimental results demonstrate the effectiveness of this proposed method. For example, our pruned VGG-16 on ImageNet achieves 4times speedup with 0.95% top-5 accuracy drop. Our pruned ResNet-50 on ImageNet achieves 2times speedup with 1.56% top-5 accuracy drop. Our pruned MobileNet on ImageNet achieves 2times speedup with 1.20% top-5 accuracy drop.

Inertial algorithm with self-adaptive step size for split common null point and common fixed point problems for multivalued mappings in Banach spaces

ABSTRACT In this article we introduce an inertial iterative scheme with self-adaptive step size for finding a common solution of split common null point problem for a finite family of maximal monotone operators and fixed point problem for a finite family of multivalued demicontractive mappings between a Banach space and Hilbert space. Strong convergence result is obtained for the proposed algorithm. The self-adaptive step size ensures no requirement for a prior knowledge or estimate of the norm of the operator. The inertial term introduced in the algorithm is efficient, it helps to avoid imposing some strong conditions usually used for inertial-type algorithms by many authors. We give some applications of our results to game theory, split equilibrium and minimum-norm problems. Numerical experiment is also presented to demonstrate the efficiency of our proposed method as well as comparing with other existing method in the literature. Our results improve and generalize many well known results in this direction in the literature.

Hybrid Reweighted Optimization Method for Gridless Direction of Arrival Estimation in Heteroscedastic Noise Environment

In this paper, we present a hybrid optimization framework for gridless sparse Direction of Arrival (DoA) estimation under the consideration of heteroscedastic noise scenarios. The key idea of the proposed framework is to combine global and local minima search techniques that offer a sparser optimizer with boosted immunity to noise variation. In particular, we enforce sparsity by means of reformulating the Atomic Norm Minimization (ANM) problem through applying the nonconvex Schatten-p quasi-norm (0&lt;p&lt;1) relaxation. In addition, to enhance the adaptability of the relaxed ANM in more practical noise scenarios, it is combined with a covariance fitting (CF) criterion resulting in a locally convergent reweighted iterative approach. This combination forms a hybrid optimization framework and offers the advantages of both optimization approaches while balancing their drawbacks. Numerical simulations are performed taking into account the configuration of co-prime array (CPA). The simulations have demonstrated that the proposed method can maintain a high estimation resolution even in heteroscedastic noise environments, a low number of snapshots, and correlated sources.

A Fast Sparse Azimuth Super-Resolution Imaging Method of Real Aperture Radar Based on Iterative Reweighted Least Squares With Linear Sketching

It is greatly significant to achieve radar forward-looking region imaging. Due to the limitation of phase ambiguity and small Doppler gradient in forward-looking region, synthetic aperture radar and Doppler beam sharpening cannot work for forward-looking imaging, while real aperture radar (RAR) has arbitrary imaging geometry. Nevertheless, restricted by the antenna aperture, azimuth resolution of RAR is coarse, super-resolution technology is required to improve its azimuth resolution. Exploiting the sparse prior information of the target, the super-resolution problem can be transformed into an $L_1$ norm minimization problem mathematically. Iterative reweighted algorithm can effectively solve the $L_1$ norm minimization problem by replacing $L_1$ norm with reweighted $L_2$ norm and computing the weight in each iteration. However, it suffers from a large computational load due to the repeated multiplications and inversions of large matrices. In this article, a fast azimuth super-resolution imaging method of RAR based on iterative reweighted least squares (IRLS) with linear sketching (LS) was proposed to achieve fast super-resolution imaging of RAR. The LS theory is employed to compress echo matrix and antenna measurement matrix into much smaller matrices via multiplying them by an embedded matrix. Then, the IRLS solver was utilized to address the reconstructed objective function. Much of the expensive computation can then be performed on the smaller matrices, thereby accelerating the algorithm. Simulations and experimental data prove that the proposed algorithm can offer a time complexity reduction without loss of imaging performance.

Least-squares imaging of diffractions by solving a hybrid L1-L2 norm minimization problem

Traditional diffraction images without a specific migration kernel for promoting focusing abilities may cause confusion to seismic interpretation because diffraction images may show a finite-array response of diffracted/scattered waves. Because diffractors are discontinuous and sparsely distributed, a least-squares diffraction-imaging method is formulated by solving a hybrid L1-L2 norm minimization problem that imposes a sparsity constraint on diffraction images. It uses two different forward modeling operators for reflections and diffractions and L2 and L1 regularizations for penalizing the amplitudes of the reflection and diffraction images, respectively. A classic Kirchhoff diffraction demigration operator is implemented on an initial diffraction image model to synthesize diffracted/scattered waves. A Kirchhoff reflection demigration operator, formulated by considering the local reflection slopes and a cosine attenuation weighting function, is implemented on an initial reflection image to synthesize the reflected waves. A modified alternating direction approach of multipliers is developed for iteratively solving this minimization problem to create diffraction images and their separated diffractions. The depths and local reflection slopes of the reflection images are fixed during this iteration. To alleviate the energy leakage between diffractions and reflections, after performing the plane-wave destruction method on the conventional migration data, its estimated reflection image and residual image are provided as the initial reflection and diffraction images, respectively. Our method can remove steep-slope reflections, increase the focusing power of the diffractions, and eliminate noise. Two numerical experiments demonstrate its capability of separating and imaging small-scale discontinuities and inhomogeneities. The exposed geologic structures in the tunnel of field coal mining further illustrate this method’s potential in ascertaining hidden faults, edges, and collapsed columns. A safety warning should be definitely required if a mining working surface is advancing these hidden geologic disasters because an emergency of water bursting or gas leakage may happen.

Coarray Interpolation for DOA Estimation Using Coprime EMVS Array

In this letter, we develop a coarray interpolation method for direction-of-arrival (DOA) estimation using the coprime electromagnetic vector-sensor (EMVS) array. Firstly, we derive the coarray signal model of the coprime EMVS array, which can be viewed as a combination of multiple polarization components. Subsequently, we fill in zero elements in each polarization component and recover the corresponding low-rank covariance matrix by solving a nuclear norm minimization (NNM) problem. By exploiting the recovered covariance matrices, we eventually construct a larger covariance matrix to carry out DOA estimation. Numerical experiment results demonstrate the superiority of the proposed algorithm over conventional methods.

Mathematical Theory of Atomic Norm Denoising in Blind Two-Dimensional Super-Resolution

This paper develops a new mathematical framework for denoising in blind two-dimensional (2D) super-resolution upon using the atomic norm. The framework denoises a signal that consists of a weighted sum of an unknown number of time-delayed and frequency-shifted unknown waveforms from its noisy measurements. Moreover, the framework also provides an approach for estimating the unknown parameters in the signal. We prove that when the number of the observed samples satisfies certain lower bound that is a function of the system parameters, we can estimate the noise-free signal, with very high accuracy, upon solving a regularized least-squares atomic norm minimization problem. We derive the theoretical mean-squared error of the estimator, and we show that it depends on the noise variance, the number of unknown waveforms, the number of samples, and the dimension of the low-dimensional space where the unknown waveforms lie. Finally, we verify the theoretical findings of the paper by using extensive simulation experiments.

DYANOM—Dykstra’s projection based atomic norm solver

In this paper, we propose a fast monotonic algorithm named DYANOM to find the optimal minimizer of the atomic norm minimization problem (ANM), which has extensive applications ranging from spectral estimation, direction of arrival estimation to system identification. DYANOM is faster as well as more accurate than the state-of-the-art methods and enjoys monotonicity property. We present numerical simulations in the context of frequency estimation and compare its numerical performance with the state-of-the-art methods.

MIMO Radar Accurate Imaging and Motion Estimation for 3-D Maneuvering Ship Target

Image deterioration problem occurs in radar imaging for ship target, which results from the complex time-varying motions of ship, the noise in channels, and the clutter on sea surface. It is hard to be solved effectively due to coherent accumulation sampling time and high-dimensional parametric model. Hence, an accurate imaging and motion estimation method based on multiple-input–multiple-out (MIMO) radar is presented. First, the multidimensional signal model is built to characterize target features accurately. To reduce the interference from sea clutters, a preprocessing strategy is exerted based on the space–time adaptive processing (STAP) theory, clutter signals can be suppressed effectually with constructing a Doppler spectrum model. Then, for accurate imaging and motion estimation, a combined trace norm minimization problem is deduced based on the relaxation of tensor rank, where the noise in sea environments is also considered. Meanwhile, generalized tensor total variation constraint is developed to ensure stable estimation and smooth imaging results when separating the noise term. Accordingly, an effective decomposition criterion is formulated based on alternating direction multiplier method (ADMM) strategy, and motion parameters can be precisely calculated based on the least square (LS) method. Finally, theoretical analysis and simulation results present the accurate performance of the proposed method.

Local R-linear convergence of ADMM-based algorithm for ℓ1-norm minimization with linear and box constraints

Abstract This paper presents an efficient algorithm based on the alternating direction method of multipliers (ADMM) for an l 1 -norm minimization problem with linear equality and box constraints. In the ADMM iterations, sub-problems, called proximal minimizations, are solved to obtain the next updating points by exploiting closed formulae. Furthermore, the local R-linear convergence is established by analysis, focusing on the dynamical structure of the ADMM iterations. Numerical examples illustrate obtained theoretical results and the effectiveness of the algorithm.

Labeled-Robust Regression: Simultaneous Data Recovery and Classification.

Rank minimization is widely used to extract low-dimensional subspaces. As a convex relaxation of the rank minimization, the problem of nuclear norm minimization has been attracting widespread attention. However, the standard nuclear norm minimization usually results in overcompression of data in all subspaces and eliminates the discrimination information between different categories of data. To overcome these drawbacks, in this article, we introduce the label information into the nuclear norm minimization problem and propose a labeled-robust principal component analysis (L-RPCA) to realize nuclear norm minimization on multisubspace data. Compared with the standard nuclear norm minimization, our method can effectively utilize the discriminant information in multisubspace rank minimization and avoid excessive elimination of local information and multisubspace characteristics of the data. Then, an effective labeled-robust regression (L-RR) method is proposed to simultaneously recover the data and labels of the observed data. Experiments on real datasets show that our proposed methods are superior to other state-of-the-art methods.

Fast Total Variation Method Based on Iterative Reweighted Norm for Airborne Scanning Radar Super-Resolution Imaging

The total variation (TV) method has been applied to realizing airborne scanning radar super-resolution imaging while maintaining the outline of the target. The iterative reweighted norm (IRN) approach is an algorithm for addressing the minimum Lp norm problem by solving a sequence of minimum weighted L2 norm problems, and has been applied to solving the TV norm. However, during the solving process, the IRN method is required to update the weight term and result term in each iteration, involving multiplications and the inversion of large matrices. Consequently, it suffers from a huge calculation load, which seriously restricts the application of the TV imaging method. In this work, by analyzing the structural characteristics of the matrix involved in iteration, an efficient method based on suitable matrix blocking is proposed. It transforms multiplications and the inversion of large matrices into the computation of multiple small matrices, thereby accelerating the algorithm. The proposed method, called IRN-FTV method, is more time economical than the IRN-TV method, especially for high dimensional observation scenarios. Numerical results illustrate that the proposed IRN-FTV method enjoys preferable computational efficiency without performance degradation.

Doppler Resilient Complementary Waveform Design for Active Sensing

Active sensing systems prefer probing waveforms with good auto-correlation properties. Faced with the difficulty in getting individual sequences with impulse-like aperiodic auto-correlation functions, we resort to the temporal waveform diversity based on complementary sets of sequences (CSS). It seems an ideal solution, but has rarely been used due to the extreme sensitivity of CSS to Doppler shifts. This paper is devoted to applying numerical methods to the design of Doppler resilient complementary waveforms (DRCW) whose ambiguity functions are almost free of range sidelobes along modest Doppler shifts. We formulate the problem as minimizing the worst-case peak sidelobe level (PSL) with low peak-to-average power ratio (PAR) constraints, and then approximate the min-max problem by a ${l}_{p}$ -norm minimization problem. A hierarchical strategy is developed to tackle the phase optimization and the amplitude-phase optimization respectively. Correspondingly, two algorithms based on phase-gradient and Majorization-Minimization are proposed, both of which can be efficiently computed via FFT. Numerical experiments show our method is effective and superior to existing ones. The DRCWs thus obtained have their range sidelobes at all lags suppressed below −60dB, and are expected to bring significant performance improvements in active sensing. Two application examples, detecting weak targets in heavy clutter and estimating the high-resolution range profile of an extended target by active radars, are given.