Non-convex Method Research Articles

Metafrontier slack-based nonparametric technologies with convex and nonconvex measurement of directional distance functions: empirical exploration of high-tech industries

ABSTRACT High-tech industries make an essential contribution to improving innovation ability and promoting industrial structure optimisation. It is essential to examine the development level and regional differences within China's high-tech industry to enhance the quality of economic development. This paper develops a metafrontier slack-based nonparametric method of the directional distance function to perform an efficiency analysis. Distinguishing the development differences of regional high-tech industries and identifying the main causes of inefficiency sources. We are the first to combine convex and nonconvex technologies to calculate group and meta frontier efficiencies, and the technical gap ratio for the high-tech industries. Empirical results indicate that the nonconvex method can provide more reasonable explanations for green innovation efficiency.

Tensor recovery based on rank adaptive and non-convex methods

The growing popularity of tensor singular value decomposition (T-SVD) in tensor recovery problems and the N-tubal rank can be applied to higher order tensors. However, this method faces new challenges as it does not fully use the rank prior information of tensors and lacks non-convex relaxation methods. Thus, this paper applies a new non-convex function, called Minimax Logarithmic Concave Penalty (MLCP), based on the N-tubal rank method. Two MLCP models based on N-tubal rank are introduced for solving low-rank tensor completion (LRTC) and tensor robust principal component analysis (TRPCA) problems, along with corresponding solving algorithms. The effectiveness and superiority of these models are demonstrated through experiments on real datasets.

A Generalized Non-convex Method for Robust Tensor Completion

A Generalized Non-convex Method for Robust Tensor Completion

4D iRIOM: 4D Imaging Radar Inertial Odometry and Mapping

Millimeter wave radar can measure distances, directions, and Doppler velocity for objects in harsh conditions such as fog. The 4D imaging radar with both vertical and horizontal data resembling an image can also measure objects' height. Previous studies have used 3D radars for ego-motion estimation. But few methods leveraged the rich data of imaging radars, and they usually omitted the mapping aspect, thus leading to inferior odometry accuracy. This paper presents a real-time imaging radar inertial odometry and mapping method, iRIOM, based on the submap concept. To deal with moving objects and multipath reflections, we use the graduated non-convexity method to robustly and efficiently estimate ego-velocity from a single scan. To measure the agreement between sparse non-repetitive radar scan points and submap points, the distribution-to-multi-distribution distance for matches is adopted. The ego-velocity, scan-to-submap matches are fused with the 6D inertial data by an iterative extended Kalman filter to get the platform's 3D position and orientation. A loop closure module is also developed to curb the odometry module's drift. To our knowledge, iRIOM based on the two modules is the first 4D radar inertial SLAM system. On our and third-party data, we show iRIOM's favorable odometry accuracy and mapping consistency against the FastLIO-SLAM and the EKFRIO. Also, the ablation study reveal the benefit of inertial data versus the constant velocity model, and scan-to-submap matching versus scan-to-scan matching.

Fast Quantum State Reconstruction via Accelerated Non-Convex Programming

We propose a new quantum state reconstruction method that combines ideas from compressed sensing, non-convex optimization, and acceleration methods. The algorithm, called Momentum-Inspired Factored Gradient Descent (MiFGD), extends the applicability of quantum tomography for larger systems. Despite being a non-convex method, MiFGD converges provably close to the true density matrix at an accelerated linear rate asymptotically in the absence of experimental and statistical noise, under common assumptions. With this manuscript, we present the method, prove its convergence property and provide the Frobenius norm bound guarantees with respect to the true density matrix. From a practical point of view, we benchmark the algorithm performance with respect to other existing methods, in both synthetic and real (noisy) experiments, performed on the IBM’s quantum processing unit. We find that the proposed algorithm performs orders of magnitude faster than the state-of-the-art approaches, with similar or better accuracy. In both synthetic and real experiments, we observed accurate and robust reconstruction, despite the presence of experimental and statistical noise in the tomographic data. Finally, we provide a ready-to-use code for state tomography of multi-qubit systems.

Set-valued estimators for Uncertain Linear Parameter-Varying systems

In this paper, we tackle the problem of state estimation for uncertain linear systems when bounds are known for the disturbances, noise and initial state. Practical systems often have parameters that cannot be measured precisely at every iteration. The framework of Uncertain Linear Parameter-Varying systems (Uncertain LPVs) have attracted attention from the community and have seen applications from the aerospace industry to mechatronic systems, among many other examples. By formulating the problem as the solution of a feasibility program, we show that the optimal convex solution can be computed through an enumeration of the vertices of the estimates. Resorting to this result, three algorithms are proposed: an approximation algorithm using only set operations; an exact convex hull method returning the optimal convex set suitable for cases where estimates do not have a large number of vertices; and an event-triggering algorithm suitable for fault/attack detection that combines both the convex and nonconvex methods. Simulations are conducted using a motor speed model where some of the parameters cannot be measured exactly pointing out that the uncertainty matrices are responsible for the accuracy of the approximation algorithm, and also that the point-based method is suitable for online estimation.

A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems

Constructing surrogate models for uncertainty quantification (UQ) on complex partial differential equations (PDEs) having inherently high-dimensional O(10n), n≥2, stochastic inputs (e.g., forcing terms, boundary conditions, initial conditions) poses tremendous challenges. The “curse of dimensionality” can be addressed with suitable unsupervised learning techniques used as a pre-processing tool to encode inputs onto lower-dimensional subspaces while retaining its structural information and meaningful properties. In this work, we review and investigate thirteen dimension reduction methods including linear and nonlinear, spectral, blind source separation, convex and non-convex methods and utilize the resulting embeddings to construct a mapping to quantities of interest via polynomial chaos expansions (PCE). We refer to the general proposed approach as manifold PCE (m-PCE), where manifold corresponds to the latent space resulting from any of the studied dimension reduction methods. To investigate the capabilities and limitations of these methods we conduct numerical tests for three physics-based systems (treated as black-boxes) having high-dimensional stochastic inputs of varying complexity modeled as both Gaussian and non-Gaussian random fields to investigate the effect of the intrinsic dimensionality of input data. We demonstrate both the advantages and limitations of the unsupervised learning methods and we conclude that a suitable m-PCE model provides a cost-effective approach compared to alternative algorithms proposed in the literature, including recently proposed expensive deep neural network-based surrogates and can be readily applied for high-dimensional UQ in stochastic PDEs.

Topology Optimization of Graded Truss Lattices Based on On-the-Fly Homogenization

Abstract We introduce a computational framework for the topology optimization of cellular structures with spatially varying architecture, which is applied to functionally graded truss lattices under quasistatic loading. We make use of a first-order homogenization approach, which replaces the discrete truss by an effective continuum description to be treated by finite elements in a macroscale boundary value problem. By defining the local truss architecture through a set of Bravais vectors, we formulate the optimization problem with regards to the spatially varying basis vectors and demonstrate its feasibility and performance through a series of benchmark problems in 2D (though the method is sufficiently general to also apply in 3D, as discussed). Both the displacement field and the topology are continuously varying unknown fields on the macroscale, and a regularization is included for well posedness. We argue that prior solutions obtained from aligning trusses along the directions of principal stresses are included as a special case. The outlined approach results in heterogeneous truss architectures with a smoothly varying unit cell, enabling easy fabrication with a tunable length scale (the latter avoiding the ill-posedness stemming from classical nonconvex methods without an intrinsic length scale).

Stability analysis of delayed neural network based on the convex method and the non-convex method

This paper investigates the asymptotic stability of neural network with time-varying delay. A new Lyapunov–Krasovskii functional (LKF) is constructed with some non-integral delay-product functional terms. Some parameter matrices in the LKF are not required to be positive definite. The convex method and the non-convex method are proposed to deal with the square of the time-varying delay appearing in the derivative of the LKF, respectively. Two less conservative delay-dependent stability criteria in the form of linear matrix inequalities (LMIs) are established. Finally, two widely used numerical examples are given to illustrate the validity and superiority of the obtained stability criteria.

A Nonconvex Method to Low-Rank Matrix Completion

In recent years, the problem of recovering a low-rank matrix from partial entries, known as low-rank matrix completion problem, has attracted much attention in many applications. However, it is a NP-hard problem due to the nonconvexity nature of the matrix rank function. In this paper, a rank Laplace function is studied to recover the low-rank matrices. Firstly, we propose an iterative Laplace thresholding algorithm to solve the regularized Laplace low-rank matrix completion problem. Secondly, some other iterative thresholding algorithms are designed to recover the low-rank matrices. Finally, we provide a series of numerical simulations to test the proposed algorithms on some low-rank matrix completion and image inpainting problems, and the results show that our algorithms perform better than some state-of-art methods in recovering the low-rank matrices.

A Unified Framework for Constructing Nonconvex Regularizations

Over the past decades, many individual nonconvex methods have been proposed to achieve better sparse recovery performance in various scenarios. However, how to construct a valid nonconvex regularization function remains open in practice. In this paper, we fill in this gap by presenting a unified framework for constructing the nonconvex regularization based on the probability density function. Meanwhile, a new nonconvex sparse recovery method constructed via the Weibull distribution is studied.

Analysis of the ratio of ℓ1 and ℓ2 norms in compressed sensing

We study the ratio of ℓ1 and ℓ2 norms (ℓ1/ℓ2) as a sparsity-promoting objective in compressed sensing. We first propose a novel criterion that guarantees that an s-sparse signal is the local minimizer of the ℓ1/ℓ2 objective; our criterion is interpretable and useful in practice. We also give the first uniform recovery condition using a geometric characterization of the null space of the measurement matrix, and show that this condition is satisfied for a class of random matrices. We also present analysis on the robustness of the procedure when noise pollutes data. Numerical experiments are provided that compare ℓ1/ℓ2 with some other popular non-convex methods in compressed sensing. Finally, we propose a novel initialization approach to accelerate the numerical optimization procedure. We call this initialization approach support selection, and we demonstrate that it empirically improves the performance of existing ℓ1/ℓ2 algorithms.

Impact force reconstruction and localization using nonconvex overlapping group sparsity

Although extensively studied, impact force identification is still a challenging task. When the location of the impact force is unknown, an under-determined problem is usually required to be tackled. In this paper, a novel impact force identification method based on the nonconvex overlapping group sparsity(NOGS) is proposed, allowing to localize the impact and recover its time history simultaneously from quite limited measurements(i.e., the number of responses is less than the number of potential impact locations). The NOGS not only enriches the prior information by taking the group sparsity structure of impact forces into consideration, but enhances the sparsity and the accuracy of estimated amplitude via its nonconvexity. A new algorithm, named fast nonconvex overlapping group sparsity algorithm(FaNogSa), derived in the light of the Majorize-Minimization(MM) principle is utilized to minimize the nonconvex objective function. Simulations and experiments are both implemented systematically on a stiffened composite structure to validate the proposed method, and two strain gauges are utilized to monitor 54 potential impacts. The corresponding results, comparing to the plain nonconvex(atan) regularization and the standard ℓ1-norm regularization, say that the proposed method is able to localize the impact and at the same time recover its time history accurately, while under the same measuring conditions the nonconvex(atan) method and the ℓ1-norm method usually fail.

An Adaptive Proximal Bundle Method with Inexact Oracles for a Class of Nonconvex and Nonsmooth Composite Optimization

In this paper, an adaptive proximal bundle method is proposed for a class of nonconvex and nonsmooth composite problems with inexact information. The composite problems are the sum of a finite convex function with inexact information and a nonconvex function. For the nonconvex function, we design the convexification technique and ensure the linearization errors of its augment function to be nonnegative. Then, the sum of the convex function and the augment function is regarded as an approximate function to the primal problem. For the approximate function, we adopt a disaggregate strategy and regard the sum of cutting plane models of the convex function and the augment function as a cutting plane model for the approximate function. Then, we give the adaptive nonconvex proximal bundle method. Meanwhile, for the convex function with inexact information, we utilize the noise management strategy and update the proximal parameter to reduce the influence of inexact information. The method can obtain an approximate solution. Two polynomial functions and six DC problems are referred to in the numerical experiment. The preliminary numerical results show that our algorithm is effective and reliable.

Scalable Incremental Nonconvex Optimization Approach for Phase Retrieval

We aim to find a solution \({\varvec{x}}\in {\mathbb {C}}^n\) to a system of quadratic equations of the form \(b_i=|{\varvec{a}}_i^*{\varvec{x}}|^2\), \(i=1,2,\ldots ,m\), e.g., the well-known NP-hard phase retrieval problem. As opposed to recently proposed state-of-the-art nonconvex methods, we revert to the semidefinite relaxation (SDR) PhaseLift convex formulation and propose a successive and incremental nonconvex optimization algorithm, termed as IncrePR, to indirectly minimize the resulting convex problem on the cone of positive semidefinite matrices. Our proposed method overcomes the excessive computational cost of typical SDP solvers as well as the need of a good initialization for typical nonconvex methods. For Gaussian measurements, which is usually needed for provable convergence of nonconvex methods, restart-IncrePR solving three consecutive PhaseLift problems outperforms state-of-the-art nonconvex gradient flow based solvers with a sharper phase transition of perfect recovery and typical convex solvers in terms of computational cost and storage. For more challenging structured (non-Gaussian) measurements often occurred in real applications, such as transmission matrix and oversampling Fourier transform, IncrePR with several consecutive repeats can be used to find a good initial guess. With further refinement by local nonconvex solvers, one can achieve a better solution than that obtained by applying nonconvex gradient flow based solvers directly when the number of measurements is relatively small. Extensive numerical tests are performed to demonstrate the effectiveness of the proposed method.

Generalized Nonconvex Approach for Low-Tubal-Rank Tensor Recovery.

The tensor-tensor product-induced tensor nuclear norm (t-TNN) (Lu et al., 2020) minimization for low-tubal-rank tensor recovery attracts broad attention recently. However, minimizing the t-TNN faces some drawbacks. For example, the obtained solution could be suboptimal to the original problem due to its loose approximation. In this article, we extract a unified nonconvex surrogate of the tensor tubal rank as a tighter regularizer, which involves many popular nonconvex penalty functions. An iterative reweighted t-TNN algorithm is proposed to solve the resulting generalized nonconvex tubal rank minimization for tensor recovery. It converges to a critical point globally with rigorous proofs based on the Kurdyka-Łojasiwicz property. Furthermore, we provide the theoretical guarantees for exact and robust recovery by developing the tensor null space property. Extensive experiments demonstrate that our approach markedly enhances recovery performance compared with several state-of-the-art convex and nonconvex methods.

On the Burer–Monteiro method for general semidefinite programs

Consider a semidefinite program involving an \(n\times n\) positive semidefinite matrix X. The Burer–Monteiro method uses the substitution \(X=Y Y^T\) to obtain a nonconvex optimization problem in terms of an \(n\times p\) matrix Y. Boumal et al. showed that this nonconvex method provably solves equality-constrained semidefinite programs with a generic cost matrix when \(p > rsim \sqrt{2m}\), where m is the number of constraints. In this note we extend their result to arbitrary semidefinite programs, possibly involving inequalities or multiple semidefinite constraints. We derive similar guarantees for a fixed cost matrix and generic constraints. We illustrate applications to matrix sensing and integer quadratic minimization.

HSI Denoising: Global Non-Location Low-Rank Factorization and Sparse Factorization Based on Iteratively Boost Method

This study proposes global and non-location low-rank factorization and sparse factorization based on iteratively boost (GNS-ITboost) scheme from the perspective of the iteration method to improve Hyperspectral image (HSI) denoising results. First, this study establishes a novel iterative boost method by jointing global low-rank factorization, non-location low-rank factorization and sparse factorization to HSI denoising. Second, we propose the non-convex method, weight schatten p-norm method to more effectively approximate the rank function to obtain better non-location low-rank factorization. Third, this study provides a new noise estimation method by prior knowledge to accelerate obtaining a better denoising effect. Finally, proximal alternating minimization and weight sparse ideas are used to obtain the solution of denoising optimization problems. Furmore, evaluation results indicate that the proposed method can outperform the state-of-the-art methods in terms of FastHyDe, GLF, LRMR, SNLRSF, SSTV.

Fast non-convex low-rank matrix decomposition for separation of potential field data using minimal memory

A fast non-convex low-rank matrix decomposition method for potential field data separation is proposed. The singular value decomposition of the large size trajectory matrix, which is also a block Hankel matrix, is obtained using a fast randomized singular value decomposition algorithm in which fast block Hankel matrix-vector multiplications are implemented with minimal memory storage. This fast block Hankel matrix randomized singular value decomposition algorithm is integrated into the \texttt{Altproj} algorithm, which is a standard non-convex method for solving the robust principal component analysis optimization problem. The improved algorithm avoids the construction of the trajectory matrix. Hence, gravity and magnetic data matrices of large size can be computed. Moreover, it is more efficient than the traditional low-rank matrix decomposition method, which is based on the use of an inexact augmented Lagrange multiplier algorithm. The presented algorithm is also robust and, hence, algorithm-dependent parameters are easily determined. The improved and traditional algorithms are contrasted for the separation of synthetic gravity and magnetic data matrices of different sizes. The presented results demonstrate that the improved algorithm is not only computationally more efficient but it is also more accurate. Moreover, it is possible to solve far larger problems. As an example, for the adopted computational environment, matrices of sizes larger than $205 \times 205$ generate "out of memory" exceptions with the traditional method, but a matrix of size $2001\times 2001$ can be calculated in $1062.29$s with the new algorithm. Finally, the improved method is applied to separate real gravity and magnetic data in the Tongling area, Anhui province, China. Areas which may exhibit mineralizations are inferred based on the separated anomalies.

Downlink channel estimation for millimeter wave communication combining low-rank and sparse structure characteristics

The acquisition of channel state information (CSI) is essential in millimeter wave (mmWave) multiple-input multiple-output (MIMO) systems. The mmWave channel exhibits sparse scattering characteristics and a meaningful low-rank structure, which can be simultaneously employed to reduce the complexity of channel estimation. Most existing works recover the low-rank structure of channels using nuclear norm theory. However, solving the nuclear norm-based convex problem often leads to a suboptimal solution of the rank minimization problem, thus degrading the accuracy of channel estimation. Previous contributions recover the channel using over-complete dictionary with the assumption that the mmWave channel can be sparsely represented under some dictionary. While over-complete dictionary may increase the computational complexity. To address these problems, we propose a channel estimation framework based on non-convex low-rank approximation and dictionary learning by exploring the joint low-rank and sparse representations of wireless channels. We surrogate the widely used nuclear norm theory with non-convex low-rank approximation method and design a dictionary learning algorithm based on channel feature classification employing deep neural network (DNN). Our simulation results reveal the proposed scheme outperform the conventional dictionary learning algorithm, Bayesian framework algorithm, and compressed sensing-based algorithms.