Minimum Norm Research Articles

Solving convex optimization problems via a second order dynamical system with implicit Hessian damping and Tikhonov regularization

AbstractThis paper deals with a second order dynamical system with a Tikhonov regularization term in connection to the minimization problem of a convex Fréchet differentiable function. The fact that beside the asymptotically vanishing damping we also consider an implicit Hessian driven damping in the dynamical system under study allows us, via straightforward explicit discretization, to obtain inertial algorithms of gradient type. We show that the value of the objective function in a generated trajectory converges rapidly to the global minimum of the objective function and depending the Tikhonov regularization parameter the generated trajectory converges weakly to a minimizer of the objective function or the generated trajectory converges strongly to the element of minimal norm from the $$\mathop {\text {argmin}}$$ argmin set of the objective function. We also obtain the fast convergence of the velocities towards zero and some integral estimates. Our analysis reveals that the Tikhonov regularization parameter and the damping parameters are strongly correlated, there is a setting of the parameters that separates the cases when weak convergence of the trajectories to a minimizer and strong convergence of the trajectories to the minimal norm minimizer can be obtained.

An efficient infinity norm minimization algorithm for under-determined inverse problems

The problem of solving under-determined systems of linear equations with minimum peak magnitude (ℓ∞ norm) has numerous applications in signal processing. These include Peak-to-Average Power Ratio (PAPR) reduction in MIMO-OFDM systems, vector quantization, approximate nearest neighbor search, optimal control in robotics, and power grid optimization. Several methods have been proposed to address this problem, but they often face limitations in computational speed or representation accuracy. Some methods also impose constraints on the frame matrix, such as restrictions on the type of its entries or its aspect ratio. In this paper, we present the Fast Iterative Peak Shrinkage Algorithm (FIPSA), which iterates over feasible solutions to consistently reduce peak magnitude and provably converge to near-optimal solutions. Our experimental results, conducted across various frame matrix types and aspect ratios, demonstrate that FIPSA consistently achieves near-minimal ℓ∞ norm values. In addition, it operates 1.3 to 7.3 times faster than previous methods, while maintaining an average representation error of 10−15. Notably, these advancements are achieved without imposing any constraints on the frame matrix.

Research on ECG signal reconstruction based on improved weighted nuclear norm minimization and approximate message passing algorithm.

In order to improve the energy efficiency of wearable devices, it is necessary to compress and reconstruct the collected electrocardiogram data. The compressed data may be mixed with noise during the transmission process. The denoising-based approximate message passing (AMP) algorithm performs well in reconstructing noisy signals, so the denoising-based AMP algorithm is introduced into electrocardiogram signal reconstruction. The weighted nuclear norm minimization algorithm (WNNM) uses the low-rank characteristics of similar signal blocks for denoising, and averages the signal blocks after low-rank decomposition to obtain the final denoised signal. However, under the influence of noise, there may be errors in searching for similar blocks, resulting in dissimilar signal blocks being grouped together, affecting the denoising effect. Based on this, this paper improves the WNNM algorithm and proposes to use weighted averaging instead of direct averaging for the signal blocks after low-rank decomposition in the denoising process, and validating its effectiveness on electrocardiogram signals. Experimental results demonstrate that the IWNNM-AMP algorithm achieves the best reconstruction performance under different compression ratios and noise conditions, obtaining the lowest PRD and RMSE values. Compared with the WNNM-AMP algorithm, the PRD value is reduced by 0.17∼4.56, the P-SNR value is improved by 0.12∼2.70.

Stabilization of a matrix via a low-rank-adaptive ODE

Let A be a square matrix with a given structure (e.g. real matrix, sparsity pattern, Toeplitz structure, etc.) and assume that it is unstable, i.e. at least one of its eigenvalues lies in the complex right half-plane. The problem of stabilizing A consists in the computation of a matrix B, whose eigenvalues have all negative real part and such that the perturbation Δ=B-A\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\Delta =B-A$$\\end{document} has minimal norm. The structured stabilization further requires that the perturbation preserves the structural pattern of A. This non-convex problem is solved by a two-level procedure which involves the computation of the stationary points of a matrix ODE. It is possible to exploit the underlying low-rank features of the problem by using an adaptive-rank integrator that follows rigidly the rank of the solution. Some benefits derived from the low-rank setting are shown in several numerical examples. These computational advantages also allow to deal with high dimensional problems.

Absolute testing of optical flats using minimum norm least squares solutions

A laser Fizeau interferometer measures surface form by comparing the wavefronts reflected from a reference surface and the surface under test. The measurement accuracy is limited by the unknown absolute form error of the reference flat. Absolute testing offers a strategy to break through the limitation. In this work, we propose a three-flat absolute testing method using minimum norm least squares solutions (MNLS). After acquiring three measurement sets from pairwise combinations among the flats, one flat is rotated by 90° or 180° for the fourth measurement. The simulation demonstrates the effectiveness of the proposed MNLS method and its robustness against rotational angle errors. Our absolute testing method is experimentally validated using a home-built and a commercial Fizeau interferometers. Correcting the absolute forms of the home-built Fizeau interferometer’s reference flat improves its measurement accuracy. The experimental outcomes of the MNLS absolute testing technique are compared with two alternative absolute testing methods. The MNLS method’s advantage is demonstrated.

Neural-Rendezvous: Provably Robust Guidance and Control to Encounter Interstellar Objects

Interstellar objects (ISOs) are likely representatives of primitive materials invaluable in understanding exoplanetary star systems. Due to their poorly constrained orbits with generally high inclinations and relative velocities, however, exploring ISOs with conventional human-in-the-loop approaches is significantly challenging. This paper presents Neural-Rendezvous—a deep-learning-based guidance and control framework for encountering fast-moving objects, including ISOs, robustly, accurately, and autonomously in real time. It uses pointwise minimum norm tracking control on top of a guidance policy modeled by a spectrally normalized deep neural network, where its hyperparameters are tuned with a loss function directly penalizing the model predictive control state trajectory tracking error. We show that Neural-Rendezvous provides a high probability exponential bound on the expected spacecraft delivery error, the proof of which leverages stochastic incremental stability analysis. In particular, it is used to construct a non-negative function with a supermartingale property, explicitly accounting for the ISO state uncertainty and the local nature of nonlinear state estimation guarantees. In numerical simulations, Neural-Rendezvous is demonstrated to satisfy the expected error bound for 100 ISO candidates. This performance is also empirically validated using our spacecraft simulator and in high-conflict and distributed unmanned aerial vehicle swarm reconfiguration with up to 20 unmanned aerial vehicles.

Bayesian Direction-of-Arrival Estimation Using Atomic Norm Minimization With Prior Knowledge

Bayesian Direction-of-Arrival Estimation Using Atomic Norm Minimization With Prior Knowledge

Rotational Motion Compensation for ISAR Imaging Based on Minimizing the Residual Norm

In inverse synthetic aperture radar (ISAR) systems, image quality often suffers from the non-uniform rotation of non-cooperative targets. Rotational motion compensation (RMC) is necessary to perform refocused ISAR imaging via estimated rotational motion parameters. However, estimation errors tend to accumulate with the estimated processes, deteriorating the image quality. A novel RMC algorithm is proposed in this study to mitigate the impact of cumulative errors. The proposed method uses an iterative approach based on a novel criterion, i.e., the minimum residual norm of the signal phases, to estimate different rotational parameters independently to avoid the issue caused by cumulative errors. First, a refined inverse function combined with interpolation is proposed to perform the RMC procedure. Then, the rotation parameters are estimated using an iterative procedure designed to minimize the residual norm of the compensated signal phases. Finally, with the estimated parameters, RMC is performed on signals in all range bins, and focused images are obtained using the Fourier transform. Furthermore, this study utilizes simulated and real data to validate and evaluate the performance of the proposed algorithm. The experimental results demonstrate that the proposed algorithm shows dominance in the aspects of estimation accuracy, entropy values, and focusing characteristics.

Integrating Electroencephalography Source Localization and Residual Convolutional Neural Network for Advanced Stroke Rehabilitation.

Motor impairments caused by stroke significantly affect daily activities and reduce quality of life, highlighting the need for effective rehabilitation strategies. This study presents a novel approach to classifying motor tasks using EEG data from acute stroke patients, focusing on left-hand motor imagery, right-hand motor imagery, and rest states. By using advanced source localization techniques, such as Minimum Norm Estimation (MNE), dipole fitting, and beamforming, integrated with a customized Residual Convolutional Neural Network (ResNetCNN) architecture, we achieved superior spatial pattern recognition in EEG data. Our approach yielded classification accuracies of 91.03% with dipole fitting, 89.07% with MNE, and 87.17% with beamforming, markedly surpassing the 55.57% to 72.21% range of traditional sensor domain methods. These results highlight the efficacy of transitioning from sensor to source domain in capturing precise brain activity. The enhanced accuracy and reliability of our method hold significant potential for advancing brain-computer interfaces (BCIs) in neurorehabilitation. This study emphasizes the importance of using advanced EEG classification techniques to provide clinicians with precise tools for developing individualized therapy plans, potentially leading to substantial improvements in motor function recovery and overall patient outcomes. Future work will focus on integrating these techniques into practical BCI systems and assessing their long-term impact on stroke rehabilitation.

Tikhonov regularized second-order plus first-order primal-dual dynamical systems with asymptotically vanishing damping for linear equality constrained convex optimization problems

In this paper, in the setting of Hilbert spaces, we consider a Tikhonov regularized second-order plus first-order primal-dual dynamical system with asymptotically vanishing damping for a linear equality constrained convex optimization problem. It is shown that convergence properties of the proposed inertial dynamical system depend upon the choice of the Tikhonov regularization parameter. When the Tikhonov regularization parameter decays rapidly to zero, we derive the fast convergence rates of the primal-dual gap, the objective residual, the feasibility violation and the gradient norm of the objective function along the generated trajectory. When the Tikhonov regularization parameter decreases slowly to zero, we prove the strong convergence of the primal trajectory of the Tikhonov regularized dynamical system to the minimal norm solution of the linear equality constrained convex optimization problem. Numerical experiments are performed to illustrate the efficiency of our approach.

Trajectory tracking control of a mobile robot using fuzzy logic controller with optimal parameters

Abstract This work investigates the use of a fuzzy logic controller (FLC) for two-wheeled differential drive mobile robot trajectory tracking control. Due to the inherent complexity associated with tuning the membership functions of an FLC, this work employs a particle swarm optimization algorithm to optimize the parameters of these functions. In order to automate and reduce the number of rule bases, the genetic algorithm is also employed for this study. The effectiveness of the proposed approach is validated through MATLAB simulations involving diverse path tracking scenarios. The performance of the FLC is compared against established controllers, including minimum norm solution, closed-loop inverse kinematics, and Jacobian transpose-based controllers. The results demonstrate that the FLC offers accurate trajectory tracking with reduced root mean square error and controller effort. An experimental, hardware-based investigation is also performed for further verification of the proposed system. In addition, the simulation is conducted for various paths in the presence of noise in order to assess the proposed controller’s robustness. The proposed method is resilient against noise and disturbances, according to the simulation outcomes.

Revisiting trace norm minimization for tensor Tucker completion: A direct multilinear rank learning approach

To efficiently express tensor data using the Tucker format, a critical task is to minimize the multilinear rank such that the model would not be over-flexible and lead to overfitting. Due to the lack of rank minimization tools in tensor, existing works connect Tucker multilinear rank minimization to trace norm minimization of matrices unfolded from the tensor data. While these formulations try to exploit the common aim of identifying the low-dimensional structure of the tensor and matrix, this paper reveals that existing trace norm-based formulations in Tucker completion are inefficient in multilinear rank minimization. We further propose a new interpretation of Tucker format such that trace norm minimization is applied to the factor matrices of the equivalent representation, rather than some matrices unfolded from tensor data. Based on the newly established problem formulation, a fixed point iteration algorithm is proposed, and its convergence is proved. Numerical results are presented to show that the proposed algorithm exhibits significant improved performance in terms of multilinear rank learning and consequently tensor signal recovery accuracy, compared to existing trace norm based Tucker completion methods.

Accelerated model-based T1, T2* and proton density mapping using a Bayesian approach with automatic hyperparameter estimation.

To achieve automatic hyperparameter estimation for the model-based recovery of quantitative MR maps from undersampled data, we propose a Bayesian formulation that incorporates the signal model and sparse priors among multiple image contrasts. We introduce a novel approximate message passing framework "AMP-PE" that enables the automatic and simultaneous recovery of hyperparameters and quantitative maps. We employed the variable-flip-angle method to acquire multi-echo measurements using gradient echo sequence. We explored undersampling schemes to incorporate complementary sampling patterns across different flip angles and echo times. We further compared AMP-PE with conventional compressed sensing approaches such as the -norm minimization, PICS and other model-based approaches such as GraSP, MOBA. Compared to conventional compressed sensing approaches such as the -norm minimization and PICS, AMP-PE achieved superior reconstruction performance with lower errors in mapping and comparable performance in and proton density mappings. When compared to other model-based approaches including GraSP and MOBA, AMP-PE exhibited greater robustness and outperformed GraSP in reconstruction error. AMP-PE offers faster speed than MOBA. AMP-PE performed better than MOBA at higher sampling rates and worse than MOBA at a lower sampling rate. Notably, AMP-PE eliminates the need for hyperparameter tuning, which is a requisite for all the other approaches. AMP-PE offers the benefits of model-based recovery with the additional key advantage of automatic hyperparameter estimation. It works adeptly in situations where ground-truth is difficult to obtain and in clinical environments where it is desirable to automatically adapt hyperparameters to individual protocol, scanner and patient.

High-Resolution and Robust One-Bit Direct-of-Arrival Estimation via Reweighted Atomic Norm Estimation.

In recent years, one-bit quantization has attracted widespread attention in the field of direction-of-arrival (DOA) estimation as a low-cost and low-power solution. Many researchers have proposed various estimation algorithms for one-bit DOA estimation, among which atomic norm minimization algorithms exhibit particularly attractive performance as gridless estimation algorithms. However, current one-bit DOA algorithms with atomic norm minimization typically rely on approximating the trace function, which is not the optimal approximation and introduces errors, along with resolution limitations. To date, there have been few studies on how to enhance resolution under the framework of one-bit DOA estimation. This paper aims to improve the resolution constraints of one-bit DOA estimation. The log-det heuristic is applied to approximate and solve the atomic norm minimization problem. In particular, a reweighted binary atomic norm minimization with noise assumption constraints is proposed to achieve high-resolution and robust one-bit DOA estimation. Finally, the alternating direction method of multipliers algorithm is employed to solve the established optimization problem. Simulations are conducted to demonstrate that the proposed algorithm can effectively enhance the resolution.

Equivalent conditions of null controllability for controlled stochastic relaxed system

ABSTRACT This paper focuses on the investigation of null controllability in the context of stochastic controlled relaxed systems with one control, aiming to establish several equivalent conditions for this property. More precisely, by the classical duality analysis, we present the equivalence between an observability inequality for the backward stochastic system associated with the relaxed system and null controllability. Furthermore, we characterize the null controllability through a minimal norm control problem, employing a constructed quadratic functional and leveraging a variational characterization of its minimizer. Finally, we provide an application of our findings and extend the analysis to relaxed systems with two controls.

Guaranteed matrix recovery using weighted nuclear norm plus weighted total variation minimization

This work presents a general framework regarding the recovery of matrices equipped with hybrid low-rank and local-smooth properties from just a few measurements consisting of linear combinations of the matrix entries. Concretely, we consider the problem of robust low-rank matrix recovery using Weighted Nuclear Norm plus Weight Total Variation (WNNWTV) minimization. First of all, based on a new restricted isometry property, we prove that the WNNWTV method possesses an error bound consisting of a low-rank approximation term, a total variation approximation term, and an observation error term. It should be noted that although there are many models considering both properties, there are very few recoverable error theories about such models. Specifically, the theoretical error bound provides an automatic mechanism to reducing regularization parameters with no need for cross-validation while keeping almost the same selection result with commonly used cross-validation technique. Subsequently, the proposed method is reformulated into a regularized unconstrained problem, and we study its optimization aspects in detail based on the Alternating Direction Method of Multipliers (ADMM). Extensive experiments on synthetic data and two applications, i.e. hyperspectral image recovery and dynamic magnetic resonance imaging recovery verified our theories and proposed algorithms.

The Growth Order of the Optimal Constants in Turán-Erőd Type Inequalities in Lq(K,μ)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L^q(K,\\mu )$$\\end{document}

In 1939 Turán raised the question about lower estimations of the maximum norm of the derivatives of a polynomial p of maximum norm 1 on the compact set K of the complex plain under the normalization condition that the zeroes of p in question all lie in K. Turán studied the problem for the interval I and the unit disk D and found that with n:=degp\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$n:= \\deg p$$\\end{document} tending to infinity, the precise growth order of the minimal possible derivative norm (oscillation order) is n\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\sqrt{n}$$\\end{document} for I and n for D. Erőd continued the work of Turán considering other domains. Finally, in 2006, Halász and Révész proved that the growth of the minimal possible maximal norm of the derivative is of order n for all compact convex domains. Although Turán himself gave comments about the above oscillation question in Lq\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L^q$$\\end{document}norms, till recently results were known only for D and I. Recently, we have found order n lower estimations for several general classes of compact convex domains, and proved that in Lq\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L^q$$\\end{document} norm the oscillation order is at least n/logn\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$n/\\log n$$\\end{document} for all compact convex domains. In the present paper we prove that the oscillation order is not greater than n for all compact (not necessarily convex) domains K and Lq\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L^q$$\\end{document}norm with respect to any measure supported on more than two points on K.

Enhanced channel estimation with atomic norm minimization and reconfigurable intelligent surfaces in mmWave MIMO systems

SummaryThe performance of millimeter‐wave (mmWave) multiple‐input multiple‐output (MIMO) systems has been significantly enhanced by the incorporation of dynamic reconfigurable intelligent surfaces (RIS). This paper proposes a novel dynamic channel estimation technique that combines dynamic atomic norm minimization with dynamic RIS to optimize RIS‐aided mmWave MIMO systems. Leveraging the dynamic nature of both atomic norm minimization and RIS, the proposed approach efficiently adapts to changing environmental conditions, providing robust and accurate channel estimation. By dynamically optimizing the RIS configuration, the system achieves improved spectral and energy efficiency, enabling high‐speed and reliable communication in challenging mmWave environments. Theoretical analysis and simulation results demonstrate the effectiveness of the proposed dynamic channel estimation technique, highlighting its potential for enhancing the performance of future wireless communication systems.

Quaternion Nuclear Norm Minus Frobenius Norm Minimization for color image reconstruction

Color image restoration methods typically represent images as vectors in Euclidean space or combinations of three monochrome channels. However, they often overlook the correlation between these channels, leading to color distortion and artifacts in the reconstructed image. To address this, we present Quaternion Nuclear Norm Minus Frobenius Norm Minimization (QNMF), a novel approach for color image reconstruction. QNMF utilizes quaternion algebra to capture the relationships among RGB channels comprehensively. By employing a regularization technique that involves nuclear norm minus Frobenius norm, QNMF approximates the underlying low-rank structure of quaternion-encoded color images. Theoretical proofs are provided to ensure the method’s mathematical integrity. Demonstrating versatility and efficacy, the QNMF regularizer excels in various color low-level vision tasks, including denoising, deblurring, inpainting, and random impulse noise removal, achieving state-of-the-art results.

The BiCG Algorithm for Solving the Minimal Frobenius Norm Solution of Generalized Sylvester Tensor Equation over the Quaternions

In this paper, we develop an effective iterative algorithm to solve a generalized Sylvester tensor equation over quaternions which includes several well-studied matrix/tensor equations as special cases. We discuss the convergence of this algorithm within a finite number of iterations, assuming negligible round-off errors for any initial tensor. Moreover, we demonstrate the unique minimal Frobenius norm solution achievable by selecting specific types of initial tensors. Additionally, numerical examples are presented to illustrate the practicality and validity of our proposed algorithm. These examples include demonstrating the algorithm’s effectiveness in addressing three-dimensional microscopic heat transport and color video restoration problems.