Minimum Norm Research Articles

Spatiotemporal traffic data imputation via tensorial weighted Schatten‐ p norm minimization

Spatiotemporal traffic data exhibit multi-granular low-rank structure due to their periodicity among different timelines. Traditional low rank data completion methods fail to characterize such properties and produce unsatisfactory results for data imputation. In this paper, a tensorial weighted Schatten-p norm minimization (TWSN) is proposed for spatiotemporal traffic data imputation. TWSN consists of an approximation term and a low-rank regularization term over the recovered tensor data, where the latter is a combination of the weighted Schatten-p norm in the matrix form of each mode of the tensor. For each mode, TWSN utilizes a selection scheme of the mode-wise weights to capture different properties of singular values of each mode of the tensor. Overall, TWSN not only plays a balancing role between the rank function and the nuclear norm, but also captures the anisotropic correlation of singular values of each mode of the tensor. TWSN is evaluated on four real-world datasets with different ping frequencies (2, 5, 10 min) and its performance is compared with several state-of-the-art methods. The experimental results show that TWSN outperforms other methods under various data missing scenarios.

A finite iterative algorithm for the general discrete-time periodic Sylvester matrix equations

The purpose of this paper is to present an iterative algorithm for solving the general discrete-time periodic Sylvester matrix equations. It is proved by theoretical analysis that this algorithm can get the exact solutions of the periodic Sylvester matrix equations in a finite number of steps in the absence of round-off errors. Furthermore, when the discrete-time periodic Sylvester matrix equations are consistent, we can obtain its unique minimal Frobenius norm solution by choosing appropriate initial periodic matrices. Finally, we use some numerical examples to illustrate the effectiveness of the proposed algorithm.

A super-resolution framework for tensor decomposition

Abstract This work considers a super-resolution framework forovercomplete tensor decomposition. Specifically, we view tensor decomposition as a super-resolution problem of recovering a sum of Dirac measures on the sphere and solve it by minimizing a continuous analog of the $\ell _1$ norm on the space of measures. The optimal value of this optimization defines the tensor nuclear norm. Similar to the separation condition in the super-resolution problem, by explicitly constructing a dual certificate, we develop incoherence conditions of the tensor factors so that they form the unique optimal solution of the continuous analog of $\ell _1$ norm minimization. Remarkably, the derived incoherence conditions are satisfied with high probability by random tensor factors uniformly distributed on the sphere, implying global identifiability of random tensor factors.

Fast heritability estimation based on MINQUE and batch training.

Heritability, the proportion of phenotypic variance explained by genome-wide single nucleotide polymorphisms (SNPs) in unrelated individuals, is an important measure of the genetic contribution to human diseases and plays a critical role in studying the genetic architecture of human diseases. Linear mixed model (LMM) has been widely used for SNP heritability estimation, where variance component parameters are commonly estimated by using a restricted maximum likelihood (REML) method. REML is an iterative optimization algorithm, which is computationally intensive when applied to large-scale datasets (e.g. UK Biobank). To facilitate the heritability analysis of large-scale genetic datasets, we develop a fast approach, minimum norm quadratic unbiased estimator (MINQUE) with batch training, to estimate variance components from LMM (LMM.MNQ.BCH). In LMM.MNQ.BCH, the parameters are estimated by MINQUE, which has a closed-form solution for fast computation and has no convergence issue. Batch training has also been adopted in LMM.MNQ.BCH to accelerate the computation for large-scale genetic datasets. Through simulations and real data analysis, we demonstrate that LMM.MNQ.BCH is much faster than two existing approaches, GCTA and BOLT-REML.

SURPRISES IN HIGH-DIMENSIONAL RIDGELESS LEAST SQUARES INTERPOLATION.

Interpolators-estimators that achieve zero training error-have attracted growing attention in machine learning, mainly because state-of-the art neural networks appear to be models of this type. In this paper, we study minimum ℓ 2 norm ("ridgeless") interpolation least squares regression, focusing on the high-dimensional regime in which the number of unknown parameters p is of the same order as the number of samples n. We consider two different models for the feature distribution: a linear model, where the feature vectors are obtained by applying a linear transform to a vector of i.i.d. entries, x i = Σ1/2 z i (with ); and a nonlinear model, where the feature vectors are obtained by passing the input through a random one-layer neural network, xi = φ(Wz i ) (with , a matrix of i.i.d. entries, and φ an activation function acting componentwise on Wz i ). We recover-in a precise quantitative way-several phenomena that have been observed in large-scale neural networks and kernel machines, including the "double descent" behavior of the prediction risk, and the potential benefits of overparametrization.

Тензорные методы длясильно выпуклых сильно вогнутых седловых задач и сильно монотонных вариационных неравенств

In this paper we propose three $p$-th order tensor methods for $\mu$-strongly-convex-strongly-concave saddle point problems (SPP). The first method is based on the assumption of $p$-th order smoothness of the objective and it achieves a convergence rate of $O \left( \left( \frac{L_p R^{p - 1}}{\mu} \right)^\frac{2}{p + 1} \log \frac{\mu R^2}{\varepsilon_G} \right)$, where $R$ is an estimate of the initial distance to the solution, and $\varepsilon_G$ is the error in terms of duality gap. Under additional assumptions of first and second order smoothness of the objective we connect the first method with a locally superlinear converging algorithm and develop a second method with the complexity of $O \left( \left( \frac{L_p R^{p - 1}}{\mu} \right)^\frac{2}{p + 1}\log \frac{L_2 R \max \left\{ 1, \frac{L_1}{\mu} \right\}}{\mu} + \log \frac{\log \frac{L_1^3}{2 \mu^2 \varepsilon_G}}{\log \frac{L_1 L_2}{\mu^2}} \right)$. The third method is a modified version of the second method, and it solves gradient norm minimization SPP with $\tilde O \left( \left( \frac{L_p R^p}{\varepsilon_\nabla} \right)^\frac{2}{p + 1} \right)$ oracle calls, where $\varepsilon_\nabla$ is an error in terms of norm of the gradient of the objective. Since we treat SPP as a particular case of variational inequalities, we also propose three methods for strongly monotone variational inequalities with the same complexity as the described above.

Approximate solution of optimal control problem for cooling process with minimum energy

In the paper, optimal control problem for cooling process with minimum energy in materials with heat conducting viscosity is considered. To solve approximately the considered problem, a finite-dimensional approximation for the solution of the corresponding boundary value problem in the form of truncated Fourier series is constructed and an integral representation for the coefficients of this series is obtained. This yields a system of integral equations with respect to control parameters. So, the problem is reduced to finding a minimum norm function from these moment relations. Applying the theorem on orthogonal decomposition of a normalized space, every approximation of control parameter and the corresponding value of a functional in analytic form is finded.

Quaternion-based weighted nuclear norm minimization for color image restoration

Color image restoration is one of the basic tasks in pattern recognition. Unlike grayscale image, each color image has three channels in the RGB color space. Due to the inner-relationship within the three channels, color image restoration is usually much more difficult than its grayscale counterpart. Indeed, new problems such as color artifacts could emerge when the grayscale image processing methods are extended to color images directly. Note that one of the most effective gray image restoration methods is the weighted nuclear norm minimization (WNNM) approach. However, when applied to color images, the results of WNNM are usually not as promising as that of grayscale images. In order to solve this problem, in this paper, we propose to restore color images with the quaternion-based WNNM method (QWNNM) since the structure of color channels can be well preserved with quaternion representation. The proposed model can be solved efficiently by the alternating direction method of multipliers (ADMM). The theoretical analysis of the optimal solution is also presented. Numerical experiments are carefully conducted with different kinds of degradation to illustrate the superior performance of our proposed QWNNM over the state-of-the-art methods, including a celebrated deep learning approach, in both visual quality and numerical results.

Robust attitude control of a quadrotor unmanned aerial vehicles using optimized disturbance observer

AbstractThis article proposes a method of realizing robust attitude stabilization control of a quadrotor UAV using the disturbance observer (DOB), where Q–filter is optimally designed considering performances of attenuating disturbance and estimation noise, and robust stability against parameter variations and sampling duration delay due to discrete time control in microcontroller. Disturbance rejection performance mainly depends on the bandwidth determined by cut‐off frequency of DOB's Q–filter. However, extension of the Q–filter's bandwidth must be restricted due to the requirement of robust stability against parameter uncertainty and sampling duration delay in onboard microcontroller for flight control of quadrotor. In order to optimize the disturbance rejection performance under the restriction of bandwidth, an norm minimization problem is defined considering the frequency bands of disturbance and noise. The optimization problem is transformed into standard control design problem by pseudo loop factorization method so as to resolve the structural constraint of Q–filter. Then, stability of the discrete time system configured by digital controller is tested in relation with the time constant of the designed DOB's Q–filter under different inertia moment values. The comparisons between simulation results by different control methods validate that the robust control method using proposed DOB provides a better performance of disturbance attenuation than that using conventional DOB.

Parallel Stylometric Document Embeddings with Deep Learning Based Language Models in Literary Authorship Attribution

This paper explores the effectiveness of parallel stylometric document embeddings in solving the authorship attribution task by testing a novel approach on literary texts in 7 different languages, totaling in 7051 unique 10,000-token chunks from 700 PoS and lemma annotated documents. We used these documents to produce four document embedding models using Stylo R package (word-based, lemma-based, PoS-trigrams-based, and PoS-mask-based) and one document embedding model using mBERT for each of the seven languages. We created further derivations of these embeddings in the form of average, product, minimum, maximum, and l2 norm of these document embedding matrices and tested them both including and excluding the mBERT-based document embeddings for each language. Finally, we trained several perceptrons on the portions of the dataset in order to procure adequate weights for a weighted combination approach. We tested standalone (two baselines) and composite embeddings for classification accuracy, precision, recall, weighted-average, and macro-averaged F1-score, compared them with one another and have found that for each language most of our composition methods outperform the baselines (with a couple of methods outperforming all baselines for all languages), with or without mBERT inputs, which are found to have no significant positive impact on the results of our methods.

Missing and Corrupted Data Recovery in Wireless Sensor Networks Based on Weighted Robust Principal Component Analysis.

Although wireless sensor networks (WSNs) have been widely used, the existence of data loss and corruption caused by poor network conditions, sensor bandwidth, and node failure during transmission greatly affects the credibility of monitoring data. To solve this problem, this paper proposes a weighted robust principal component analysis method to recover the corrupted and missing data in WSNs. By decomposing the original data into a low-rank normal data matrix and a sparse abnormal matrix, the proposed method can identify the abnormal data and avoid the influence of corruption on the reconstruction of normal data. In addition, the low-rankness is constrained by weighted nuclear norm minimization instead of the nuclear norm minimization to preserve the major data components and ensure credible reconstruction data. An alternating direction method of multipliers algorithm is further developed to solve the resultant optimization problem. Experimental results demonstrate that the proposed method outperforms many state-of-the-art methods in terms of recovery accuracy in real WSNs.

A New Variational Model for Shape Graph Registration with Partial Matching Constraints

This paper introduces a new extension of Riemannian elastic curve matching to a general class of geometric structures, which we call (weighted) shape graphs, that allows for shape registration with partial matching constraints and topological inconsistencies. Weighted shape graphs are the union of an arbitrary number of component curves in Euclidean space with potential connectivity constraints between some of their boundary points, together with a weight function defined on each component curve. The framework of higher-order invariant Sobolev metrics is particularly well suited for constructing notions of distances and geodesics between unparametrized curves. The main difficulty in adapting this framework to the setting of shape graphs is the absence of topological consistency, which typically results in an inadequate search for an exact matching between two shape graphs. We overcome this hurdle by defining an inexact variational formulation of the matching problem between (weighted) shape graphs of any underlying topology, relying on the convenient measure representation given by varifolds to relax the exact matching constraint. We then prove the existence of minimizers to this variational problem when we choose Sobolev metrics of sufficient regularity and a total variation (TV) regularization on the weight function. We propose a numerical optimization approach which adapts the smoothed fast iterative shrinkage-thresholding algorithm (SFISTA) to deal with $TV$ norm minimization and allows us to reduce the matching problem to solving a sequence of smooth unconstrained minimization problems. We finally illustrate the capabilities of our new model through several examples showcasing its ability to tackle partially observed and topologically varying data.

School-aged children diagnosed with an FASD exhibit visuo-cortical network disturbance: A magnetoencephalography (MEG) study.

Children with prenatal alcohol exposure (PAE) often suffer from cognitive and neurobehavioral dysfunction throughout their lives, which may rise to a level of concern such that children receive a diagnosis under the fetal alcohol spectrum disorders (FASD) umbrella. Magnetoencephalography (MEG) contributes direct insight into neural processing and functional connectivity measures with temporal precision to understand cortical processing disorders that manifest during development. The impairment of perception may become more consequential among school-aged children with an FASD in the process of intellectual functioning and behavioral maturation. Fifty participants with the age range of 8-13 years participated in our study following parental informed consent and child assent. For each participant, visual responses were recorded using magnetoencephalography (MEG) while performing a prosaccade task with central stimuli (fovea centralis) and peripheral stimuli (left and right of central) presented on a screen, requiring participants to shift their gaze to the stimuli. After source analysis using minimum norm estimation (MNE), we investigated visual responses from each participant by measuring the latency and amplitude of visual evoked fields. Delayed peak latency of the visual response was identified in the primary visual area (calcarine fissure) and visual association areas (v2, v3) in young children with an FASD for both stimulus types (central and peripheral). But the difference in visual response latency was only statistically significant (p≤0.01) for the peripheral (right) stimulus. We also observed reduced amplitude (p≤0.006) of visual evoked response in children with an FASD for the central stimulus type in both primary and visual association areas. Multiple visual areas show impairment in children with an FASD, with visual delay and conduction disturbance more prominent in response to peripheral stimuli. Children with an FASD also exhibit significantly reduced amplitude of neural activation to central stimuli. These sensory deficits may lead to slow cognitive processing speed through continued intra-cortical network disturbance in children with an FASD.

Super-Resolution DOA Estimation for Wideband Signals Using Non-Uniform Linear Arrays With No Focusing Matrix

We focus on developing an effective Direction Of Arrival (DOA) estimation method for wideband sources based on the super-resolution concept. In this letter, we represent the data model of a wideband DOA estimation problem in a new form. Then, using this model, we propose an atomic norm minimization problem that leads to an accurate wideband DOA estimation method with no need for any focusing matrices. Moreover, this method requires no initial estimate and can be implemented by non-uniform linear arrays. Numerical simulations show that in comparison to some well-known techniques, the proposed method generates outstanding accuracy and higher robustness to noise. The effectiveness of the method is also verified in presence of close adjacent sources.

A sociocultural norm perspective on Big Five prediction.

The Big Five predict numerous preferences, decisions, and behaviors-but why? To help answer this key question, the present research develops the sociocultural norm perspective (SNP) on Big Five prediction-a critical revision and extension of the sociocultural motives perspective. The SNP states: Agreeableness, Extraversion, and Conscientiousness predict outcomes positively if those outcomes are socioculturally normative. Openness, by contrast, predicts outcomes negatively if they are socioculturally normative. Moreover, the SNP specifies unique mechanisms that underlie those predictions. Two mechanisms are social (social trust for Agreeableness, social attention for Extraversion) and two are cognitive (rational thought for Conscientiousness, independent thought for Openness). The present research develops the SNP by means of three large-scale experiments (Ntotal = 7,404), which used a new, tailor-made experimental paradigm-the minimal norm paradigm. Overall, the SNP provides norm-based, culture-focused, and mechanism-attentive explanations for why the Big Five predict their outcomes. The SNP also has broader relevance: It helps explain why Big Five effects vary across cultures and, thus, dispels the view that such variation threatens the validity of the Big Five. It suggests that the psychology of norms would benefit from attention to the Big Five. Finally, it helps bridge personality, social, and cross-cultural psychology by integrating their key concepts-the Big Five, conformity, and sociocultural norms. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

A Randomized Distributed Kaczmarz Algorithm and Anomaly Detection

The Kaczmarz algorithm is an iterative method for solving systems of linear equations. We introduce a randomized Kaczmarz algorithm for solving systems of linear equations in a distributed environment, i.e., the equations within the system are distributed over multiple nodes within a network. The modification we introduce is designed for a network with a tree structure that allows for passage of solution estimates between the nodes in the network. We demonstrate that the algorithm converges to the solution, or the solution of minimal norm, when the system is consistent. We also prove convergence rates of the randomized algorithm that depend on the spectral data of the coefficient matrix and the random control probability distribution. In addition, we demonstrate that the randomized algorithm can be used to identify anomalies in the system of equations when the measurements are perturbed by large, sparse noise.

Sound source localization of non-synchronous measurements beamforming based on the truncated nuclear norm regularization

Due to the size and aperture limitations of the microphone array, poor resolution and limited working frequency range are challenging for acoustic imaging with beamforming methods. In this work, two fast iteration algorithms, alternating direction method of multipliers (ADMM) and accelerated proximal gradient line search method (APGL) based on truncated nuclear norm regularization (TNNR), are proposed to complete the cross-spectral matrix for non-synchronous measurements beamforming. Compared with current matrix completion algorithm based on nuclear norm minimization (NNM), TNNR is regarded as a better approximation to the rank function, which means it can approach the real solution better and restore the sound sources accurately at a low signal-to-noise ratio (SNR) environment. Among the TNNR-APGL and TNNR-ADMM, the TNNR-APGL algorithm has a lower matrix completion error and a shorter calculation time. The localization results of TNNR-ADMM method are more robust to the number of measurements. The experiment results show that TNNR methods greatly improve the resolution of conventional beamforming (CBF) and suppress the sidelobes under low SNR conditions, which indicates that the non-synchronous measurements beamforming based on TNNR methods has the potential to be applied in the industrial scene and low SNR environment.

Infrared Small Target Detection Based on Partial Sum Minimization and Total Variation

In the advanced applications, based on infrared detection systems, the precise detection of small targets has become a tough work today. This becomes even more difficult when the background is highly dense in addition to the nature of small targets. The problem raised above is solved in various ways, including infrared patch image (IPI) based methods which are considered to have the best performance. In addition, the greater shrinkage of singular values in the methods based on IPI leads to the problem of nuclear norm minimization (NNM), which leads to the problem of incorrectly recognizing small targets in a highly complex background. Hence, this paper proposed a new method for infrared small target detection (ISTD) via total variation and partial sum minimization (TV-PSMSV). The proposed TV-PSMVS in this work basically replaces the IPI’s NNM with partial sum minimization (PSM) of singular values and, additionally, the total variance (TV) regularization term is inducted to the background patch image (BPI) to suppress the complex background and enhance the target object of interest. The mathematical solution of the proposed TV-PSMSV approach was performed using alternating direction multiplier (ADMM) to verify the proposed solution. The experimental evaluation using real and synthetic data set was performed, and the result revealed that the proposed TV-PSMSV outperformed existing referenced methods in the terms of background suppression factor (BSF) and the signal to gain ratio (SCRG).

3MO-AHP: an inconsistency reduction approach through mono-, multi- or many-objective quality measures

PurposeAlthough the multi-criteria technique analytic hierarchy process (AHP) has successfully been applied in many areas, either selecting or ranking alternatives or to derive priority vector (weights) for a set of criteria, there is a significant drawback in using this technique if the pairwise comparison matrix (PCM) has inconsistent comparisons, in other words, a consistency ratio (CR) above the value of 0.1, the final solution cannot be validated. Many studies have been developed to treat the inconsistency problem, but few of them tried to satisfy different quality measures, which are minimum inconsistency (), the total number of adjusted pairwise comparisons (), original rank preservation (), minimum average weights adjustment () and finally, minimum L1 matrix norm between the original PCM and the adjusted PCM ().Design/methodology/approachThe approach is defined in four steps: first, the decision-maker should choose which quality measures she/he wishes to use, ranging from one to all quality measures. In the second step, the authors encode the PCM to be used in a many-objective optimization algorithm (MOOA), and each pairwise comparison can be adjusted individually. The authors generate consistent solutions from the obtained Pareto optimal front that carry the desired quality measures in the third step. Lastly, the decision-maker selects the most suitable solution for her/his problem. Remarkably, as the decision-maker can choose one (mono-objective), two (multi-objective), three or more (many-objectives) quality measures, not all MOOAs can handle or perform well in mono- or multi-objective problems. The unified non-sorting algorithm III (U-NSGA III) is the most appropriate MOOA for this type of scenario because it was specially designed to handle mono-, multi- and many-objective problems.FindingsThe use of two quality measures should not guarantee that the adjusted PCM is similar to the original PCM; hence, the decision-maker should consider using more quality measures if the objective is to preserve the original PCM characteristics.Originality/valueFor the first time, a many-objective approach reduces the CR to consistent levels with the ability to consider one or more quality measures and allows the decision-maker to adjust each pairwise comparison individually.

Sparse Data-Driven Quadrature Rules via ℓ p -Quasi-Norm Minimization

Abstract This paper is concerned with the use of the focal underdetermined system solver to recover sparse empirical quadrature rules for parametrized integrals from existing data. This algorithm, originally proposed for image and signal reconstruction, relies on an approximated ℓ p {\ell^{p}} -quasi-norm minimization. Compared to ℓ 1 {\ell^{1}} -norm minimization, the choice of 0 < p < 1 {0<p<1} provides a natural framework to accommodate usual constraints which quadrature rules must fulfil. We also extend an a priori error estimate available for the ℓ 1 {\ell^{1}} -norm formulation by considering the error resulting from data compression. Finally, we present numerical examples to investigate the numerical performance of our method and compare our results to both ℓ 1 {\ell^{1}} -norm minimization and nonnegative least squares method. Matlab codes related to the numerical examples and the algorithms described are provided.