Exact Minimax Research Articles

Variable selection, monotone likelihood ratio and group sparsity

In the pivotal variable selection problem, we derive the exact nonasymptotic minimax selector over the class of all s-sparse vectors, which is also the Bayes selector with respect to the uniform prior. While this optimal selector is, in general, not realizable in polynomial time, we show that its tractable counterpart (the scan selector) attains the minimax expected Hamming risk to within factor 2, and is also exact minimax with respect to the probability of wrong recovery. As a consequence, we establish explicit lower bounds under the monotone likelihood ratio property and we obtain a tight characterization of the minimax risk in terms of the best separable selector risk. We apply these general results to derive necessary and sufficient conditions of exact and almost full recovery in the location model with light tail distributions and in the problem of group variable selection under Gaussian noise and under more general anisotropic sub-Gaussian noise. Numerical results illustrate our theoretical findings.

Optimal orthogonal group synchronization and rotation group synchronization

Abstract We study the statistical estimation problem of orthogonal group synchronization and rotation group synchronization. The model is $Y_{ij} = Z_i^* Z_j^{*T} + \sigma W_{ij}\in{\mathbb{R}}^{d\times d}$ where $W_{ij}$ is a Gaussian random matrix and $Z_i^*$ is either an orthogonal matrix or a rotation matrix, and each $Y_{ij}$ is observed independently with probability $p$. We analyze an iterative polar decomposition algorithm for the estimation of $Z^*$ and show it has an error of $(1+o(1))\frac{\sigma ^2 d(d-1)}{2np}$ when initialized by spectral methods. A matching minimax lower bound is further established that leads to the optimality of the proposed algorithm as it achieves the exact minimax risk.

Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices

We consider random-design linear prediction and related questions on the lower tail of random matrices. It is known that, under boundedness constraints, the minimax risk is of order d/n in dimension d with n samples. Here, we study the minimax expected excess risk over the full linear class, depending on the distribution of covariates. First, the least squares estimator is exactly minimax optimal in the well-specified case, for every distribution of covariates. We express the minimax risk in terms of the distribution of statistical leverage scores of individual samples, and deduce a minimax lower bound of d/(n−d+1) for any covariate distribution, nearly matching the risk for Gaussian design. We then obtain sharp nonasymptotic upper bounds for covariates that satisfy a “small ball”-type regularity condition in both well-specified and misspecified cases. Our main technical contribution is the study of the lower tail of the smallest singular value of empirical covariance matrices at small values. We establish a lower bound on this lower tail, valid for any distribution in dimension d≥2, together with a matching upper bound under a necessary regularity condition. Our proof relies on the PAC-Bayes technique for controlling empirical processes, and extends an analysis of Oliveira devoted to a different part of the lower tail.

Minimax rates in sparse, high-dimensional change point detection

We study the detection of a sparse change in a high-dimensional mean vector as a minimax testing problem. Our first main contribution is to derive the exact minimax testing rate across all parameter regimes for n independent, p-variate Gaussian observations. This rate exhibits a phase transition when the sparsity level is of order ploglog(8n) and has a very delicate dependence on the sample size: in a certain sparsity regime, it involves a triple iterated logarithmic factor in n. Further, in a dense asymptotic regime, we identify the sharp leading constant, while in the corresponding sparse asymptotic regime, this constant is determined to within a factor of 2. Extensions that cover spatial and temporal dependence, primarily in the dense case, are also provided.

On estimation of isotonic piecewise constant signals

Consider a sequence of real data points $X_{1},\ldots ,X_{n}$ with underlying means $\theta ^{*}_{1},\dots ,\theta ^{*}_{n}$. This paper starts from studying the setting that $\theta ^{*}_{i}$ is both piecewise constant and monotone as a function of the index $i$. For this, we establish the exact minimax rate of estimating such monotone functions, and thus give a nontrivial answer to an open problem in the shape-constrained analysis literature. The minimax rate under the loss of the sum of squared errors involves an interesting iterated logarithmic dependence on the dimension, a phenomenon that is revealed through characterizing the interplay between the isotonic shape constraint and model selection complexity. We then develop a penalized least-squares procedure for estimating the vector $\theta ^{*}=(\theta^{*}_{1},\dots ,\theta ^{*}_{n})^{\mathsf{T}}$. This estimator is shown to achieve the derived minimax rate adaptively. For the proposed estimator, we further allow the model to be misspecified and derive oracle inequalities with the optimal rates, and show there exists a computationally efficient algorithm to compute the exact solution.

An exact minimax penalty function approach to solve multitime variational problems

This paper aims to examine an appropriateness of the exact minimax penalty function method applied to solve the partial differential inequation (PDI) and partial differential equation (PDE) constrained multitime variational problems. The criteria for equivalence between the optimal solutions of a multitime variational problem with PDI and PDE constraints and its associated unconstrained penalized multitime variational problem is studied in this work. We also present some examples to validate the results derived in the paper.

An Exact Minimax Penalty Function Method and Saddle Point Criteria for Nonsmooth Convex Vector Optimization Problems

In this paper, the exact minimax penalty function method is applied to solve constrained multiobjective optimization problems involving locally Lipschitz functions. The criteria for a saddle point for the original vector optimization problem are studied with the help of the penalized unconstrained vector optimization problem. Furthermore, we determine the conditions for which the (weak) efficient solutions of the vector optimization problem are equivalent to those of the associated, penalized unconstrained vector optimization problem. Some examples of nonsmooth multiobjective problems solved by using the exact minimax penalty method are presented to illustrate the results established in the paper.

Sharp optimality for regression with real-time data

A minimax estimator for a nonparametric regression model is proposed when real-time data are assumed and its asymptotic behavior of minimax risk in the sup-norm for the Hölder function class is studied. The optimal rate of convergence and exact minimax constant are found for the estimator.

Exactness of penalization for exact minimax penalty function method in nonconvex programming

The exact minimax penalty function method is used to solve a nonconvex differentiable optimization problem with both inequality and equality constraints. The conditions for exactness of the penalization for the exact minimax penalty function method are established by assuming that the functions constituting the considered constrained optimization problem are invex with respect to the same function η (with the exception of those equality constraints for which the associated Lagrange multipliers are negative—these functions should be assumed to be incave with respect to η). Thus, a threshold of the penalty parameter is given such that, for all penalty parameters exceeding this threshold, equivalence holds between the set of optimal solutions in the considered constrained optimization problem and the set of minimizer in its associated penalized problem with an exact minimax penalty function. It is shown that coercivity is not sufficient to prove the results.

Decision-theoretic issues in heterogeneity variance estimation

Motivated by a heteroscedastic random effects setting of meta-analysis, a general model for the between-study variance is studied from the decision-theoretic point of view. This model leads to estimation of a linear in the variance reciprocals, random function or to simultaneous inference on curve-confined natural parameters of independent heterogeneous \(\chi ^2\)-random variables with given degrees of freedom. A form of the Stein phenomenon for the suggested loss functions is noted; the exact minimax value is determined, and minimax estimators are derived.

Matrix completion by singular value thresholding: Sharp bounds

We consider the matrix completion problem where the aim is to estimate a large data matrix for which only a relatively small random subset of its entries is observed. Quite popular approaches to matrix completion problem are iterative thresholding methods. In spite of their empirical success, the theoretical guarantees of such iterative thresholding methods are poorly understood. The goal of this paper is to provide strong theoretical guarantees, similar to those obtained for nuclear-norm penalization methods and one step thresholding methods, for an iterative thresholding algorithm which is a modification of the softImpute algorithm. An important consequence of our result is the exact minimax optimal rates of convergence for matrix completion problem which were know until now only up to a logarithmic factor.

A Lower Bound for the Penalty Parameter in the Exact Minimax Penalty Function Method for Solving Nondifferentiable Extremum Problems

In the paper, we consider the exact minimax penalty function method used for solving a general nondifferentiable extremum problem with both inequality and equality constraints. We analyze the relationship between an optimal solution in the given constrained extremum problem and a minimizer in its associated penalized optimization problem with the exact minimax penalty function under the assumption of convexity of the functions constituting the considered optimization problem (with the exception of those equality constraint functions for which the associated Lagrange multipliers are negative—these functions should be assumed to be concave). The lower bound of the penalty parameter is given such that, for every value of the penalty parameter above the threshold, the equivalence holds between the set of optimal solutions in the given extremum problem and the set of minimizers in its associated penalized optimization problem with the exact minimax penalty function.

SADDLE POINT CRITERIA AND THE EXACT MINIMAX PENALTY FUNCTION METHOD IN NONCONVEX PROGRAMMING

A new characterization of the exact minimax penalty function method is presented. The exactness of the penalization for the exact minimax penalty function method is analyzed in the context of saddle point criteria of the Lagrange function in the nonconvex differentiable optimization problem with both inequality and equality constraints. Thus, new conditions for the exactness of the exact minimax penalty function method are established under assumption that the functions constituting considered constrained optimization problem are invex with respect to the same function $\eta $ (exception with those equality constraints for which the associated Lagrange multipliers are negative - these functions should be assumed to be incave with respect to the same function $\eta $). The threshold of the penalty parameter is given such that, for all penalty parameters exceeding this treshold, the equivalence holds between a saddle point of the Lagrange function in the considered constrained extremum problem and a minimizer in its associated penalized optimization problem with the exact minimax penalty function.

Model selection in regression under structural constraints

The paper considers model selection in regression under the additional structural constraints on admissible models where the number of potential predictors miht be even larger than the available sample size. We develop a Bayesian formalism which is used as a natural tool for generating a wide class of model selection criteria based on penalized least squares estimation with various complexity penalties associated with a prior on a model size. The resulting criteria are adaptive to structural constraints. We establish the upper bound for the quadratic risk of the resulting MAP estimator and the corresponding lower bound for the minimax risk over a set of admissible models of a given size. We then specify the class of priors (and, therefore, the class of complexity penalties) where for the “nearly-orthogonal” design the MAP estimator is asymptotically at least nearly-minimax (up to a log-factor) simultaneously over an entire range of sparse and dense setups. Moreover, when the numbers of admissible models are “small” (e.g., ordered variable selection) or, on the opposite, for the case of complete variable selection, the proposed estimator achieves the exact minimax rates.

Sharp oracle inequalities for aggregation of affine estimators

We consider the problem of combining a (possibly uncountably infinite) set of affine estimators in non-parametric regression model with heteroscedastic Gaussian noise. Focusing on the exponentially weighted aggregate, we prove a PAC-Bayesian type inequality that leads to sharp oracle inequalities in discrete but also in continuous settings. The framework is general enough to cover the combinations of various procedures such as least square regression, kernel ridge regression, shrinking estimators and many other estimators used in the literature on statistical inverse problems. As a consequence, we show that the proposed aggregate provides an adaptive estimator in the exact minimax sense without neither discretizing the range of tuning parameters nor splitting the set of observations. We also illustrate numerically the good performance achieved by the exponentially weighted aggregate.

Model selection and sharp asymptotic minimaxity

We obtain sharp minimax results for estimation of an n-dimensional normal mean under quadratic loss. The estimators are chosen by penalized least squares with a penalty that grows like ck log(n/k), for k equal to the number of nonzero elements in the estimating vector. For a wide range of sparse parameter spaces, we show that the penalized estimator achieves the exact minimax rate with the correct multiplication constant if and only if c equals 2. Our results unify the theory obtained by many other authors for penalized estimation of normal means. In particular we establish that a conjecture by Abramovich et al. (Ann Stat 34:584–653, 2006) is true.

Optimal calibration for multiple testing against local inhomogeneity in higher dimension

Based on two independent samples X 1, . . . , X m and X m+1, . . . , X n drawn from multivariate distributions with unknown Lebesgue densities p and q respectively, we propose an exact multiple test in order to identify simultaneously regions of significant deviations between p and q. The construction is built from randomized nearest-neighbor statistics. It does not require any preliminary information about the multivariate densities such as compact support, strict positivity or smoothness and shape properties. The properly adjusted multiple testing procedure is shown to be sharp-optimal for typical arrangements of the observation values which appear with probability close to one. The proof relies on a new coupling Bernstein type exponential inequality, reflecting the non-subgaussian tail behavior of a combinatorial process. For power investigation of the proposed method a reparametrized minimax set-up is introduced, reducing the composite hypothesis “p = q” to a simple one with the multivariate mixed density (m/n)p + (1 − m/n)q as infinite dimensional nuisance parameter. Within this framework, the test is shown to be spatially and sharply asymptotically adaptive with respect to uniform loss on isotropic Holder classes. The exact minimax risk asymptotics are obtained in terms of solutions of the optimal recovery.

What Happens in the Field Stays in the Field: Exploring Whether Professionals Play Minimax in Laboratory Experiments

The minimax argument represents game theory in its most elegant form: simple but with stark predictions. Although some of these predictions have been met with reasonable success in the field, experimental data have generally not provided results close to the theoretical predictions. In a striking study, Palacios-Huerta and Volij (2008) presented evidence that potentially resolves this puzzle: both amateur and professional soccer players play nearly exact minimax strategies in laboratory experiments. In this paper, we establish important bounds on these results by examining the behavior of four distinct subject pools: college students, bridge professionals, world-class poker players, who have vast experience with high-stakes randomization in card games, and American professional soccer players. In contrast to Palacios-Huerta and Volij's results, we find little evidence that real-world experience transfers to the lab in these games—indeed, similar to previous experimental results, all four subject pools provide choices that are generally not close to minimax predictions. We use two additional pieces of evidence to explore why professionals do not perform well in the lab: (i) complementary experimental treatments that pit professionals against preprogrammed computers and (ii) post-experiment questionnaires. The most likely explanation is that these professionals are unable to transfer their skills at randomization from the familiar context of the field to the unfamiliar context of the lab.

Weyl eigenvalue asymptotics and sharp adaptation on vector bundles

This paper examines the estimation of an indirect signal embedded in white noise on vector bundles. It is found that the sharp asymptotic minimax bound is determined by the degree to which the indirect signal is embedded in the linear operator. Thus when the linear operator has polynomial decay, recovery of the signal is polynomial where the exact minimax constant and rate are determined. Adaptive sharp estimation is carried out using a blockwise shrinkage estimator. Application to the spherical deconvolution problem for the polynomially bounded case is made.

Spectral calibration of exponential Lévy models

We investigate the problem of calibrating an exponential Lévy model based on market prices of vanilla options. We show that this inverse problem is in general severely ill-posed and we derive exact minimax rates of convergence. The estimation procedure we propose is based on the explicit inversion of the option price formula in the spectral domain and a cut-off scheme for high frequencies as regularisation.