Minimax Rates of ℓp-Losses for High-Dimensional Linear Errors-in-Variables Models over ℓq-Balls.

Xin Li,Dongya Wu

doi:10.3390/e23060722

Abstract

In this paper, the high-dimensional linear regression model is considered, where the covariates are measured with additive noise. Different from most of the other methods, which are based on the assumption that the true covariates are fully obtained, results in this paper only require that the corrupted covariate matrix is observed. Then, by the application of information theory, the minimax rates of convergence for estimation are investigated in terms of the -losses under the general sparsity assumption on the underlying regression parameter and some regularity conditions on the observed covariate matrix. The established lower and upper bounds on minimax risks agree up to constant factors when , which together provide the information-theoretic limits of estimating a sparse vector in the high-dimensional linear errors-in-variables model. An estimator for the underlying parameter is also proposed and shown to be minimax optimal in the -loss.

Highlights

In various fields of applied sciences and engineering, such as machine learning [1], a fundamental problem is to estimate an underlying parameter β∗ ∈ Rd of a linear regression model as follows yi = h Xi·, β∗ i + ei, for i = 1, 2, . . . , n, (1)where {( Xi·, yi )}in=1 are i.i.d. observations (Xi· ∈ Rd ) and e ∈ Rn is the random noise
We turn to our main results on lower and upper bounds on minimax risks
Let Pβ denote the distribution of y in the linear regression model with additive errors, when β is given and Z is observed

Summary

Introduction

Xn· )> ∈ Rn×d and y, e ∈ Rn. The covariates Xi· N) are always assumed to be fully observed in standard formulations. This assumption is far away from reality since, in general, the measurement error cannot be avoided. In many real-world applications, due to the lack of observation or the instrumental constraint, the collected data, such as remote sensing data, may always be perturbed and tend to be noisy [2]. It has been shown in [3]

Objectives

Results

Conclusion