Abstract
We consider the sparse regression model where the number of parameters p is larger than the sample size n. The difficulty when considering high-dimensional problems is to propose estimators achieving a good compromise between statistical and computational performances. The Lasso is solution of a convex minimization problem, hence computable for large value of p. However stringent conditions on the design are required to establish fast rates of convergence for this estimator. Dalalyan and Tsybakov [17–19] proposed an exponential weights procedure achieving a good compromise between the statistical and computational aspects. This estimator can be computed for reasonably large p and satisfies a sparsity oracle inequality in expectation for the empirical excess risk only under mild assumptions on the design. In this paper, we propose an exponential weights estimator similar to that of [17] but with improved statistical performances. Our main result is a sparsity oracle inequality in probability for the true excess risk.
Highlights
We observe n independent pairs (X1, Y1), ..., (Xn, Yn) ∈ X × R such that (1.1)Yi = f (Xi) + Wi, 1 i n, where f : X → R is the unknown regression function and the noise variablesW1, . . . , Wn are independent of the design (X1, . . . , Xn) and such that EWi = 0 and EWi2 σ2 for some known σ2 > 0 and any 1 i n
Dalalyan and Tsybakov [19, 20, 21, 22] propose an exponential weights procedure related to the PAC-Bayesian approach with good statistical and computational performances
We propose to study two exponential weights estimation procedures
Summary
We observe n independent pairs (X1, Y1), ..., (Xn, Yn) ∈ X × R (where X is any measurable set) such that (1.1). Dalalyan and Tsybakov [19, 20, 21, 22] propose an exponential weights procedure related to the PAC-Bayesian approach with good statistical and computational performances. They consider deterministic design, establishing their statistical result only for the empirical excess risk instead of the true excess risk R(·) − R(θ). Note that in a work parallel to ours, Rigollet and Tsybakov [46] consider exponentially weighted aggregates with discrete priors and suggest another version of the Metropolis-Hastings algorithm to compute their estimator.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
