Estimation in Linear Models

Abstract

This chapter studies the estimation problem for a linear model. The first four sections are fairly classical and the presented results are based on the direct analysis of the linear estimation procedures. Sections 4.5 and 4.6 reproduce in a very short form the same results but now based on the likelihood analysis. The presentation is based on the celebrated chi-squared phenomenon which appears to be the fundamental fact yielding the exact likelihood-based concentration and confidence properties. The further sections are complementary and can be recommended for a more profound reading. The issues like regularization, shrinkage, smoothness, and roughness are usually studied within the nonparametric theory; here we try to fit them to the classical linear parametric setup. A special focus is on semiparametric estimation in Sect. 4.9. In particular, efficient estimation and chi-squared result are extended to the semiparametric framework.The main tool of the study is the quasi maximum likelihood method. We especially focus on the validity of the presented results under possible model misspecification. Another important issue is the way of measuring the estimation loss and risk. We distinguish below between response estimation or prediction and the parameter estimation. The most advanced results like chi-squared result in Sect. 4.6 are established under the assumption of a Gaussian noise. However, a misspecification of noise structure is allowed and addressed.