Least squares parameter estimation and multi-innovation least squares methods for linear fitting problems from noisy data

Abstract

Least squares is an important method for solving linear fitting problems and quadratic optimization problems. This paper explores the properties of the least squares methods and the multi-innovation least squares methods. It demonstrates lemmas and theorems about the least squares and multi-innovation least squares parameter estimation algorithms after reviewing and surveying some important contributions in the area of system identification, such as the auxiliary model identification idea, the multi-innovation identification theory, the hierarchical identification principle, the coupling identification concept and the filtering identification idea. The results of the least squares and multi-innovation least squares algorithms for linear regressive systems with white noises can be extended to other systems with colored noises.