The Dual of the Maximum Likelihood Method

Abstract

The Maximum Likelihood method estimates the parameter values of a statistical model that maximize the corresponding likelihood function, given the sample information. This is the primal approach that, in this paper, is presented as a mathematical programming specification whose solution requires the formulation of a Lagrange problem. A remarkable result of this setup is that the Lagrange multipliers associated with the linear statistical model (regarded as a set of constraints) is equal to the vector of residuals scaled by the variance of those residuals. The novel contribution of this paper consists in developing the dual model of the Maximum Likelihood method under normality assumptions. This model minimizes a function of the variance of the error terms subject to orthogonality conditions between the errors and the space of explanatory variables. An intuitive interpretation of the dual problem appeals to basic elements of information theory and establishes that the dual maximizes the net value of the sample information. This paper presents the dual ML model for a single regression and for a system of seemingly unrelated regressions.