Analytical form of globally optimal solution of weighted sum of intraclass separation and interclass separation

Abstract

The Fisher linear discriminant analysis (FLDA)-based method is a common method for jointly optimizing the intraclass separation and the interclass separation of the projected feature vectors by defining the objective function as the ratio of the intraclass separation over the interclass separation. To address the eigenproblem of the FLDA, a quadratic equality constraint is imposed on the square of the $$l_{2}$$ norm of the decision vector. However, the constrained optimization problem is highly nonconvex. This paper proposes to reformulate the objective function as a weighted sum of the intraclass separation and the interclass separation subject to the same quadratic equality constraint on the square of the $$l_{2}$$ norm of the decision vector. Although both the objective function and the feasible set of the optimization problem are still nonconvex, this paper shows that the global minimum of the objective functional value is equal to the minimum singular value of the Hessian matrix of the objective function. Also, the globally optimal solution of the optimization problem is in the null space of the Hessian matrix minus this singular value multiplied by the identity matrix. As it is only required to find the singular value of the Hessian matrix, no numerical optimization-based computer aided design tool is required to find the globally optimal solution. Therefore, the globally optimal solution can be found in real time. Experimental results demonstrate the effectiveness of our proposed method.