Schur Complement Trick for Positive Semi-definite Energies

Abstract

The “Schur complement trick” appears sporadically in numerical optimization methods [Schur 1917; Cottle 1974]. The trick is especially useful for solving Lagrangian saddle point problems when minimizing quadratic energies subject to linear equality constraints [Gill et al. 1987]. Typically, to apply the trick, the energy’s Hessian is assumed positive definite. I generalize this technique for positive semi-definite Hessians. 1 Positive definite energies Let us consider a quadratic energy optimization problem subject to linear equality constraints: minimize x 1 2 xAx− xf + constant, (1) subject to Bx = g, (2) where x, f ∈ R, A ∈ Rn×n, B ∈ Rm×n and g ∈ R. Solving with the Lagrange multiplier method results in a system of linear [ A B B 0 ] [ x λ ] = [ f g ] , where λ ∈ R is a vector of Lagrange multipliers. To retain generality, let us replace the zero block in our system matrix with a variable C. [ A B B C ] [ x λ ] = [ f g ] , where λ ∈ R is a vector of Lagrange multipliers. By assuming that A is positive definite, the Schur complement trick proceeds by multiplying the first set of equations by BA−1: BAAx+BABλ = BAf , (3) Bx+BABλ = BAf . (4) Now, substitute the second set of equations Bx+Cλ = g for Bx and solve the resulting equation for λ: (g −Cλ) +BABλ = BAf , (5) (BAB −C)λ = BAf − g, (6) λ = (BAB −C) ( BAf − g ) . (7) Finally, find the primary solution by solving the first equation using the newly found values for λ: x = A(f −Bλ). Assuming a factorization of A may be precomputed, this trick allows quickly solving optimization problems involving the same energy Hessian A, but different linear coefficients f and different constraints Bx = g. So long as the number of constraints is significantly smaller than the number of variables (m n), the cost of solving against the Schur complement (BA−1BT − C) will be small compared to a full factorization (e.g. LDLT) of the system matrix [AB;BC]. This trick is beneficial in scenarios where the energy is fixed but a small number of constraints are changing. m