Widely linear (WL) models have been demonstrated to be superior to conventional strictly linear models for the estimation of noncircular complex and quaternion signals. Existing studies on their performance bounds focus on the analysis of mean square error (MSE). However, the single degree of freedom within standard MSE allows for only the minimization of error power, with no means to understand how the error contribution is distributed across the data channels. To this end, we introduce novel complex and quaternion valued complementary quadratic cost functions for complex and quaternion signal estimation, which are extensions of the recently proposed complementary MSE metric. It is shown that for WL minimum MSE estimation and least squares regression, the complementary cost function and the standard cost function attain the same stationary point. We also show that for the former, the stationary point is a saddle point. This novel finding provides insight into the performance of complex and quaternion WL estimators, and offers a rigorous foundation for further developments in this field.