A description is given of a general closure principle involving the minimization of the mean square error. The procedure based upon this principle can be applied to the truncation of the BBGKY hierarchy at various stages and to the approximation of unwanted terms arising in the equation of motion method by linear combinations of the observables to be retained. On a general level the significance of the closure principle is described in terms of the geometry of function space, and several useful general properties of the principle are derived. A discussion is devoted to the relation between the closure error (i.e., the least mean square error) and the error in the end result (e.g., the free energy, the radial distribution function, etc.); however, the results, while providing some insight, are not sufficiently refined to provide upper bounds to errors in all problems of statistical mechanics where the method is applicable. On the level of specific application it is shown that the principle yields results identical to the random phase approximation and to the linearized version of the Kirkwood superposition approximation in two special cases. Later sections of the paper describe in greater than usual generality, the formalism connecting thermodynamic properties and other equilibrium properties with the microscopic equations of motion in which closure approximations have been introduced. Two illustrative examples of the application of the over-all method were made to the case of a classical system of electrons in a uniform background of compensating charge, one leading to the well-known results of Debye and the other to a more accurate and elaborate theory developed in quantitative detail elsewhere.