Geometrical diagnostic methods were often applied to distinguish the gravitational models. But it is scarce to investigate the differences between the different formalisms of modified gravitational theories (e.g. the metric formalism and the Palatini formalism). In this paper, we discriminate the gravitational theory with the different formalisms by using the geometrical diagnostic methods. For a considered modified theory of gravity (e.g. the f(R) theory or GBD theory), we can see that the difference between the two formalisms is remarkable according to the diagnostic results. And relative to the ΛCDM model, there are more deviations in metric formalism than those in Palatini formalism, according to the {r, s} diagnostic. Given that the GBD (generalized Brans-Dicke theory) is a time-variable Newton gravitational constant (VG) theory, the differences between the VG theory and the constant-G theory are studied. It indicates that the variation of Newton’s gravitational constant could induce notable effects on geometrical quantities (e.g. r, s and q) in both metric formalism and Palatini formalism.