We establish the mathematical fundamentals for a unified description of curvature, torsion, and non-metricity 2-forms in the way extending the so-called Möbius representation of the affine group, which is the method to convert the semi-direct product into the ordinary matrix product, to revive the fertility of gauge theories of gravity. First of all, we illustrate the basic concepts for constructing the metric-affine geometry. Then the curvature and torsion 2-forms are described in a unified manner by using the Cartan connection of the Möbius representation of the affine group. In this unified-description, the curvature and torsion are derived by Cartan’s structure equation with respect to a common connection 1-form. After that, extending the Möbius representation, the dilation and shear 2-forms, or equivalently, the non-metricity 2-form, are introduced in the same unified manner. Based on the unified-description established in this paper, introducing a new group parametrization and applying the Inönü–Wigner group contraction to the full theory, the relationships among symmetries, geometric quantities, and geometries are investigated with respect to the three gauge groups: the metric-affine group and its extension, and an extension of the (anti)-de Sitter group in which the non-metricity exists. Finally, possible applications to theories of gravity are briefly discussed.