LetG be a compact group of transformation (global symmetry group) of a manifoldE (multidimensional universe) with all orbits of the same type (one stratum). We studyG invariant metrics onE and show that there is one-to-one correspondence between those metrics and triples (g μv,A äμ ,h αβ), whereg μv is a (pseudo-) Riemannian metric on the space of orbits (space-time),A äμ is a Yang-Mills field for the gauge groupN|H, whereN is the normalizer of the isotropy groupH inG, andh αβ are certain scalar fields characterizing geometry of the orbits (internal spaces). The scalar curvature ofE is expressed in terms of the component fields onM. Examples and model building recipes are also given. The results generalize those of non-abelian Kaluza-Klein theories to the case where internal spaces are not necessarily group manifolds.