Motivated by symmetry-protected topological phases (SPTs) with both spatial symmetry (e.g., lattice rotation) and internal symmetry (e.g., spin rotation), we propose a class of exotic topological terms, which generalize the well-known Wen-Zee topological terms of quantum Hall systems [X.-G. Wen and A. Zee, Phys. Rev. Lett. 69, 953 (1992)]. These generalized Wen-Zee terms are expressed as wedge product of spin connection and usual gauge fields (1-form or higher) in various dimensions. In order to probe SPT orders, we externally insert "symmetry twists" like domain walls of discrete internal symmetry and disclinations that are geometric defects with nontrivial Riemann curvature. Then, generalized Wen-Zee terms simply tells us how SPTs respond to those symmetry twists. Classifying these exotic topological terms thus leads to a complete classification and characterization of SPTs within the present framework. We also propose SPT low-energy field theories, from which generalized Wen-Zee terms are deduced as topological response actions. Following the Abstract of Wen-Zee paper, our work enriches alternative possibilities of condensed-matter realization of unification of electromagnetism and "gravity".