Symmetry protected topological (SPT) states have boundary 't Hooft anomalies that obstruct an effective boundary theory realized in its own dimension with UV completion and an on-site $G$-symmetry. In this work, yet we show that a certain anomalous non-on-site $G$ symmetry along the boundary becomes on-site when viewed as an extended $H$ symmetry, via a suitable group extension $1\to K\to H\to G\to1$. Namely, a non-perturbative global (gauge/gravitational) anomaly in $G$ becomes anomaly-free in $H$. This guides us to construct exactly soluble lattice path integral and Hamiltonian of symmetric gapped boundaries, always existent for any SPT state in any spacetime dimension $d \geq 2$ of any finite symmetry group, including on-site unitary and anti-unitary time-reversal symmetries. The resulting symmetric gapped boundary can be described either by an $H$-symmetry extended boundary of bulk $d \geq 2$, or more naturally by a topological emergent $K$-gauge theory with a global symmetry $G$ on a 3+1D bulk or above. The excitations on such a symmetric topologically ordered boundary can carry fractional quantum numbers of the symmetry $G$, described by representations of $H$. (Apply our approach to a 1+1D boundary of 2+1D bulk, we find that a deconfined gauge boundary indeed has spontaneous symmetry breaking with long-range order. The deconfined symmetry-breaking phase crosses over smoothly to a confined phase without a phase transition.) In contrast to known gapped interfaces obtained via symmetry breaking (either global symmetry breaking or Anderson-Higgs mechanism for gauge theory), our approach is based on symmetry extension. More generally, applying our approach to SPT, topologically ordered gauge theories and symmetry enriched topologically ordered (SET) states, leads to generic boundaries/interfaces constructed with a mixture of symmetry breaking, symmetry extension, and dynamical gauging.