Domain walls between different topological phases are one of the most interesting phenomena that reveal the non-trivial bulk properties of topological phases. Very recently, gapped domain walls between different topological phases have been intensively studied. In this paper, we systematically construct a large class of lattice models for gapless domain walls between twisted and untwisted gauge theories with arbitrary finite group $G$. As simple examples, we numerically study several finite groups(including both Abelian and non-Abelian finite group such as $S_3$) in $2$D using the state-of-the-art loop optimization of tensor network renormalization algorithm. We also propose a physical mechanism for understanding the gapless nature of these particular domain wall models. Finally, by taking advantage of the classification and construction of twisted gauge theories using group cohomology theory, we generalize such constructions into arbitrary dimensions, which might provide us a systematical way to understand gapless domain walls and topological quantum phase transitions.