In the matrix model approaches of string/M theories, one starts from a generic symmetry gl(∞) to reproduce the space-time manifold. In this paper, we consider the generalization in which the space-time manifold emerges from a gauge symmetry algebra which is not necessarily gl(∞). We focus on the second nontrivial example after the toroidal compactification, the coset space G/H, and propose a specific infinite-dimensional symmetry which realizes the geometry. It consists of the gauge-algebra valued functions on the coset and Lorentzian generator pairs associated with the isometry. We show that the 0-dimensional gauge theory with the mass and Chern-Simons terms gives the gauge theory on the coset with scalar fields associated with H.