The study of non-equilibrium quantum dynamics receives attentions recently. However, the nature and effects of nonequilibirumness such as detailed-balance breaking and the relationship to the underlying intrinsic geometry is still unclear. In this study, we uncover that a gauge field will be induced by the nonequilibriumness in the coherence representation and furthermore its internal geometrical curvature is directly related to the degree of detailed balance breaking. The nonequilibriumness of the quantum system induces an intrinsic geometric curvature which can enhance the quantum coherence, leading to the possibility of the space time origin for the non-local quantum correlations or the possibility of curved space time emergence from nonequilibrium quantum dynamics. We also uncovered that the internal curvature of the gauge field provides a bridge to connect the generalized quantum fluctuation dissipation theorem to the fluctuation theorem and time irreversibility of the quantum dynamics. The quantum time irreversibility is due to the path dependent factor along any particular path in an internal curved space, which is analogous to the Wilson lines (or Wilson loops) in Abelian gauge theory. We also found that the steady state quantum coherence disappears when the non-trivial internal curvature vanishes for the quantum system coupled with environments. When the curvature is relatively small indicating weak detailed balance breaking, the coherence increases as curvature increases. The internal curvature can provide a general and direct quantitative measure of the detail-balance breaking for any quantum/classical non-equilibrium systems, even without knowing the underlying steady state distribution or the steady state flux. Using an example of two harmonic oscillators coupled to two environments with different temperatures, we explicitly show the dependence of the internal curvature and quantum coherence.