Ehrenfest urns with interaction that are connected in a ring is considered as a paradigm model for non-equilibrium thermodynamics and is shown to exhibit two distinct non-equilibrium steady states (NESS) of uniform and non-uniform particle distributions. As the inter-particle attraction varies, a first order non-equilibrium phase transition occurs between these two NESSs characterized by a coexistence regime. The phase boundaries, the NESS particle distributions near saddle points and the associated particle fluxes, average urn population fractions, and the relaxational dynamics to the NESSs are obtained analytically and verified numerically. A generalized non-equilibrium thermodynamics law is also obtained, which explicitly identifies the heat, work, energy and entropy of the system.