The signed distance field-based discrete element method (SDF-DEM) has demonstrated significant success in various applications; however, a key challenge hindering its widespread adoption lies in the establishment of its connection with conventional contact models. To address this challenge, this study introduces two formulations of contact potential within SDF-DEM, drawing analogies to both linear and Hertzian contact models. Comprehensive relationship between the parameters of the proposed contact potentials of SDF-DEM and those of conventional contact models is established. The energy conservation characteristic of SDF-DEM is verified through a two-particle colliding and bouncing test, and the critical timestep issue is investigated and addressed. Discrete element simulations are conducted for a triaxial compression test and a rockfall test, involving both spherical particles and general irregularly shaped particles. The results underscore the accuracy and numerical stability of the SDF-DEM with the developed potential models. This work is anticipated to contribute not only to advancing the understanding of SDF-DEM and the potential-based contact theory but also to providing robust framework that bridges the gap of SDF-DEM with conventional models.