Phase-field models (PFMs) have proven to accurately predict complex crack patterns like crack branching, merging, and crack fragmentation, but they are computationally costly. Adaptive mesh refinement (AMR) algorithms based on nodal damage are recently developed to reduce computational expense. However, these AMR algorithms are unable to simulate the crack initiation accurately without a priori local refinement. To solve this problem, in this work, we have proposed an AMR algorithm based on the effective crack driving energy. A multi-level mark–unmark scheme is developed integrating the effective crack driving energy-based followed by a damage-based scheme, which can capture the crack initiation and propagation very effectively. The proposed AMR algorithm works efficiently on brittle, cohesive, and dynamic fractures. The three popular PFMs, namely AT1, AT2, and PF-CZM are selected from the literature, implemented within a single code, and used to study the AMR algorithm. The proposed AMR algorithm reduces the simulation time by 5–50 times, depending on the type of problem, as compared to simulations that adopt a priori non-adaptively refined meshes.