Electric distribution utilities are required to continuously deliver reliable electric power to their customers. Regulatory utility commissions often practise reward and penalty schemes to regulate reliability performance of utility companies annually with respect to a desired performance targets. However, the conventional regulation procedures are commonly found based on the customer-based standard reliability indices, which are not able to discern the service characteristics behind the electric meters and, hence, fail to holistically characterise the actual impact of electricity interruption. This study proposes a new method to evaluate the load-based reliability indices in power distribution systems using advanced metering infrastructure data. Furthermore, the authors introduce a reward/penalty regulation scheme for utility regulators to provide a reliability oversight using the proposed load-based reliability metrics. The new load-based reliability metric and the reward/penalty scheme proposed bring about superior advantages as the distribution grids become further complex with a high penetration of distributed energy resources and enabled microgrid flexibilities. Numerical analyses on different settings with and without microgrid considerations reveal the applicability and effectiveness of the proposed approach in real-world scenarios.