One of the applications of energy storage systems (ESSs) is to support frequency regulation in power systems. In this paper, we consider such an application and address the challenges of uncertain frequency changes, limited energy storage, as well as distribution network constraints. We formulate a bi-level optimization problem that includes the operation objectives of the system operator and the ESSs, using chance constraints to account for uncertain frequency changes. The frequency regulation decision of the system operator depends on the ESSs decision to participate in the regulation service as well as the distribution network constraints. Due to the interdependencies between the ESSs demand fluctuations and the distribution network power flow changes, the system operator requires the ESSs’ operation information for frequency regulation decisions, which may not be available from the ESSs. Therefore, we propose a decentralized algorithm such that the system operator and the ESSs can pursue their own operation objectives, while ensuring the distribution network constraints are satisfied. We evaluate the performance of our method on IEEE 37-bus and 123-bus test feeders by considering combinations of ESSs with different sizes. Simulation results demonstrate that our approach can successfully coordinate the ESSs to regulate the frequency deviations.