This paper proposes an adaptive protection scheme (APS) based on the original heap-based optimization (HBO) and a modified HBO (MHBO). APS is used to solve protection relays coordination problems that include directional overcurrent relays (DOCRs) as well as the distance relay’s second zone times. The complexity of the coordination problem increases with the impact of distributed generators (DGs) switching (ON/OFF). Topological changes in grid configuration frequently occur in distributing networks, equipped with DGs, causing changes in the values and direction of short circuit currents. This issue becomes a challenge for protection systems to avoid relays miscoordination and save a network’s reliability. In the proposed MHBO, the Original HBO is modified by three points, population are divided into subgroups, then they are unified into one group gradually, those subgroups are exchanging some search agents between themselves, these search agents are called travelling agents, and the last one is about, upgrading an internal equation in the original algorithm. For validating the proposed relays coordination, the IEEE 8-bus test system, and the IEEE 14-bus distribution network are selected as case studies. The obtained simulated results of the proposed algorithm show better performance compared with those obtained by the previous algorithms.