The fog computing paradigm brings together storage, communication, and computation resources closer to users’ end-devices. Therefore, fog servers are deployed at the edge of the network, offering low latency access to users. With the expansion of such fog computing services, different providers will be able to deploy multiple resources within a restricted geographical proximity. In this paper, we investigate an incentive-based cooperation scheme across fog providers. We propose a distributed cooperative algorithm amongst fog computing providers where fully collaborative fog nodes are subject to different loads. The proposed algorithm leverages the power-of-two result and exploits a cooperation probability, namely the probability that a given provider collaborates by accepting a computation request from another provider, as a mean to achieve a fair cooperation. We adopt an analytical approach based on exploiting a simplified performance model to demonstrate numerically that a set of optimal accepting probabilities exits when the number of server nodes goes to infinity. This result then drives the design of our distributed algorithm. Second, in our experimental approach, we perform a set of simulation analysis to verify the validity of the proposed solution when the number of servers is limited.