Overlay multicast is widely accepted as an alternative to IP multicast for implementing group communications due to its easy deployment. One important issue to deal with is the node failures or ungraceful departures from the overlay multicast tree. Fast detection is a key to minimize the disruption of service to the affected nodes participating in the multicast session. In this paper, we propose a cooperative failure detection mechanism that can greatly reduce the failure detection time. We quantify three important measures, i.e., the expected detection time, the probability of false failure detection, and the overhead, and study the fundamental tradeoff among them in failure detection mechanisms. The analysis and simulations show that the proposed cooperative failure detection mechanism can significantly reduce the failure detection time while maintaining the probability of false positive at the same level, at the cost of slightly increased overhead.