An important way to maintain human cooperation is punishing defection. However, since punishment is costly, how can it arise and evolve given that individuals who contribute but do not punish fare better than the punishers? This leads to a violation of causality, since the evolution of punishment is prior to the one of cooperation behaviour in evolutionary dynamics. Our public goods game computer simulations based on generalized Moran Process, show that, if there exists a 'behaviour-based sympathy' that compensates those who punish at a personal cost, the way for the emergence and establishment of punishing behaviour is paved. In this way, the causality violation dissipates. Among humans sympathy can be expressed in many ways such as care, praise, solace, ethical support, admiration, and sometimes even adoration; in our computer simulations, we use a small amount of transfer payment to express 'behaviour-based sympathy'. Our conclusions indicate that, there exists co-evolution of sympathy, punishment and cooperation. According to classical philosophy literature, sympathy is a key factor in morality and justice is embodied by punishment; in modern societies, both the moral norms and the judicial system, the representations of sympathy and punishment, play an essential role in stable social cooperation.