Third-party punishment games have consistently shown that people are willing to bear personal costs to punish others who act selfishly, even as uninvolved observers. However, the traditional third-party punishment game places participants in contrived situations that mandate direct punishment decisions, potentially inflating the prevalence of such actions compared to those observed in more naturalistic settings. In light of this obligatory nature, one might speculate that if given the autonomy to step aside, people could be inclined to forgo punishment rather than penalize unfairness. The present study developed the Situation-Selective Third-Party Punishment Game (SS-TPPG), an experimental paradigm, to investigate whether avoidance of witnessing unfairness stems from a reluctance to make punitive decisions or a desire to avoid observing unfairness altogether. Three studies (total N = 810) consistently revealed that avoidance was driven by both a reluctance to witness unfair treatment and an aversion to administering punishments. Notably, participants who typically avoided observing unfair treatment were inclined to punish when forced to observe it. Furthermore, when given the opportunity to punish indirectly, participants were less likely to avoid observing unfair distributions. These results suggest that the elevated instances of direct punishment noted in the traditional third-party punishment game might be reflective of the game's structure, which constrains participants' ability to avoid witnessing unfair distributions.