The study of the dynamics of flybys is featured in several space-mission-related applications, spanning from the design of interplanetary orbits to the computation of impact probabilities with given celestial bodies in planetary protection and defense tasks. While patched-conics, two-body models suffice in accuracy for trajectory planning cases, collision probability assessment requires instead more insightful approaches. High-fidelity simulations have shown impact patterns to happen outside nominally colliding trajectories in terms of two-body analysis, all caused by distant or weak interactions between the given trajectory-celestial body pairs. This work proposes a novel concept of sphere of influence based on the Jacobian of the three-body dynamics, resulting in a wider definition of encounters that encompasses weak interactions up to an arbitrary threshold. The so-defined Jacobian spheroids are then used to construct the Keyhole map, a graphical tool that maps in the orbital elements space both nominal and off-nominal keyholes, i.e., collision paths accounting for three-body weak interactions. The proposed concepts arise from and are compared against the on-ground casualty risk assessment of the European Space Agency’s mission JUICE.