Retrofit heat exchanger network (HEN) optimization is a fundamentally unique problem which requires the consideration of existing structures, compared to grassroots design problems. The optimization of retrofit HENs is particularly difficult due to the integration of both existing and newly acquired equipment. The re-routing of existing equipment can lead to various network topologies, increasing the complexity of considerations. In this work, we exploit the P-graph framework to solve retrofit HEN problems, guaranteeing to find the topology of optimal solutions within the constrained space of the HEN retrofit problem. The P-graph framework has additional advantages that allows topologically-efficient search space, simplifies additional unit placement, considers unit positional swapping (re-sequencing and re-piping within search constraints), considers stream splitting, and n-best solution visualization. The pinch minimum utility constraint also provides a bound for the maximum number of modifications in the HEN, significantly reducing search space. The proposed P-graph-based approach is demonstrated using a real refinery case study to show its capability in obtaining the topology of the optimal HEN, highlighting the economic and energy benefits. Further extensions to other retrofit process integration problems (e.g. retrofit water network, hydrogen network etc.) will be enabled via the proposed P-graph approach.