In order to obtain the calculation method of the unloading area of the internal gear pump during oil trapping, a pair of internal gears including an external gear and an internal gear was used as the research object to simulate the oil trapping process. The geometric relationship during the meshing process was established, and the unloading area expression was obtained by using the geometric pattern expansion method with the variable f as the independent variable. Guided by a mathematical model, two improved optimization schemes were proposed for the internal gear tooth profile, and the unloading area expressions sud, suda, and sudb were obtained. Taking the meshing gear pair with module 3 and number of teeth 13/19 as examples, the simulation results were very consistent with the existing literature. The reliability of the model and the feasibility of the optimization scheme are obtained based on the theoretical analysis and calculation results. This calculation method of unloading area can be applied to the same type of gear pump design in the future, providing a reference for the design of high pressure and low noise gear pumps.