An essential aspect of the design of a pebble bed high-temperature gas-cooled reactor (HTGR) is the consideration of heat transfer. A particle-scale heat transfer model was developed based on the previous GPU-DEM numerical model. This model includes the heat conduction of the particles in contact, the radiative heat transfer among particles, and unidirectional convection heat transfer in the flow field. The comparison of the predicted results with the results of the TF-PBEC experiment demonstrates the accuracy of the conduction and radiation modules within the current model. Based on the current model and the porous media model, a numerical study of the natural convection heat transfer in the High-Temperature gas-cooled Reactor-Pebble-bed Module (HTR-PM) after reactor shutdown is performed. This paper systematically introduces the model design, pebble bed generation, flow field calculation, and heat transfer simulation. The results indicate that natural convection contributes to removing residual heat in the reactor core, and a higher pressure leads to better convection heat transfer. At extremely low Reynolds numbers (Re < 10), convection heat transfer is comparable to conduction and radiation, while for pressurized pebble beds (Re = 50–500), convection heat transfer dominates. Furthermore, the influence of different models used to calculate the Nusselt number of the results is compared and analyzed. For most cases, the differences in the temperature distribution of the pebble bed obtained from the different correlations are minor. However, the impact of porosity on the local Nusselt number cannot be ignored in the extremely low Reynolds numbers flows. Overall, the particle-scale heat transfer model can provide more effective reference information for the design and construction of an HTGR.