The heat transfer characteristics of regenerator in regenerative coolers such as pulse tube and Stirling coolers is important. Various methodologies of characterizing the Nusselt number between the regenerator matrix and working fluid have been established and applied. However, most of the existing formulas are based on the study of steady flow or oscillating flow with low pressure ratios at above ambient temperature. In this paper, a tube model with finite wall thickness was constructed to numerically simulate simplified #400 SS mesh and packed HoCu2 sphere, with the aim of investigating the Nusselt number in regenerators under various flow conditions, including different Valensi numbers and temperature ranges. The results provide more evidence to support that within the operating conditions, including Valensi numbers and pressure ratios typically used in coolers, the Nusselt number of oscillating flow and steady flow are similar. Additionally, a heat transfer deterioration effect is observed below 40 K, which is related to the heat capacity ratio between the solid matrix and working fluid. Based on these findings, the study emphasizes the importance of further investigating the heat transfer characteristics of oscillating flow at low temperatures.