Hydrogen can be a critical part of the energy transition and salt caverns have been considered for underground hydrogen storage. Although there have been some successful experiences of hydrogen storage in salt caverns in Europe and the United States, it is essential to exercise caution when selecting caverns for storage to ensure safe operations. This study investigates the relationship between rock heterogeneity and helium diffusion by testing salt-rock cores from the Lotsberg Formation, Alberta, Canada. We characterized salt rock heterogeneity by analyzing the results of X-ray diffraction (XRD) analysis, thin-section microscopy, scanning electron microscopy/energy dispersive X-ray spectrometry (SEM/EDS), and high-resolution CT scans. Then, diffusion tests were conducted using a custom-designed HPHT visualization cell on core samples with different compositions and crystal sizes. The samples in this study were categorized as Lotsberg Salt and Lotsberg Marlstone. Lotsberg Salt mainly consisted of halite with trace amounts of carbonate impurities, while Lotsberg Marlstone comprised halite with a carbonate matrix containing carbonates, clays, quartz, and muscovite. Helium diffusion through pure and intact halite crystals was negligible, and the micro-scale halite crystal boundaries are non-open or poorly unconnected for helium diffusion. Halite samples with macro-scale grain boundaries and carbonate impurities exhibited more interconnected pore networks, resulting in increased rates of helium diffusion. However, the formation of secondary halite veins between grain boundaries can lead to tightly sealed boundaries that have poor connectivity, limiting helium diffusion within the sample. Salt-rock cores with intergranular fractures and unsealed grain boundaries showed accelerated pressure declines, attributed to the preferential diffusion of helium through these fracture and crack pathways.