Stability and precision of atomic devices are closely tied to the quality and stability of the internal atmosphere of the atomic vapor cells on which they rely. Such an atmosphere can be stabilized by building the cell with low permeation materials such as sapphire or aluminosilicate glass in microfabricated devices. Recently, we have shown that permeation barriers made of Al2O3 thin-film coatings deposited on standard borosilicate glass could be an alternative for buffer gas pressure stabilization. In this study, we, hence, investigate how helium permeation is influenced by the thickness, ranging from 5 to 40 nm, of such Al2O3 thin films coated by atomic layer deposition. Permeation rates are derived from long-term measurements of the pressure-shifted transition frequency of a coherent population trapping (CPT) atomic clock. From thicknesses of 20 nm onward, a significant enhancement of the cell hermeticity is experienced, corresponding to two orders of magnitude lower helium permeation rate. In addition, we test cesium vapor cells filled with neon as a buffer gas and whose windows are coated with 20 nm of Al2O3. As for helium, the permeation rate of neon is significantly reduced, thanks to alumina coatings, leading to a fractional frequency stability of 4×10−12 at 1 day when the cell is used in a CPT clock. These features outperform the typical performances of uncoated Cs–Ne borosilicate cells and highlight the significance of Al2O3 coatings for buffer gas pressure stabilization.