It is still critical to prepare a high-quality absorber layer for high-performance Cu2ZnSnSe4 (CZTSe) multi-component thin film solar cell. The gas pressure during the selenization process is commonly referred to as the pressure of inert gas in the tube furnace, while the exact selenium partial pressure is difficult to be controlled. Therefore, the grain growth under different selenium partial pressures cannot be made clear, and the film quality cannot be controlled as well. In this work, we use a sealed quartz tube as the selenization vessel, which can provide a relatively high and controllable selenium partial pressure during the selenization process. To further tailor the grain growth, lithium doping is also utilized. We find that lithium can greatly promote the growth of CZTSe films as the selenium partial pressure is controlled near the selenium saturation vapor pressure. Combined with ALD-Al2O3, the crystallization quality of CZTSe absorber films is significantly enhanced and the efficiency of CZTSe solar cells achieved a significant improvement. This work clarifies the effect of controllable Se pressure on CZTSe film growth and can lead to better results in CZTSe and other multi-compound thin film solar cells.