Metaverse has been going viral across the globe, and virtual reality head‐mounted displays (VR‐HMDs) as the basic infrastructure and entrance to cater to this evolution of social interaction are showing rapid development recently. Pancake optics based on optical path folding provides the feasibility of VR‐HMDs in the form of glasses and is wide recognition. The pancake system has become the current mainstream owing to its large exit pupil diameter (EPD), high resolution, novel compact, and lightweight. Owing to the use of polarization elements, the birefringence in the lens will change the polarization state of the light, resulting in uneven brightness and stray light, seriously reducing the optical performance of the system, so it is necessary to control the birefringence of the lens. In this study, we proposed a pancake system based on a 2.1‐inch LCD display which achieves 10 mm EPD at 11 mm, diagonal field of view (FOV) of 96°, overall length (OAL) of the system less than 20 mm, and support diopter adjustment from ‐7D to ‐1D. The residual birefringence of the lens can be effectively suppressed in four stages: 1) controlling the thickness ratio between the center and the edge of the lens during the optical design, 2) setting a reasonable position and size of the injection gate and cooling system during the mold design process, 3) adopting reasonable injection parameters during the precision injection molding process and 4) eliminating the birefringence of the lens with the lens annealing process. Ultra‐precision machining and correction of the mold cores have been employed to achieve high surface accuracy and low birefringence of the lens. The optomechanical design, assembly, and adjustment process are discussed. Finally, the prototype of the pancake system is experimentally tested, and the results demonstrate pretty well.