Scene memory is prone to systematic distortions potentially arising from experience with the external world. Boundary transformation, a well-known memory distortion effect along the near-far axis of the three-dimensional space, represents the observer's erroneous recall of scenes' viewing distance. Researchers argued that normalization to the prototypical viewpoint with the high-probability viewing distance influenced this phenomenon. Herein, we hypothesized that the prototypical viewpoint also exists in the vertical Angle of View (AOV) dimension and could cause memory distortion along scenes' vertical axis. Human subjects of both sexes were recruited to test this hypothesis and two behavioral experiments were conducted, revealing a systematic memory distortion in the vertical AOV in both the forced choice (n = 79) and free adjustment (n = 30) tasks. Furthermore, the regression analysis implied that the complexity information asymmetry in scenes' vertical axis and the independent subjective AOV ratings from a large set of online participants (n = 1208) could jointly predict AOV biases. Furthermore, in a functional Magnetic Resonance Imaging (fMRI) experiment (n = 24), we demonstrated the involvement of areas in the ventral visual pathway (V3/V4, PPA, and OPA) in AOV bias judgment. Additionally, in a Magnetoencephalography (MEG) experiment (n = 20), we could significantly decode the subjects' AOV bias judgments ∼140 ms after scene onset and the low-level visual complexity information around the similar temporal interval. These findings suggest that AOV bias is driven by the normalization process and associated with the neural activities in the early stage of scene processing.Significance Statement Perceiving a scene with high precision is critical for our navigation and interaction with the surrounding environment. However, systematic memory distortion is quite common. Herein, we discovered that scene memory could be distorted to the upper or lower visual field. According to the behavioral results, multiple measures of scenes' complexity information were critically involved in the formation of this memory distortion. Furthermore, the results support the normalization theory regarding the existence of a high-probability prototypical viewpoint in scene processing. Our findings also suggested that the normalization process-induced scene memory distortion could be beneficial for the observer's future action selection. The identified complexity measures could be used in designing Artificial Intelligence (AI) systems with navigational functions.