The single-lens imaging system enhances image quality through restoration algorithm, which has the advantage of miniaturization compared with complex multi-element lens. However, in the design of large field of view (FoV), the single lens faces greater challenges in correcting off-axis aberrations compared to complex multi-element lens. It is more susceptible to spatially varying degradation, and the Point Spread Function (PSF) exhibits significant spatial variation. The previous restoration methods are not accurate in the representation of PSF model with spatial variation, and have limited effect on the blur restoration of large FoV single lens. In this work, we perform more accurate characterization of PSF’s change pattern property as well as shape and energy distribution property. To this end, we first use a PSF changing pattern fitting process to obtain the PSF continuous changing pattern of the whole FoV. This pattern is then used as a prior input into a spatially varying restoration network to achieve the restoration results. The results on the simulation dataset show that our design can improve the PSNR from 20.37 to 28.62, and can achieve a large FoV imaging system of 84°, while we still get better performance than the previous method on the real shot dataset.