Depth of field (DoF) defines the depth range of a 3D scene that can be observed by viewers. In the conventional lens-based imaging process, the diffraction effect of lens limits the display depth range, which has an adverse effect on the display quality of lens-based light-field display systems and it is the major obstacle to the practical application of light-field display. Aiming to break through the inherent shallow DOF characteristics of lens and obtain better display effect, an extended depth of field method with a designed diffraction optical element (DOE) which combined multi-depth fusion and end-to-end optimization is proposed. By analyzing the effect of diffraction on optical imaging process, a DOE enables balance display quality over the whole depth range is designed, and a pre-filtering convolutional neural network (CNN) is constructed to complete end-to-end optimization framework and avoid image blur. The validity of the proposed method is confirmed by simulation experiments and the depth range is extended effectively without display sharpness declining severely.