Proton dose deposition results are influenced by various factors, such as irradiation angle, beamlet energy and other parameters. The calculation of the proton dose deposition matrix (DDM) can be highly complex but is crucial in intensity-modulated proton therapy (IMPT). In this work, we present a novel deep learning (DL) approach using multi-source features for proton DDM prediction. The DL5 proton DDM prediction method involves five input features containing beamlet geometry, dosimetry and treatment machine information like patient CT data, beamlet energy, distance from voxel to beamlet axis, distance from voxel to body surface, and pencil beam (PB) dose. The dose calculated by Monte Carlo (MC) method was used as the ground truth dose label. A total of 40 000 features, corresponding to 8000 beamlets, were obtained from head patient datasets and used for the training data. Additionally, seventeen head patients not included in the training process were utilized as testing cases. The DL5 method demonstrates high proton beamlet dose prediction accuracy, with an average determination coefficient R 2 of 0.93 when compared to the MC dose. Accurate beamlet dose estimation can be achieved in as little as 1.5 milliseconds for an individual proton beamlet. For IMPT plan dose comparisons to the dose calculated by the MC method, the DL5 method exhibited gamma pass rates of γ(2 mm, 2%) and γ(3 mm, 3%) ranging from 98.15% to 99.89% and 98.80% to 99.98%, respectively, across all 17 testing cases. On average, the DL5 method increased the gamma pass rates to γ(2 mm, 2%) from 82.97% to 99.23% and to γ(3 mm, 3%) from 85.27% to 99.75% when compared with the PB method. The proposed DL5 model enables rapid and precise dose calculation in IMPT plan, which has the potential to significantly enhance the efficiency and quality of proton radiation therapy.