In directed energy deposition (DED), local material microstructure and tensile strength are determined by the thermal history experienced at each spatial location on the part. While prior research has investigated the effect of thermal history on mechanical properties, a tensile strength prediction model that is physically interpretable and parsimonious with good predictive accuracy is still needed. This paper investigates a data-driven predictive model with Shapley additive explanation (SHAP)-based model interpretation to address this issue. First, physically meaningful thermal features translated from prior experimental works are used as inputs to a neural network for tensile property prediction. SHAP values are then computed for the individual input features to quantify their respective influences on tensile property predictions and reduce model complexity using the metric of cumulative relative variance (CRV). Prediction of experimentally acquired Inconel 718 (IN718) tensile strength demonstrates that feature influences quantified by the developed method can be verified by findings from prior works, confirming the physical interpretability of the neural network prediction logic. In addition, model complexity reduction based on CRV has shown that fewer than 10% of the original features are required by the parsimonious model to achieve the same predictive accuracy of tensile strength as reported in prior literature, thereby demonstrating the effectiveness of SHAP-based feature reduction method in improving DED process characterization.