Establishing a unified fatigue life prediction model and quantifying the uncertainty in the mechanical behavior of materials are critical to ensure the structural integrity and equipment performance. For the commonly-used strain-based fatigue methods, existing estimation methods exhibit inevitable deviations, while data-driven methods have shown poor extrapolation ability and interpretability. Therefore, this paper aims to develop a probabilistic framework for strain-based fatigue life prediction and uncertainty quantification (UQ) to provide an indication for fatigue design/assessment using interpretable machine learning (ML) techniques. Based on Shapley additive explanations (SHAP) and symbolic regression (SR), interpretable prediction models with concise expressions and outstanding prediction performance are established and optimized according to the priori physical knowledge. Moreover, accounting for the material variability, the probabilistic assessment with UQ excellently validates the prediction model, and quantifies the variability of ε-N curves. The proposed framework provides valuable reference and shows promising prospects in fatigue design for engineering components.