With coupled dynamic interactions of vehicle-bridge-wind-wave (VBWW) system, fatigue damage accumulations at complicated weldments of the orthotropic steel deck (OSD) for coastal slender bridges could be critical and might affect structural safety and reliability. Due to the stochastic nature of the environmental loadings including vehicles, wind and waves, it is challenging to include uncertainties for the assessment of the fatigue damage accumulations in the bridge’s life cycle. In the present study, an efficient probabilistic fatigue damage assessment framework for coastal slender bridges is proposed with a machine learning algorithm to include the coupled stochastic dynamic loads in the VBWW system. Firstly, stochastic load models are developed based on the long-term field measurements for realistic modeling of the truck load and the correlated wind and wave load, which serve as the input for the VBWW system to extract the stress time histories at critical structural details using multi-scale finite-element analysis (FEA). After calculating the equivalent stress range and the corresponding number of cycles using the rain-flow counting method, the daily equivalent fatigue damage is obtained using the linear fatigue damage rule. To reduce the calculation cost, a machine learning algorithm is utilized for probabilistic modeling of the daily equivalent fatigue damage by integrating uniform design and support vector regression to link the multiple random inputs of environmental loadings with the single output of the stress time history. The fatigue life of critical structural details, therefore, can be obtained using the established limit-state function with a target reliability index. A prototype cable-stayed bridge in a coastal region is presented to demonstrate the effectiveness of the proposed simulation framework. Finally, the impacts of the traffic growth including the traffic volume and the gross vehicle weight on the fatigue life of three welded joints are investigated and discussed, as well.