Orthotropic steel deck (OSD) in a long-span cable-stayed bridge is vulnerable to fatigue damage at the deck-to-rib (DTR) joints due to excessive vehicle-induced dynamic stresses. A framework for fatigue damage prognosis of OSD in long-span cable-stayed bridges is thus presented in this paper. The main features of the framework include hourly traffic loading simulation and prediction, multi-scale finite element bridge model, coupled vehicle-bridge system, mesh-insensitive equivalent stress responses at a DTR joint, response surface models for equivalent stress responses, OSD and pavement interaction, asphalt pavement temperature effect, fatigue test-generated S-N curve, and fatigue damage prognosis. To evaluate the feasibility of the proposed framework and to manifest the effects of pavement roughness, asphalt temperature, and vehicle transverse locations on fatigue damage accumulation, a real long span cable-stayed bridge is investigated as a case study. The results indicate that the developed framework is applicable and that the fatigue damage of a DTR joint is underestimated if the time-variant temperature of asphalt pavement and road surface condition are not considered but the fatigue damage is overestimated without consideration of variable transverse locations of vehicles.