One of the safety-critical components of ballasted track systems is railway sleepers whose main functions are to (i) transfer vertical load, (ii) maintain rail gauge, and (iii) restrain longitudinal rail movement. Railway sleepers can be manufactured using timber, concrete, steel, composite, and any other engineered materials. Prestressed concrete sleepers are the most commonly used type worldwide because of their superior value-for-money performance. In practice, railway sleepers experience thousands of cycles of aggressive wheel–rail dynamic loads and wear deterioration can be observed over their service life. Not only does the deterioration affect track quality and geometries, but it also undermines the structural integrity of the track structures. The wear and abrasion directly decrease the capacity of railway sleepers, resulting in the reduction in service life. In this paper, the emphasis is placed on the assessment of the fatigue life of prestressed concrete railway sleepers with imperfect geometry. This study is the world’s first to establish a new fatigue simulation of railway concrete sleepers considering accumulative non-constant amplitudes, which has been validated using full-scale experimental results and empirical analyses. Parametric studies have been conducted to obtain new insights into the fatigue performance of the worn sleepers. The new findings will improve railway sleeper maintenance and inspection criteria, and will provide a new guideline on track-condition monitoring networks.