According to the International Union of Railways, railway networks count more than one million kilometers of tracks worldwide, a number that is to rise further as the goal is to promote rail transportation as a sustainable means to face the challenge of increased mobility. However, such a vast expansion further necessitates efficient and reliable infrastructure monitoring schemes able to guarantee the quality and safety of rail transportation. Traditional monitoring approaches, relying on visual inspection and portable measuring devices, cannot rise to the task as they do not allow for continuous inspection of extended portions of rail infrastructure. Therefore, mobile monitoring methodologies based on dedicated diagnostic vehicles have emerged as an alternative. Despite revolutionizing traditional monitoring methods, such vehicles are usually expensive and can only operate under the suspension of regular rail service. In this work, we propose an alternative approach for mobile sensing of railway infrastructure based on on-board monitoring data collected from low-cost vibration sensors, e.g., accelerometers, which can be mounted on in-service trains. Specifically, we focus on identifying the roughness profile of the tracks and propose a fusion of reduced-order vehicle models with a Bayesian inference approach for joint input-state estimation. To enhance the inference, we opt for a prior updating of the vehicle model parameters on the basis of an unscented Kalman filter and available measurements from a diagnostic vehicle. The key contributions of this work are (i) the consideration of the dynamic interaction between trains and tracks, which is usually ignored in rail roughness estimation, (ii) the adoption of reduced train vehicle models that decrease the computational effort of the identification task, (iii) the updating of the vehicle parameters to account for inconsistencies in the model used, and (iv) the application of the proposed methodology to actual acceleration measurements collected from a diagnostic vehicle of the Swiss Federal Railways network.