Subjected to complex loadings from the wheel–rail interaction, turnout rail is prone to crack damage. This paper aims to develop a condition evaluation method for crack-alike damage detection of in-service turnout rail. A covariance-based structural condition index (CI) is firstly constructed by fusing the time-frequency components of responses, generating a series of patterns governed by the interrelationships between column members in the CI matrix. The damage-sensitive interrelationships latent in CI are then modeled using Bayesian regression and historical data, and baseline patterns are built with predictions of the models and new inputs. The deviations between the baseline patterns and the actual patterns of the newly observed CI members are quantitatively assessed. To synthetically consider the individual assessment results, a technique is developed to combine the individual assessment results into one synthetic result by designing a group of suitable weights taking into consideration both probabilistic confidence and reference model error. If the deviations are within a tolerable range, no damage is flagged; otherwise, damage existence and severity are reported. A case study is conducted, in which monitoring data from the database of a railway turnout are applied to build the CI matrix and examine the damage identification performance of this method. Good agreement between actual conditions and assessment results is found in different testing scenarios in the case study, demonstrating the effectiveness of the proposed method.