This study proposes and investigates an innovative two-dimensional Convolutional Neural Network (2D CNN) architecture for bridge damage classification. Employing strain time-history data transformed into grayscale images, the approach seamlessly combines feature extraction and classification, allowing for the precise identification and categorization of structural damage. The method’s effectiveness was validated through field experiments on a bridge mock-up subjected to several controlled damage states under nonstationary, commercial vehicle loads. The experiments included a range of scenarios, from minor impacts to significant structural damages, and the CNN model’s robustness was tested against fluctuating loads and introduced noise. Demonstrating remarkable accuracy, the 2D CNN successfully classified different damage states with over 95% accuracy, effectively identifying a damage state that was visually undetectable. Additionally, the proposed architecture was shown to be versatile when using varying number of sensors, wide range of frequency of measured sensor data, and elevated levels of measurement noise.