In bridge health monitoring systems, abnormal monitoring data inevitably appear due to the complex operational conditions, and these anomalous data will seriously affect the accuracy and reliability of the results of bridge health assessment. Hence, it is vital to detect the abnormal data before further processing. However, the pattern diversity and sample scarcity usually make the accuracy of data anomaly detection unsatisfactory. A novel anomaly detection method for bridge health monitoring data based on encoded images and convolutional neural network (CNN) is proposed in this paper. First, the raw time series are encoded into images to highlight the intrinsic features in the anomaly data, and multiple encoding techniques are adopted to represent the data in different perspectives. Then, the CNN is utilized to establish the mapping between the encoded images and the anomaly patterns for achieving the data anomaly detection. At last, the proposed method is verified with the acceleration data measured from an actual large span cable-stayed bridge. It shows that the proposed method can significantly improve the performance of data anomaly detection while the results are hardly affected by the small sample sizes, and a detection accuracy of over 95% is achieved based on the multiple encoded images.