Deep learning based bridge damage diagnosis methods can successfully use labeled data to detect bridge damage. These successful applications usually need that the training samples (source domain) and test samples (target domain) obey the same probability distribution. However, it is difficult to acquire a large amount of labeled data with damage information from actual bridges. It is also difficult to apply a model trained with bridge A to diagnose bridge B because of the distribution discrepancy of data from different bridges or environments. Therefore, transferring a well-trained damage diagnosis model to another bridge with unlabeled data remains a major challenge. Motivated by transfer learning, this paper proposes a new intelligent damage diagnosis method for bridges, namely, sub-domain adaptive deep transfer learning network (SADTLN), to solve the feature generalization problem in different bridges. In our method, a multi-kernel local maximum mean discrepancy (MK-LMMD) based sub-domain adaptation module, including a domain classifier for aligning the global distribution and a sub-domain multi-layer adaptation for aligning local distribution, is proposed for transfer learning, so that the learned features are domain-invariant. Experiments prove the effectiveness and advancement of the proposed method. This exploration will promote the practical application of intelligent bridge structural damage diagnosis.