Thyroid nodule is one of the most common endocrine diseases in the adult population. Early screening and diagnosis of thyroid nodules is of great significance to patients’ subsequent treatment. An objective and accurate diagnosis algorithm for thyroid nodules is vital to improve the efficiency of clinical diagnosis and reduce the work pressure of doctors. In this paper, we propose a fused deep learning model for the diagnosis of benign and malignant thyroid nodules. Based on the ACR TI-RADS proposed by the American Academy of Radiology, the model first extracts 33 clinically significant statistical features of composition, echogenicity, shape, margin, and echogenic foci. After PCA dimensionality reduction, the top 4 features are fused with the feature map of EfficientNet B3. The thyroid nodules are classified into benign and malignant based on the fused feature map. This model is trained and tested on 3828 clinically collected ultrasound images of thyroid nodules. It achieves an accuracy of 96.4% and an AUC of 0.965. Compared with traditional machine learning algorithms and state-of-the-art deep learning networks, the fused model we proposed has improved classification performance and amount of parameters, and is also clinically interpretable.