White blood cell (WBC) is an essential part of the human immune system. To diagnose blood diseases, hematologists have to think about the WBC information. For instance, the number of each type of WBCs often implies the health condition of the human body. Thus, the classification of white blood cell images plays a significant role in the medical diagnosis process. However, manual WBC inspection is time-consuming and labor-intensive for experts, which means automated classification methods are needed for WBC recognition. Another problem is that the traditional automatic recognition system needs a large amount of annotated medical images for training, which is highly costly. In this respect, the semi-supervised learning framework has recently been widely used for medical diagnosis due to its specificity, which can explore relevant information from massive unlabeled data. In this study, a novel semi-supervised white blood cell classification method is proposed, named by Fine-grained Interactive Attention Learning (FIAL). It consists of a Semi-Supervised Teacher-Student (SSTS) module and a Fine-Grained Interactive Attention (FGIA) mechanism. In detail, SSTS employs limited labeled WBC images and generates predicted probability vectors for a large amount of unlabeled WBC samples, like a human. After top-k selection in predicted probabilities, the efficient data can be exploited from unlabeled WBC images for training. With a very small amount of annotated WBC images, FIAL achieves an average accuracy of 93.2% on BCCD dataset when giving 75 labeled images for each category, which sufficiently elaborates our excellent capability on semi-supervised white blood cell image classification task.