To improve the classification performance of the convolutional neural network (CNN) for polarimetric synthetic aperture radar (PolSAR) images with limited labeled samples, this letter proposes a memory CNN (MCNN) for semisupervised PolSAR image classification using both the labeled and unlabeled samples. Specifically, the MCNN introduces a memory module to realize an assimilation–accommodation interaction between the network and the module in the model training process. Compared with the traditional CNN-based methods, the advantage of the introduced interaction mechanism can exploit the memory information during the model training including both the learned feature representation and the model inference uncertainty. Under the framework of memory mechanism, the semisupervised learning can be implemented simply and effectively by introducing an unsupervised memory loss. We evaluate the proposed method on three benchmark PolSAR data sets. The experimental results show the advantages of the MCNN over the supervised, semisupervised, and unsupervised methods in the PolSAR image classification with limited labeled samples.