ABSTRACT This paper mainly addresses the lack of labelled data and insufficient data utilization in PolSAR image classification. We propose a channel adaptive Complex-Valued Fully Convolutional Networks based on Transfer learning (TF-CVFCN). To solve the problem of less labelled data, we borrowed from the transfer learning method, which can use a large amount of easily obtained relevant or even irrelevant data for pre-training and then use a small amount of source data for network training fine-tuning. Since PolSAR data can obtain different data information by using different decomposition methods, the current research trend is to process PolSAR data in the complex-valued containing more information. In our work, to cater to the proposed network model and make the network design more liberal, a new transfer learning method is proposed, which uses a new method to realize partial layer parameter transfer, and achieves the purpose of optimizing the traditional Transfer learning method. On this basis, we optimize the network structure to improve the classification accuracy of PolSAR data, and evaluate its performance on three datasets. Furthermore, we lighten the TF-CVFCN and optimize the relevant parameters using Competitive Multi-Objective Particle Swarm Optimizer, emphasizing the importance of this parameter optimization in large networks.