Noting the fact that the high-dimensional data composed of various polarimetric features has better decouplability than polarimetric synthetic aperture radar (PolSAR) image source data, in this letter, multiple polarimetric features are extracted and stacked to form a high-dimensional feature cube as the input of convolutional neural networks (CNNs) to improve the performance of PolSAR image classification. Directly utilizing the polarimetric features will produce a performance degradation and the recalibration of them is indispensable. However, classical feature selection methods are independent of the classifier, which means that the stimulated features may not be the classification-friendly ones. To avoid separated procedures and improve the performance, attention-based polarimetric feature selection convolutional network, called AFS-CNN, is proposed to implement end-to-end feature selection and classification. The relationship between input polarimetric features can be captured and embedded through attention-based architecture to ensure the validity of high-dimensional data classification. Experiments on two PolSAR benchmark data sets verify the performance of the proposed method. Furthermore, this work is quite flexible, which is reflected in that the proposal can be used as a plug-and-play component of any CNN-based PolSAR classifier.