We present machine learning models for the prediction of thermal and mechanical properties of polymers based on the graph convolutional network (GCN). GCN-based models provide reliable prediction performances for the glass transition temperature (T g), melting temperature (T m), density (ρ), and elastic modulus (E) with substantial dependence on the dataset, which is the best for T g (R 2 ∼ 0.9) and worst for E (R 2 ∼ 0.5). It is found that the GCN representations for polymers provide prediction performances of their properties comparable to the popular extended-connectivity circular fingerprint (ECFP) representation. Notably, the GCN combined with the neural network regression (GCN-NN) slightly outperforms the ECFP. It is investigated how the GCN captures important structural features of polymers to learn their properties. Using the dimensionality reduction, we demonstrate that the polymers are organized in the principal subspace of the GCN representation spaces with respect to the backbone rigidity. The organization in the representation space adaptively changes with the training and through the NN layers, which might facilitate a subsequent prediction of target properties based on the relationships between the structure and the property. The GCN models are found to provide an advantage to automatically extract a backbone rigidity, strongly correlated with T g, as well as a potential transferability to predict other properties associated with a backbone rigidity. Our results indicate both the capability and limitations of the GCN in learning to describe polymer systems depending on the property.