Human physical activity identification based on wearable sensors is of great significance to human health analysis. A large number of machine learning models have been applied to human physical activity identification and achieved remarkable results. However, most human physical activity identification models can only be trained based on labeled data, and it is difficult to obtain enough labeled data, which leads to weak generalization ability of the model. A Pruning Growing SOM model is proposed in this paper to address the limitations of small-scale labeled dataset, which is unsupervised in the training stage, and then only a small amount of labeled data is used for labeling neurons to reduce dependency on labeled data. In training stage, the inactive neurons in network can be deleted by pruning mechanism, which makes the model more consistent with the data distribution and improves the identification accuracy even on unbalanced dataset, especially for the action categories with poor identification effect. In addition, the pruning mechanism can also speed up the inference of the model by controlling its scale.