Borehole hydraulic fracturing in coal seams can prevent dynamic coal mine disasters and promote the sustainability of the mining industry, and microseismic signal recognition is a prerequisite and foundation for microseismic monitoring technology that evaluates the effectiveness of hydraulic fracturing. This study constructed ultra-lightweight CNN models specifically designed to identify microseismic waveforms induced by borehole hydraulic fracturing in coal seams, namely Ul-Inception28, Ul-ResNet12, Ul-MobileNet17, and Ul-TripleConv8. The three best-performing models were selected to create both a probability averaging ensemble CNN model and a voting ensemble CNN model. Additionally, an automatic threshold adjustment strategy for CNN identification was introduced. The relationships between feature map entropy, training data volume, and model performance were also analyzed. The results indicated that our in-house models surpassed the performance of the InceptionV3, ResNet50, and MobileNetV3 models from the TensorFlow Keras library. Notably, the voting ensemble CNN model achieved an improvement of at least 0.0452 in the F1 score compared to individual models. The automatic threshold adjustment strategy enhanced the identification threshold’s precision to 26 decimal places. However, a continuous zero-entropy value in the feature maps of various channels was found to detract from the model’s generalization performance. Moreover, the expanded training dataset, derived from thousands of waveforms, proved more compatible with CNN models comprising hundreds of thousands of parameters. The findings of this research significantly contribute to the prevention of dynamic coal mine disasters, potentially reducing casualties, economic losses, and promoting the sustainable progress of the coal mining industry.