D-P-Transformer: A Distilling and Probsparse Self-Attention Rockburst Prediction Method

Abstract

Rockburst may cause damage to engineering equipment, disrupt construction progress, and endanger human life. To this day, the occurrence of rockburst remains complex and difficult to predict. This study proposes the D-P-Transformer algorithm to address this issue by improving the embedding structure of the Transformer for specific applications to rockburst data. To reduce the computational requirement, sparse self-attention is adopted to replace self-attention. A distilling operation and multiple layer replicas are simultaneously used to enhance the robustness and speed up the algorithm’s process. Taking all relevant rockburst factors into consideration, multiple experiments are conducted on seven large-scale rockburst datasets with different training ratios to verify the reliability of the proposed D-P-Transformer rockburst prediction algorithm. As compared to the original algorithm, the proposed algorithm shows average reductions of 24.45%, 46.56%, 17.32%, and 48.11% in the mean absolute error (MAE), mean square error (MSE), root mean square error (RMSE), and mean absolute percentage error (MAPE), respectively. The results indicate that the novel D-P-Transformer rockburst prediction algorithm is superior to the Transformer prediction algorithm, and could be used for coal mine rockburst prediction analysis.