Feature pyramid self-attention network for respiratory motion prediction in ultrasound image guided surgery.

Abstract

The robot-assisted automated puncture system under ultrasound guidance can well improve the puncture accuracy in ablation surgery. The automated puncture system requires advanced definition of the puncture location, while the displacement of thoracic-abdominal tumors caused by respiratory motion makes it difficult for the system to locate the best puncture position. Predicting tumor motion is an effective way to help the automated puncture system output a more accurate puncture position. In this paper, we propose a self-attention-based feature pyramid algorithm FPSANet for time-series forecasting, which can extract both linear and nonlinear dependencies of time series. Firstly, we use the temporal convolutional network as the backbone to extract different scale time-series features, and the self-attention module is followed to weigh more significant features to improve nonlinear prediction. Secondly, we use autoregressive models to perform linear prediction. Finally, we directly combine the above two kinds of predictions as the final prediction. FPSANet is trained and tested on our private datasets captured from clinical individuals, and we predict the target position after 50ms, 150ms, 300ms and 400ms. The result shows the evaluation criteria of the MAE is less than 1mm at 50ms and 150ms, and less than 2mm at 300ms. Compared with the AR model, bidirectional LSTM and RVM, our method not only outperforms both models in accuracy (AR: ~ 7.7%; bidirectional LSTM: ~ 75.9%; RVM: ~ 76.5%) but is also more stable on different types of respiratory curves. Respiratory motion in the liver in actual clinical practice vary widely from person to person, while sometimes having less distinct periodic patterns. Under these conditions, our algorithm has the advantage of excellent stability for prediction on various sequences, and its running time of performing single sequence prediction can meet clinical requirements.