The air turbine starter (ATS) is an equipment employed for initiating aircraft engines, in which rolling bearings play a central role. Predicting the remaining useful life (RUL) of rolling bearings constitutes a challenging task within the domain of prognostics and health management (PHM). Improving computational efficiency to predict the RUL of bearings is a core problem addressed in this paper, and thus, this paper introduces the parallel causal convolutional hierarchical dilated temporal convolutional network (HD-TCN). HD-TCN effectively captures long-term temporal dependencies and significantly enhances computational efficiency compared to serial convolutional recurrent neural networks (RNNs). In addition, the introduction of the feature data stacking (FDS) module allows features from different hierarchical levels and dimensions to participate in the prediction task, ensuring both computational efficiency and prediction accuracy. To consolidate the extracted multidimensional features, the network incorporates the matrix multiplication dimensionality reduction transformation (MMDRT) module, which reduces the dimensionality of the data and further improves the computational efficiency as well. The introduction of the MMDRT module resulted in reductions of 43.4% in root mean square error (RMSE). The proposed method achieved reductions in RMSE by 38.3% and 46.4% compared to the transformer and long short-term memory (LSTM) models, respectively. Finally, the effectiveness of the proposed method is validated using the XJTU-SY data set and civil aircraft bearing components RUL prediction test bench bearing data. The results show that the proposed approach can predict the RUL of bearings with high accuracy.