In autonomous mining scenarios, excavation trajectory planning plays a significant role since it considerably influences the working performance of the unmanned mining excavator (UME). Aiming at the limited dynamical characterization of traditional theoretical methods that yield unsatisfactory performance in trajectory planning, herein we propose a data-driven excavation trajectory planning framework based on deep learning for improving the operation performance in autonomous excavation scenarios. First, the actual sensing data is collected, and a temporal convolutional recurrent neural network (TCRNN) combining stacked dilated convolutions with an attention-based sequence to sequence (Seq2Seq) module is proposed to predict the digging force accurately. Then, the surrounding material profile is perceived based on Lidar, and a polynomial response surface is performed to point cloud reconstruction. Considering mining efficiency, energy consumption and stability in the excavation process, a multi-objective function is constructed, and a data-driven excavation trajectory planning framework based on TCRNN is established. To obtain the accurate solution of the planning framework, a discretization strategy based on Radau pseudospectral method is developed to model solving to ensure the working performance in autonomous mining. An actual unmanned prototype experiment is performed to investigate the performance of the proposed framework. The results demonstrate the effectiveness and superiority of the proposed framework.