ABSTRACT In autonomous driving, accurately predicting the trajectories of surrounding vehicles is essential, particularly in dense and heterogeneous urban traffic. We propose a graph-structured model with a category layer to efficiently forecast the target vehicle’s trajectory. The model enables flexible selection of interacting objects based on environmental interactions and extracts spatial-temporal features using a graph convolutional network. A categorical layer is introduced to account for the different influences of dynamic agents, while vehicle dynamics constraints ensure the feasibility of predicted trajectories. We developed a new heterogeneous and dense urban unsignalized intersection dataset (HID), capturing complex urban interactions, and conducted extensive experiments on HID, ApolloScape, and TRAF datasets. Results demonstrate that our model outperforms benchmark methods across diverse urban scenarios, and the integration of key modules significantly enhances prediction accuracy and performance.