Autonomous vehicles offer significant advantages for transportation systems, particularly in enhancing traffic safety. To achieve this goal, it is crucial to comprehensively understand and predict the future trajectories of pedestrians in proximity to autonomous vehicles. Many contemporary approaches for predicting pedestrian trajectories heavily rely on neural networks, especially recurrent neural networks. However, these approaches do not explicitly incorporate the dynamics of pedestrian movement and instead rely on data-driven black-box models. Consequently, these models may fall short in terms of interpretability and fail to adhere to the fundamental principles of kinematics. In response to these limitations, our work introduces an innovative model for pedestrian trajectory prediction grounded in neural differential constraints. We aim to investigate temporal changes in pedestrian state variables, such as position and speed, using neural networks. During the prediction process, the output of the neural network is governed by differential equations. This approach ensures that the generated trajectories align with the fundamental principles of physics, harnessing the combined power of neural networks and physics-based pedestrian motion models. Furthermore, our research endeavors to develop a cohesive framework that seamlessly integrates pedestrian movement patterns with the influence of ego-vehicles, while also considering potential destinations to inform future trajectory planning. We conducted extensive experiments on two publicly available real-world datasets to assess the effectiveness of our model in enhancing prediction accuracy and providing coherent explanations of pedestrian motion, comparing it to state-of-the-art methods.