Due to factors such as dense populations and narrow viewing angles, previous deep-learning models for detecting passenger behavior in elevators often lack effectiveness. Traditional cloud-based data transmission methods have issues with high latency, high resource usage, and privacy threats, particularly during periods of high usage. To address these issues, we proposed a falling behavior detection system for elevator passengers based on deep learning and edge computing. A two-stream neural network model improved by 3D ResNet is presented, which utilizes edge computing for elevator passenger fall detection. Our homemade dataset of elevator passenger falling behavior is utilized to train and evaluate the system. The results demonstrate that the system is effective in detecting passengers’ falling behavior in elevators, with an average accuracy of 89.2%. The feasibility of the system in an elevator is also verified, and it has performed well. The application of this system in this field holds significant research value.