To improve safety in public transportation, a major issue is how to avoid traffic accidents. To this end, a recent report has demonstrated that more than 90% of accidents in the United States were due to drivers’ abnormal behaviors. Relevant to this observation, many recent studies have proposed to use different sensors to monitor drivers’ behaviors and apply learning algorithms to detect abnormal behaviors. Nevertheless, most existing systems are expensive and inconvenient to be deployed or significantly affected by the environment. In this article, we propose and develop a novel and effective solution, namely, SafeDriving, that collects signals from electromyography (EMG) sensors and then utilizes an effective deep-learning model to detect abnormal behaviors in real time. Specifically, we first utilize a wearable EMG sensor that can be attached to a driver’s forearm to collect a large amount of sensing data from human drivers, for which we define five typical abnormal driving behaviors (i.e., fetching forward, picking up, turning the steering wheel sharply, turning back, and touching sunroof) and label each sample accordingly. Next, using the labeled data, we design and train multiple state-of-the-art classifiers to improve the performance of SafeDriving, e.g., convolutional neural network (CNN), long short-term memory (LSTM), and gated recurrent unit (GRU). The extensive experiments demonstrate that GRU can lead to the best performance with an average accuracy of 93.94%. Based on this observation, we further investigate other important factors, such as the binding area of the sensor, the tightness of binding, the duration of the sample, etc. The proposed SafeDriving system provides an effective approach to reliably assess drivers’ driving behaviors with affordable commodity sensors and be further used in public safety.