Intelligent vehicles attempt to improve the daily transportation of people, but their safety has been questioned after numerous trafic accidents. This paper proposes a Safety Driving Framework (SDF) to improve the ability of intelligent vehicles to avoid risks during emergencies. When performing autonomous driving tasks, the SDF interacts with the environment using a camera, makes decisions using convolutional neural networks, and can perform an emergency stop if it encounters an obstacle. In this paper, we examine the potential of attention mechanisms to enhance the performance of convolutional neural networks and construct four convolutional neural networks with attention mechanisms to use in experiments. Additionally, we extend a dataset and enhance the robustness of the model by implementing data augmentation (DA) techniques. We train the model using 10-fold cross-validation. In this article, we build an intelligent driving platform and a simulation track for simulation testing. The experimental results show that CNNs using data reinforcement and with an attention mechanism perform better than existing models. In particular, ENetb0-SE has an average recognition rate of 95.6% for obstacles and an accident rate of 2%, which is much better than existing models.