Integrated Recognition Assistant Framework Based on Deep Learning for Autonomous Driving: Human-Like Restoring Damaged Road Sign Information

Abstract

Unpredictable situations frequently occur in real driving environments, and it is often difficult to recognize road signs. In this case, autonomous vehicles (AVs) have a limited ability to predict areas that cannot be detected, making it difficult to judge objects accurately when some information is lost. Therefore, we propose a framework that helps AVs infer proper information under limited conditions. The entire process consists of three steps. First, the missing part of the road sign is restored using the image generative pre-trained transformer model. Next, the sample image with the highest classification accuracy and restored quality is selected among several sample images. Finally, the selected image is provided to users through the designed user interface. The proposed framework improved recognition accuracy compared with unrestored accuracy, indicating the possibility of application as a driving assistance system, and is meaningful in that it is a system that mimics human reasoning ability.