Automated transportation systems will become the trend in the future, while the design challenges in improving accessibility and usability for visually impaired passengers will have to be resolved. This paper aims to use human factors methods in designing a concept inclusive user interface for the Autonomous Driving System for Dedicated Vehicles (ADS-DV). The concept prototype design evaluated in this study, ‘Smart Autonomous Vehicle for Visually Impaired’ (SAVVI), is designed to accommodate the accessibility needs of visually challenged passengers. We broke down the autonomous transit service for the people with a visual impairment from six essential tasks, including booking a ride, navigating to the vehicle, entering and sitting assistance, initiating the ride, the car traveling to the destination, and exiting the vehicle. First, a hierarchical task analysis was conducted, and a user interface prototype was designed and tested by visually challenged participants. The responses collected from the testing and surveys were divided into qualitative and quantitative data for better comprehension and implementation.