Introducing autonomous vehicles (AVs) on the market is likely to bring changes in the mobility of travelers. In this work, extensive research is conducted to study the impact of different levels of automation on the mobility of people, and full driving automation needs further study because it is still under development. The impacts of AVs on travel behavior can be studied by integrating AVs into activity-based models. The contribution of this study is the estimation of AVs’ impacts on travelers’ mobility when different travel demands are provided, and also the estimation of AVs’ impact on the modal share considering the different willingness of pay to travel by AVs. This study analyses the potential impacts of AVs on travel behavior by investigating a sample of 8500 travelers who recorded their daily activity plans in Budapest, Hungary. Three scenarios are derived to study travel behavior and to find the impacts of the AVs on the conventional transport modes. The scenarios include (1) a simulation of the existing condition, (2) a simulation of AVs as a full replacement for conventional transport modes, and (3) a simulation of the AVs with conventional transport modes concerning different marginal utilities of travel time in AVs. The simulations are done by using the Multi-Agent Transport Simulation (MATSim) open-source software, which applies a co-evolutionary optimization algorithm. Using the scenarios in the study, we develop a base model, determine the required fleet size of AVs needed to fulfill the demand of the different groups of travelers, and predict the new modal shares of the transport modes when AVs appear on the market. The results demonstrate that the travelers are exposed to a reduction in travel time once conventional transport modes are replaced by AVs. The impact of the value of travel time (VOT) on the usage of AVs and the modal share is demonstrated. The decrease in the VOT of AVs increases the usage of AVs, and it particularly decreases the usage of cars even more than other transport modes. AVs strongly affect the public transport when the VOT of AVs gets close to the VOT of public transport. Finally, the result shows that 1 AV can replace 7.85 conventional vehicles with acceptable waiting time.