The security of autonomous vehicles heavily depends on localization systems that integrate multiple sensors, which are vulnerable to sensor attacks and increase the risk of accidents. Given the diversity of sensor attacks and the dynamic changing of driving scenarios of autonomous vehicles, an adaptive and effective attack detection and defense framework faces a considerable challenge. This paper proposes a novel real-time adaptive attack detection and defense framework based on density, which can detect and identify attacked sensors and effectively recover data. We first develop a reinforcement learning multi-armed Bandit-based Density-Based Spatial Clustering of Applications with Noise (BDBSCAN) algorithm that selects hyperparameters adaptively. The Adaptive Extended Kalman Filter (AEKF) combines with the vehicle dynamic model on the localization system and extracts data features used for the BDBSCAN algorithm to monitor potential sensor attacks. If attack detection indicates possible system compromise, AEKF is further employed on localization sensors with anomalies identified through the BDBSCAN algorithm of the attacked sensors. To ensure precision and reliability, the data recovery incorporates a redundancy mechanism to apply a decision tree to select the optimal state estimation between AEKF and Extended Kalman Filter (EKF) to replace corrupted sensor data. To evaluate the effectiveness and adaptability of the proposed framework, we conducted 15,000 experiments using the real-world KITTI and V2V4Real datasets across various driving and sensor attack scenarios. The results demonstrate that our proposed framework achieves 100% accuracy and 0% false alarm rate in various driving scenarios for attack detection within 0.15 s, with a recovery time of 0.08 s.