With the development of network technology, WLAN-based indoor localization plays an increasingly important role. Most current localization methods are based on the comparison between the received signal strength indication (RSSI) and the RSS in the database, whose nearest reference point is the location point. However, since a uniform standard for measuring components of smartphones has not yet been established, the Wi-Fi chipsets on different smartphones may have different sensitivity levels to different Wi-Fi access points (APs) and channels. Even for the same signal, RSSI values obtained by different terminals at the same time and the same location may be different. Therefore, the impact of terminal heterogeneity on localization accuracy can be overlooked. To address this issue, a fusion method based on received signal strength difference and compressive sensing (RSSD-CS) is proposed in this paper, which can reduce the influence caused by the terminal heterogeneity. Besides, a fingerprint database is reconstructed from the existing reference point data. Experiments show that the proposed RSSD-CS algorithm can achieve high localization accuracy in indoor localization, and the accuracy is enhanced by 20.5% and 15.6% compared to SSD and CS algorithm.