For mobile users, the ability to accurately acquire their own location is critical. By locating, mobile users can get information about their environment and access location-related services. RSS fingerprint-based indoor localization method collects a training database of measurement fingerprints and uses a machine learning classifier to determine a person’s location from a new fingerprint. However, as the environment changes over time due to furniture or other objects being moved, the new fingerprints diverge from those in the original database. Therefore, an RSS-difference based localization system is designed to deal with the above problem. This method combines back-propagation neural network (BPNN) and weighted K-Nearest Neighbor (WKNN) method to improve the fingerprint similarity based indoor location method (FSIL). We train BPNN in the off-line stage to obtain the optimal BPNN parameter settings. In the online stage, K nearest neighbor points are firstly selected based on the improved FSIL algorithm, and then the difference of signal strength values between the K nearest neighbor points and the target user is input into the BPNN network, to obtain the Euclidean distance between the K nearest neighbor points and the target user, and finally the WKNN algorithm is used to obtain the user’s ultimate location. Simulation experiments based on the LDPL model and the Wireless Insite software, as well as the test results based on the indoor localization dataset IPIN2016, show that the localization accuracy in complex indoor scenarios can be improved by at least 11% when using the method proposed in this paper.