Positioning accuracy, robustness and real-time ability are the three critical elements in visible light positioning (VLP) system. While only a few exiting studies take these three critical elements into consideration at the same time to judge a practical VLP system, causing positioning and tracking method cannot be applied into real situation with different interferences. Therefore, to further improve the three aspects in the round and make it more feasible, a novel VLP method based on image sensor using particle filter tracking algorithm is proposed in this paper. The proposed positioning method locates the position of the terminal equipment with the proportional relationship of world coordinate, image coordinate and camera coordinate. The particle filter is utilized for the fast detection of LED in the image to improve the robustness of VLP system, which is not used for positioning What is more, particle filter algorithm uses HSV (hue, saturation, value) color histogram of signal source as features to realize tracking through importance sampling, weight updating, state estimation and resampling. All three components of HSV are considered and various processing steps of particle filter enhance tracking accuracy and robustness simultaneously. Experimental results show that the particle filter can provide an accuracy of 2.95 cm, which are acceptable in practical application scenarios, demonstrating that the LED detection is so accurate that the positioning accuracy of the positioning algorithm is not affected by the tracking algorithm. At the same time, the proposed algorithm maintains good stability when interference occurs and average computational time for each frame is only 0.021 s, which denotes that the computational cost required for particle filter is so small that the positioning algorithm is not affected by the tracking algorithm and can still have good real-time performance. All these data and performance confirm that the proposed tracking detection algorithm possess practical value and can handle background interference and shielding effect in traditional VLP system based on image sensor.