Visible Light Positioning (VLP) techniques have attracted considerable attention owing to their cost-effectiveness, high precision, and ubiquitous infrastructure. However, conventional VLP demands supplementary hardware for Light Emitting Diode (LED) modulation, which entails retrofitting existing indoor light systems for positioning applications. Thus, it is costly for wide-scale adoption. Moreover, the built-in Photodiodes (PDs) in smartphones can only provide a limited sampling rate (around 10 Hz), making it challenging to identify LEDs and extract each LED’s light intensity by Frequency Division Multiplexing and Fast Fourier Transform. To alleviate these problems, we develop a novel inertial-aided Unmodulated Visible Light Positioning system (uLiDR) for resource-constrained platforms like smartphones. Firstly, we evaluate the feasibility of unmodulated LED for precise ranging using smartphones. Next, we introduce a selective fusion strategy to mitigate LED occlusion in pedestrian navigation. As slight tilts of the phone are unavoidable during walking, we leverage attitude information calculated by the built-in sensors to handle the phone’s tilt in VLP. Then, we implement the tightly-coupled integration of uVLP and Pedestrian Dead Reckoning (PDR) in an optimization-based framework. The initial location is vital for PDR, and thereby, we proposed an Angle of Arrival (AoA)-based method to provide a precise initial location estimation for PDR. Experimental results show that the proposed uLiDR can achieve precise pedestrian navigation and effectively suppress the error accumulation of PDR.