In this article, a robust control scheme for trajectory tracking of very lightweight single-link flexible arms is discussed. Since the payload is one of the most variable parameters in a manipulator, the control is designed to achieve an accurate tracking of the desired tip trajectory for any value of the robot tip mass, or even for a tip mass changing during the maneuver. The proposed controller also guarantees stability for small uncertainties in parameters such as stiffness or motor friction. In addition, the effect of spillover on the performance of the controlled system is analyzed, and it is proven that stability and a good performance are preserved independently from the non-modeled high-order dynamics. The control scheme is based on a two nested loops structure. Each of these loops implements a Generalized Proportional Integral (GPI) controller. Moreover, the outer loop includes a disturbance compensation term based on a disturbance observer, which achieves the required insensitivity to payload changes. The theoretical analysis is supported by an extensive set of numerical simulations which shows controlled system response when variations in the robot payload, or dynamics neglected in the controller design, are considered. Finally, some experiments have been carried out in order to test the performance of the tip trajectory tracking of the proposed control system.