Tower cranes are a class of important and useful tools for the transportation of large cargoes, especially in high building construction. Moreover, the combination of jibs' rotational movements and trolleys' translational movements can fulfill three-dimensional payload transportation in a huge outdoor workspace. However, complicated dynamics and different unfavorable environmental conditions in practice make the effective control of tower cranes very challenging. In this article, based on the original dynamic models without any linearization manipulations, we present a new output feedback control method to accomplish rapid jib and trolley positioning and payload sway suppression. In particular, the velocity signals are accurately obtained by an elaborately designed observer, instead of numerical differentiation manipulations to the measurable output variables (which may distort the original velocities to some extent). Based on the velocity estimates, the suggested controller can also handle uncertain friction. Additionally, the actuator constraints are fully considered during controller design, i.e., the calculated control inputs are both within the permitted ranges, thereby avoiding actuator saturation and degrading the control performance. For the entire closed-loop system, including the proposed controller, the state observer, and tower cranes, the asymptotic stability is strictly proven by Lyapunov-based techniques. Finally, some comparative and robustness verifications are implemented by hardware experiments.