SUMMARY Tuned mass dampers with viscous damping and isolation systems with friction pendulums have been proposed and widely applied over the past several decades. By combining these two ideas, a friction pendulum tuned mass damper (FPTMD) is proposed in this study. Because the restoring and friction forces are provided by the spherical surface of the FPTMD, springs and dampers are not needed. Moreover, suspension is not necessary and the installation space is greatly reduced. The optimal design of the FPTMD with varying friction coefficients for wind-excited high-rise structures is investigated. The optimization procedures are demonstrated by the FPTMD implemented on Taipei 101 under white-noise wind force, and the optimization results are validated by three-dimensional graphs. From the results of the sensitivity study, the effectiveness of the FPTMD with two different patterns of friction coefficients is sensitive to the tuning frequency ratio but not very sensitive to the friction parameters. Moreover, an FPTMD with a friction coefficient that linearly varies with displacement is even less sensitive to the friction parameters and the amplitude of excitation. The feasibility of the FPTMD with two different patterns of friction coefficient is illustrated by Taipei 101 subjected to the design wind force with a return period of 50 years. Following design optimization and numerical verification, the effect of vibration reduction for Taipei 101 is demonstrated. Copyright © 2012 John Wiley & Sons, Ltd.