This paper is concerned with the tracking control problem of a voice coil motor (VCM) actuated servo gantry system. By utilizing an adaptive control technique combined with a sliding mode approach, an adaptive sliding mode control (ASMC) law with friction compensation scheme is proposed in presence of both frictions and external disturbances. Based on the LuGre dynamic friction model, a dual-observer structure is used to estimate the unmeasurable friction state, and an adaptive control law is synthesized to effectively handle the unknown friction model parameters as well as the bound of the disturbances. Moreover, the proposed control law is also implemented on a VCM servo gantry system for motion tracking. Simulations and experimental results demonstrate good tracking performance, which outperform traditional control approaches.