Electro-hydraulic servo system is often affected by parameter uncertainties and mismatched random disturbances, which are two major obstacles to achieving high-performance servo control. In this article, a sliding mode adaptive controller is proposed to solve these two problems. The proposed controller employs sliding mode control method to eliminate unknown mismatched disturbances, adaptive design to deal with parametric uncertainties. A new smooth and continuous sliding mode control law is proposed to solve the design conflicts between sliding mode control technology and back-stepping adaptive control technology. The stability of the algorithm is proved based on the Lyapunov theory. Theoretically, the algorithm can make the position tracking error arbitrarily small. Through the co-simulation platform of AMESim and MATLAB/Simulink, the control performance of the developed control algorithm is verified. The results show that the developed sliding mode adaptive controller scheme can achieve high-performance tracking under the condition of parametric uncertainties and unknown mismatched disturbances.