This paper addresses the adaptive task-space bilateral teleoperation for heterogeneous master and slave robots to guarantee stability and tracking performance, where a novel semi-autonomous teleoperation framework is developed to ensure the safety and enhance the efficiency of the robot in remote site. The basic idea is to stabilize the tracking error in task space while enhancing the efficiency of complex teleoperation by using redundant slave robot with subtask control. To unify the study of the asymmetric time-varying delays, passive/nonpassive exogenous forces, dynamic parameter uncertainties and dead-zone input in the same framework, a novel switching technique-based adaptive control scheme is investigated, where a special switched error filter is developed. By replacing the derivatives of position errors with their filtered outputs in the coordinate torque design, and employing the multiple Lyapunov-Krasovskii functionals method, the complete closed-loop master (slave) system is proven to be state-independent input-to-output stable. It is shown that both the position tracking errors in task space and the adaptive parameter estimation errors remain bounded for any bounded exogenous forces. Moreover, by using the redundancy of the slave robot, the proposed teleoperation framework can autonomously achieve additional subtasks in the remote environment. Finally, the obtained results are demonstrated by the simulation.