This paper investigates the task-space control framework for bilateral teleoperation with communication time delays. Teleoperation in task space R3 × SO(3) presents some distinctive features different from its joint-space counterpart, i.e., SO(3) is nonconvex and bears quite different structure from Euclidean space Rn. Through analyzing the energy flows at the two ports of the teleoperator, we rigorously define the task-space interaction passivity of the teleoperator. Based on this passivity framework, we propose delay-robust control schemes to achieve master–slave position/orientation synchronization. Singularity-free task-space interaction passivity of the closed-loop teleoperator is ensured by the proposed task-space control framework. Using Lyapunov–Krasovskii stability tool and Schwarz inequality, we analyze the performance of the proposed teleoperation control scheme. We also discuss the problems incurred by time-varying delays and the corresponding solutions. Simulation study on a master–slave teleoperator composed of two kinematically dissimilar six-degree of freedom (DOF) manipulators is performed to illustrate the performance of the proposed control approach.