To solve the trajectory tracking problem of insufficient response and the large tracking error of remotely operated vehicles (ROVs) under the interference of large ocean currents, this paper proposes a double-loop sliding mode controller with an ocean current observer. The designed controller consisted of an outer-loop controller (the position controller) and an inner-loop controller (the velocity controller): the outer controller was designed by the position error, and a reference velocity was created for the inner loop to achieve accurate positioning and attitude tracking. The reference control input was treated as a new target to design the inner-loop controller, enabling the ROV to achieve accurate reference velocity tracking. Based on the theoretical idea of active disturbance rejection control, a kinematic equation-based ocean current observer was designed to estimate and compensate for large unknown currents to ensure accurate trajectory tracking performance under large currents. The simulation results proved that the double-loop sliding-mode control scheme with an ocean current observer always showed good tracking performance, demonstrating the excellent control performance and high robustness of the scheme. Compared with the high-complexity control schemes such as neural network-based PID control or fuzzy sliding mode control, it effectively improves the robustness to ocean current disturbances without increasing the computational effort excessively, and is more practical in ROV systems with limited computational power.