This work describes an adaptive trajectory tracking controller for underactuated underwater vehicles. The control design process is two-fold; first, a high-level kinematic controller is designed to produce velocity commands that steer the vehicle towards the desired trajectory, and second, a low-level controller is designed to utilize the mathematical model of the vehicle to produce force and torque commands for the thrusters onboard the underwater vehicle. A novel adaptive Nussbaum-function-based controller is proposed and compared with an adaptive proportional controller with integral feedback. Further, a conditional adaptation scheme is developed to combat effects of noise, disturbance, or uncertainty. The proposed scheme governs adaptation such that the gains only adapt when it is necessary, even in the presence of noise. The target vehicle for the controller is a four-thruster quadcopter-like vehicle, which does not have direct control over all degrees of freedom. The issue of having an underactuated thruster arrangement is common in many classes of inspection robots. The devised adaptive control law exhibits fast convergence for the adaptation. Four different parametrized trajectories are tested, and the performance of the proposed algorithms, when considering the tracking errors and required control effort, is shown to be superior to an adaptive proportional controller.