This paper presents a design of an adaptive PID gain tuning based on deep deterministic policy gradient reinforcement learning agent for PID computed-torque control of robot manipulators, taking the presence of unmodelled dynamics and external disturbances into consideration. The proposed approach adaptively computes the outer-loop PID controller gains, that minimise trajectory tracking errors and reject disturbances, while the closed-loop dynamics remain stable. Since the control scheme requires the knowledge of the robot's dynamics, both kinematic and dynamic equations of n-link serial manipulator are developed. The agent is implemented on UR5e robot manipulator model, using the most valid dynamic and kinematic parameters provided by the manufacturer and related works. Simulation results show that the proposed approach is robust against bounded internal and external disturbances, and achieves a good trajectory tracking performance, due to the adaptability of gain tuning over the conventional PID controller.