Industrial robots have been widely used in the industries of automotive, machining, electrical and electronic, rubber and plastics, aerospace, food, etc., owing to their high efficiency and flexibility in contrast to large scaled machining centers. However, the poor accuracy resulted from the serial configuration of industrial robots has restricted their applications to high-precision machining for several decades. In this paper, an error compensation technique is proposed using the visual guidance to effectively improve the pose accuracy of industrial robots. Firstly, the effect of the establishment method of tracking coordinate system for a visual sensor on the pose measurement error is analyzed and the establishment method is optimized. Next, a fuzzy PID controller is designed and integrated with a KUKA robot controller KRC via the KUKA robot sensor interface to guide the robot to the desired pose in real-time. Finally, experimental tests are implemented to validate the effectiveness of the proposed approach.