This paper presents an image-based controller to perform the guidance of a free-floating robot manipulator. The manipulator has an eye-in-hand camera system, and is attached to a base satellite. The base is completely free and floating in space with no attitude control, and thus, freely reacting to the movements of the robot manipulator attached to it. The proposed image-based approach uses the system's kinematics and dynamics model, not only to achieve a desired location with respect to an observed object in space, but also to follow a desired trajectory with respect to the object. To do this, the paper presents an optimal control approach to guiding the free-floating satellite-mounted robot, using visual information and considering the optimization of the motor commands with respect to a specified metric along with chaos compensation. The proposed controller is applied to the visual control of a four-degree-of-freedom robot manipulator in different scenarios.