The recent and continuously growing scientific interest for asteroids composition and its relation to solar system’s origins led the space industry to put high effort into asteroid missions during the last years. Nowadays, several space companies are investing resources into Hera mission (which has been proposed by the European Space Agency). In addition to visiting the Didymos asteroid binary system to acquire important scientific discoveries, Hera mission will also be the first mission to operate autonomously around an asteroid. The spacecraft will fuse data from its Asteroid Framing Camera (AFC), star-tracker, laser altimeter, thermal infrared camera and inertial sensors to build up a coherent model of its surroundings and be able to behave autonomously. Despite that, the AFC is considered to be the most crucial data source because of its dual science and navigation functionality. In fact, although images acquired by the camera are going to be used for scientific purposes, during the mission their main usage will be devoted to navigating autonomously around the main asteroid, Didymain. Because of this reason, several hardware-in-the-loop tests are being made in laboratories to analyze the accuracy of specific feature detection, extraction and tracking algorithms. Along with the computer vision part, navigation filters are being tested on the captured images to prove the feasibility of shape reconstruction and relative pose estimation (i.e., relative position, velocity, attitude and angular velocity). This paper presents an algorithm which has been built to use images acquired by a camera to reconstruct Didymain’s relative pose and shape and validates it with both computer simulations and experimental tests. First, the algorithm is going to be explained in all its parts: the computer vision block makes use of feature detection and extraction methods (e.g., KAZE, SIFT, etc.) and tracking algorithms (e.g., Kanade–Lucas–Tomasi Feature Tracker) to extract meaningful information from the captured images; the navigation filter, developed following an Unscented Kalman-like scheme, is going to use those data to reconstruct the asteroid’s pose and shape. Second, a simulator built in Matlab environment, which uses a 3D CAD of the asteroid to generate synthetic images in closed loop, is being used to prove the feasibility of this concept: several simulation scenario with different initial conditions will be analyzed to prove the robustness of this approach. Finally, the algorithm is going to be tested and validated through experimental tests performed at platform-art©, GMV’s Advanced Robotic Testbed in Madrid, where real images of a Didymain asteroid’s mock-up are going to be framed and used in the loop. The results obtained by the software testing and hardware-in-the-loop validation steps show that the proposed technique attains a good estimation accuracy as long as the relative configuration between spacecraft and asteroid is not singular (thus unobservable), meeting mission requirements and thus being suitable for an on-board implementation.