European Proximity Operations Simulator Research Articles

Lidar Pose Tracking of a Tumbling Spacecraft Using the Smoothed Normal Distribution Transform

Lidar sensors enable precise pose estimation of an uncooperative spacecraft in close range. In this context, the iterative closest point (ICP) is usually employed as a tracking method. However, when the size of the point clouds increases, the required computation time of the ICP can become a limiting factor. The normal distribution transform (NDT) is an alternative algorithm which can be more efficient than the ICP, but suffers from robustness issues. In addition, lidar sensors are also subject to motion blur effects when tracking a spacecraft tumbling with a high angular velocity, leading to a loss of precision in the relative pose estimation. This work introduces a smoothed formulation of the NDT to improve the algorithm’s robustness while maintaining its efficiency. Additionally, two strategies are investigated to mitigate the effects of motion blur. The first consists in un-distorting the point cloud, while the second is a continuous-time formulation of the NDT. Hardware-in-the-loop tests at the European Proximity Operations Simulator demonstrate the capability of the proposed methods to precisely track an uncooperative spacecraft under realistic conditions within tens of milliseconds, even when the spacecraft tumbles with a significant angular rate.

Domain randomisation and CNN-based keypoint-regressing pose initialisation for relative navigation with uncooperative finite-symmetric spacecraft targets using monocular camera images

Vision-based relative navigation technology is a key enabler of several areas of the space industry such as on-orbit servicing, space debris removal, and formation flying. A particularly demanding scenario is navigating relative to a non-cooperative target that does not offer any navigational aid and is unable to stabilise its attitude. This research integrates a convolutional neural network (CNN) and an EPnP-solver in a pose initialisation system. The system’s performance is benchmarked on images gathered from the European Proximity Operations Simulator EPOS 2.0 laboratory. A synthetic dataset is generated using Blender as a rendering engine. A segmentation-based pose estimation CNN is trained using the synthetic dataset and the resulting pose estimation performance is evaluated on a set of real images gathered from the cameras of the EPOS 2.0 robotic close-range relative navigation laboratory. It is demonstrated that a synthetic-image-trained CNN-based pose estimation pipeline is able to successfully perform in a close-range visual relative navigation setting on real camera images of a 6-facet symmetrical spacecraft.

A robust navigation filter fusing delayed measurements from multiple sensors and its application to spacecraft rendezvous

A filter is an essential part of many control systems. For example guidance, navigation and control systems for spacecraft rendezvous require a robust navigation filter that generates estimates of the state in a smooth and stable way. This is important for a safe spacecraft navigation within rendezvous missions. Delayed, asynchronous measurements from possibly different sensors require a new filter technique which can handle these different challenges. A new method is developed which is based on an Extended Kalman Filter with several adaptations in the prediction and correction step. Two key aspects are extrapolation of delayed measurements and sensor fusion in the filter correction. The new filter technique is applied on different close-range rendezvous examples and tested at the hardware-in-the-loop facility EPOS 2.0 (European Proximity Operations Simulator) with two different rendezvous sensors. Even with realistic delays by using an ARM-based on-board computer in the hardware-in-the-loop tests the filter is able to provide accurate, stable and smooth state estimates in all test scenarios.

10-Year Anniversary of the European Proximity Operations Simulator 2.0—Looking Back at Test Campaigns, Rendezvous Research and Facility Improvements

Completed in 2009, the European Proximity Operations Simulator 2.0 (EPOS 2.0) succeeded EPOS 1.0 at the German Space Operations Center (GSOC). One of the many contributions the old EPOS 1.0 facility made to spaceflight rendezvous is the verification of the Jena-Optronik laser-based sensors used by the Automated Transfer Vehicle. While EPOS 2.0 builds upon its heritage, it is a completely new design aiming at considerably more complex rendezvous scenarios. During the last ten years, GSOC’s On-Orbit-Servicing & Autonomy group, who operates, maintains and evolves EPOS 2.0, has made numerous contributions to the field of uncooperative rendezvous, using EPOS as its primary tool. After general research in optical navigation in the early 2010s, the OOS group took a leading role in the DLR project “On-Orbit-Servicing End-to-End Simulation” in 2014. EPOS 2.0 served as the hardware in the loop simulator of the rendezvous phase and contributed substantially to the project’s remarkable success. Over the years, E2E has revealed demanding requirements, leading to numerous facility improvements and extensions. In addition to the OOS group’s research work, numerous and diverse open-loop test campaigns for industry and internal (DLR) customers have shaped the capabilities of EPOS 2.0 significantly.

End-to-end simulation and verification of GNC and robotic systems considering both space segment and ground segment

In the framework of a project called on-orbit servicing end-to-end simulation, the final approach and capture of a tumbling client satellite in an on-orbit servicing mission are simulated. The necessary components are developed and the entire end-to-end chain is tested and verified. This involves both on-board and on-ground systems. The space segment comprises a passive client satellite, and an active service satellite with its rendezvous and berthing payload. The space segment is simulated using a software satellite simulator and two robotic, hardware-in-the-loop test beds, the European Proximity Operations Simulator (EPOS) 2.0 and the OOS-Sim. The ground segment is established as for a real servicing mission, such that realistic operations can be performed from the different consoles in the control room. During the simulation of the telerobotic operation, it is important to provide a realistic communication environment with different parameters like they occur in the real world (realistic delay and jitter, for example).

Close Range Tracking of an Uncooperative Target in a Sequence of Photonic Mixer Device (PMD) Images

This paper presents a pose estimation routine for tracking attitude and position of an uncooperative tumbling spacecraft during close range rendezvous. The key innovation is the usage of a Photonic Mixer Device (PMD) sensor for the first time during space proximity for tracking the pose of the uncooperative target. This sensor requires lower power consumption and higher resolution if compared with existing flash Light Identification Detection and Ranging (LiDAR) sensors. In addition, the PMD sensor provides two different measurements at the same time: depth information (point cloud) and amplitude of the reflected signal, which generates a grayscale image. In this paper, a hybrid model-based navigation technique that employs both measurements is proposed. The principal pose estimation technique is the iterative closed point algorithm with reverse calibration, which relies on the depth image. The second technique is an image processing pipeline that generates a set of 2D-to-3D feature correspondences between amplitude image and spacecraft model followed by the Efficient Perspective-n-Points (EPnP) algorithm for pose estimation. In this way, we gain a redundant estimation of the target’s current state in real-time without hardware redundancy. The proposed navigation methodology is tested in the German Aerospace Center (DLR)’s European Proximity Operations Simulator. The hybrid navigation technique shows the capability to ensure robust pose estimation of an uncooperative tumbling target under severe illumination conditions. In fact, the EPnP-based technique allows to overcome the limitations of the primary technique when harsh illumination conditions arise.

European Proximity Operations Simulator 2.0 (EPOS) - A Robotic-Based Rendezvous and Docking Simulator

The European Proximity Operations Simulator (EPOS) 2.0 located at the German Space Operations Center (GSOC) in Oberpfaffenhofen, Germany, is a robotic based test facility of the German Aerospace Center (DLR) used for simulation of rendezvous and docking (RvD) processes. Hardware such as rendezvous sensors (cameras, laser scanners) or docking tools, as well as software (e.g. for navigation and control) can be tested and verified. The facility consists of two robotic manipulators with each six degrees of freedom, a linear slide of 25m length on which one robot can be moved in the laboratory, and a computer-based monitoring and control system. EPOS 2.0 allows for real-time simulations of the rendezvous and docking process during the most critical phase (separation from 25m to 0m) of proximity and docking/berthing operations.

Hardware-in-the-Loop Rendezvous Tests of a Novel Actuators Command Concept

Integration, test and validation results, in a real-time environment, of a novel concept for spacecraft control are presented in this paper. The proposed method commands simultaneously a group of actuators optimizing a given set of objective functions based on a multiobjective optimization technique. Since close proximity maneuvers play an important role in orbital servicing missions, the entire GNC system has been integrated and tested at a hardware-in-the-loop (HIL) rendezvous and docking simulator known as European Proximity Operations Simulator (EPOS). During the test campaign at EPOS facility, a visual camera has been used to provide the necessary measurements for calculating the relative position with respect to the target satellite during closed-loop simulations. In addition, two different configurations of spacecraft control have been considered in this paper: a thruster reaction control system and a mixed actuators mode which includes thrusters, reaction wheels, and magnetic torqrods. At EPOS, results of HIL closed-loop tests have demonstrated that a safe and stable rendezvous approach can be achieved with the proposed GNC loop.

Vision-based localization for on-orbit servicing of a partially cooperative satellite

This paper proposes ground-in-the-loop, model-based visual localization system based on transmitted images to ground, to aid rendezvous and docking maneuvers between a servicer and a target satellite. In particular, we assume to deal with a partially cooperative target, i.e. passive and without fiducial markers, but supposed at least to keep a controlled attitude, up to small fluctuations, so that the approach mainly involves translational motion. For the purpose of localization, video cameras provide an effective and relatively inexpensive solution, working at a wide range of distances with an increasing accuracy and robustness during the approach. However, illumination conditions in space are especially challenging, due to the direct sunlight exposure and to the glossy surface of a satellite, that creates strong reflections and saturations and therefore a high level of background clutter and missing detections. We employ a monocular camera for mid-range tracking (20–5m) and stereo camera at close-range (5–0.5m), with the respective detection and tracking methods, both using intensity edges and robustly dealing with the above issues. Our tracking system has been extensively verified at the facility of the European Proximity Operations Simulator (EPOS) of DLR, which is a very realistic ground simulation able to reproduce sunlight conditions through a high power floodlight source, satellite surface properties using multilayer insulation foils, as well as orbital motion trajectories with ground-truth data, by means of two 6 DOF industrial robots. Results from this large dataset show the effectiveness and robustness of our method against the above difficulties.

Relative pose estimation of satellites using PMD-/CCD-sensor data fusion

Rendezvous & Docking to passive objects, as of relevance for space debris removal, raises new challenges with respect to relative navigation. Whenever the position and orientation (pose) of an object is required in terrestrial and in space applications, sensor systems such as laser scanners and stereo vision systems are often employed. This paper presents an approach to pose estimation using a 3D time-of-flight camera for ranging information in combination with a high resolution grayscale camera. We have designed a pose estimation method that fuses the data streams of the two sensors in order to benefit from each sensors׳ advantages. A rigorous test campaign on a Real-Time Hardware-In-The-Loop Rendezvous and Docking Simulator – the European Proximity Operations Simulator (EPOS) – was performed in order to evaluate the performance of the resulting algorithm. The proposed pose estimation method does not exceed an average distance error of 3cm while being capable of providing pose estimates at up to 60 FPS on recent hardware. Thus, when regarding proximity operations, an attractive sensor system is used to characterize the dynamics of the target object for safe approach results.

Analytical and experimental stability investigation of a hardware-in-the-loop satellite docking simulator

The European Proximity Operations Simulator of the DLR-German Aerospace Center is a robotics-based simulator that aims at validating and verifying a satellite docking phase. The generic concept features a robotics tracking system working in closed loop with a force/torque feedback signal. Inherent delays in the tracking system combined with typical high stiffness at contact challenge the stability of the closed-loop system. The proposed concept of operations is hybrid: the feedback signal is a superposition of a measured value and of a virtual value that can be tuned in order to guarantee a desired behavior. This paper is concerned with an analytical study of the system’s closed-loop stability, and with an experimental validation of the hybrid concept of operations in one dimension. The robotics simulator is modeled as a second-order loop-delay system and closed-form expressions for the critical delay and associated frequency are derived as a function of the satellites’ mass and the contact dynamics stiffness and damping parameters. A numerical illustration sheds light on the impact of the parameters on the stability regions. A first-order Pade approximation provides additional means of stability investigation. Experiments were performed and test results are described for varying values of the mass and the damping coefficients. The empirical determination of instability is based on the coefficient of restitution and on the observed energy. There is a very good agreement between the critical damping values predicted by the analysis and observed during the tests. The contact duration shows also a very good fit between analysis and experiment. In addition, results from a one-dimensional contact experiment carried on an air-floating testbed are successfully emulated using the proposed hybrid docking simulator. This illustrates the flexibility of the hybrid simulator, where various contact dynamics can be emulated without changing any hardware elements.