Descent Images Research Articles

Vision based Chang’E-4 landing point localization

On December 8, 2018, China launched the Chang’E-4 lunar probe and implemented soft landing and patrol exploration on the lunar farside for the first time. Fast and high-precision positioning of the landing point is a critical step of the mission, and also an important prerequisite for surface operations of the lander and rover. Based on the high-precision image matching and geometric transformation methods, and considering the engineering requirements, the landing point location of the Chang’E-4 lander was initially determined using the near real-time high compression ratio descent sequence images. Then, the position was refined using the replayed low compression ratio descent images, and the landing point position was calculated to be (177.5884°E, 45.4565°S). The landing point localization method and result have been successfully applied to actual engineering tasks for the first time in China, thus providing important support to the topographic analysis of the landing area and mission planning of the follow-up teleoperations.

DESCENT TRAJECTORY RECOVERY OF CHANG’E-4 LANDER BASED ON DESCENT IMAGES

Abstract. Chang’e-4 lander, carrying Yutu-2 rover, was successfully landed on the far side of lunar surface in Von Kármán crater inside the South Pole-Aitken basin on January 3rd, 2019. The descent images, captured by the descent camera mounted on the lander, captured the sequential descent images and recorded the scene changes during the entry, descent and landing (EDL) process. This paper proposed a bundle adjustment based geometric processing method for descent and landing trajectory recovery using descent images. A frame camera based self-calibration model was introduced for high precision estimation of interior and exterior parameters of descent images simultaneously in a least squares manners. Evenly distributed GCPs were selected from the landing area in a digital orthophoto map generated from LROC NAC images and SLDEM2015. The experimental results demonstrated the effectiveness of the proposed method in Chang’e-4 descent trajectory recovery.

TOPOGRAPHIC ANALYSIS OF CHANG’E-4 LANDING SITE USING ORBITAL, DESCENT AND GROUND DATA

Abstract. The Chang’e-4 (CE-4) probe, which includes a lander and a rover, was successfully landed in Von Kármán crater inside the South Pole-Aitken basin on January 3, 2019. Landing site mapping and topographic analyses have been performed at different scales using orbital, descent and ground data to support mission operations and various scientific investigations. We produced a 0.9 m-resolution seamless digital orthophoto map (DOM) mosaic of the CE-4 landing site region covering 51 km × 30 km using 100 Lunar Reconnaissance Orbiter Camera (LROC) Narrow-Angle Camera (NAC) images. Within an area of 3.2 km × 3.0 km of the the DOM around the lander, more than 11,000 craters with diameter greater than 5 m have been identified and measured. A 0.03 m- resolution DOM, which covers an area of 211 m × 187 m, was generated using descent images, and has been used as one of the base maps for overall rover traverse planning. During surface operations, local DEMs with 0.02 m resolution are routinely produced and topographic analysis have been performed at each waypoint using Navcam images and at some locations using Pancam images, to support waypoint-to-waypoint path planning, science target selection and scientific investigations.

Single crater-aided inertial navigation for autonomous asteroid landing

In this paper, a novel crater-aided inertial navigation approach for autonomous asteroid landing mission is developed. It overcomes the major deficiencies of existing approaches in the literature, which mainly focuses on the case where craters are abundant in the camera field of view. As a result, traditional crater based methods require at least three craters to achieve crater matching, which limits their application in final landing phase where craters are scarce in the camera’s field of view. In contrast, the proposed algorithm enables single crater based crater matching based on a novel 2D-3D crater re-projection model. The re-projection model adopts inertial measurements as a reference, and re-projects the 3D crater model onto descent images to achieve the matching to its counterpart. An asteroid landing simulation toolbox is developed to validate the performance of the proposed approach. Through comparison with the state-of-the-art local image feature and crater based navigation algorithms, the proposed approach is validated to achieve a competitive performance in terms of feature matching and pose estimation accuracy with a much lighter computational cost.

A novel inertial-aided feature detection model for autonomous navigation in planetary landing

This paper proposes a novel visual feature detection model for navigation applications in the context of autonomous planetary landing. A novel Hessian marker is developed in the first place, which aims at adapting to the affine distortion between descent image and navigational map, such design is established upon the SURF detection model. Then, an inertial-aided characteristic scale determination method is proposed based upon the proposed Hessian marker, leading to the complete inertial-aided visual feature detection model. Finally, a planet-representative simulation platform is developed for validating the proposed algorithm, the performance of proposed approach in terms of repeatability rate, robustness against the image affine distortion, and navigation accuracy is verified by comparing with the state-of-the-art algorithms under the specifically developed platform.

Analysis of Rock Abundance on Lunar Surface From Orbital and Descent Images Using Automatic Rock Detection

AbstractInformation on the distribution and abundance of rocks on the lunar surface is essential for understanding the characteristics of lunar regolith and for selecting suitable landing sites for lunar exploration missions. This paper first presents a novel automatic rock detection approach based on the gradient differences in illumination direction. Multiple‐source images covering different regions of the lunar surface are then used for rock detection and rock abundance analysis, including the Chang'E‐3 descent images (0.02–0.17 m/pixel) around the landing area, the Lunar Reconnaissance Orbiter (LRO) Narrow Angle Camera images (0.3–1.1 m/pixel) at the landing sites of the Chang'E‐3, Luna 17, and Luna 23, and the LRO Narrow Angle Camera image mosaic (1.5 m/pixel) in the Oceanus Procellarum area. A rock abundance model using an exponential form is derived from the results of the abovementioned analysis to describe the overall cumulative fractional area of rocks versus their diameters. The rock detection results and the derived rock abundance estimated by the proposed model are compared with the ground measurements obtained using the rover images of the Chang'E‐3 landing site and the LRO Diviner Radiometer‐derived rock abundance. A comparison analysis indicates that the derived rock abundance model can feasibly represent the rock abundance in various situations on local and large scales. In general, a remarkable consistency is observed between our results and the results obtained using the ground measurements at the Chang'E‐3 landing site and the orbit measurements from the Diviner radiometer, while our rock detection results and rock abundance model exhibit better performance in presenting detailed information in local areas. Further analyses on rock abundance and the crater morphology and other crater characteristics in the Oceanus Procellarum area indicate that it is feasible to use rock abundance information to estimate the surface maturity.

Spatially adaptive denoising for X-ray cardiovascular angiogram images

Abstract The X-ray angiogram image denoising is always one of the most popular research in the field of computer vision. While the methods removed the noise, the useful structure (such as peripheral vascular) had also been smoothed, the fundamental reason is that the denoising methods cannot efficiently distinguish structural areas from flat areas. In this paper, we have proposed a spatially adaptive image denoising (SAID) method which contains two steps: spatially adaptive gradient descent (SAGD) image denoising and dual-domain filter (DDF). The SAGD denoising method contains the following parts: first of all, the wavelet shrinkage method is used to estimate redundant information which is composed of the noise and useful structures; secondly, according to the characteristic of second order matrix, a spatially adaptive gradient factor (SAGF) has been constructed to distinguish the structure from flat areas; finally, the SAGF replaces the original gradient factor and then the SAGD image denoising method is formed. To further improve the quality of the SAGD image, the SAGD image is re-denoised by a modified DDF which is guided with a rotationally invariant non-local filter (RINLF) in spatial domain and gets structural details by wavelet shrinkage in frequency domain. The results of simulation experiments verify that the proposed SAID method can get well quantitative and qualitative results which are even superior to those using the state-of-the-art denoising methods. Even more, the fluctuation of peak signal-to-noise ratio (PSNR) value is very small with a small disturbance of SAGF, which illustrates that our algorithm is more robust than the prior progressive image denoising (PID) method. Moreover, the comparison results of the extensive experiments on clinical X-ray cardiovascular angiogram images further illustrate that our method can yield clearer cardiovascular images which can provide more useful vascular information for clinicians to analyze and diagnose the cardiovascular diseases.

FUSION OF MULTI-SCALE DEMS FROM DESCENT AND NAVCM IMAGES OF CHANG’E-3 USING COMPRESSED SENSING METHOD

Abstract. The multi-source DEMs generated using the images acquired in the descent and landing phase and after landing contain supplementary information, and this makes it possible and beneficial to produce a higher-quality DEM through fusing the multi-scale DEMs. The proposed fusion method consists of three steps. First, source DEMs are split into small DEM patches, then the DEM patches are classified into a few groups by local density peaks clustering. Next, the grouped DEM patches are used for sub-dictionary learning by stochastic coordinate coding. The trained sub-dictionaries are combined into a dictionary for sparse representation. Finally, the simultaneous orthogonal matching pursuit (SOMP) algorithm is used to achieve sparse representation. We use the real DEMs generated from Chang’e-3 descent images and navigation camera (Navcam) stereo images to validate the proposed method. Through the experiments, we have reconstructed a seamless DEM with the highest resolution and the largest spatial coverage among the input data. The experimental results demonstrated the feasibility of the proposed method.

Landmark-based autonomous navigation for pinpoint planetary landing

A landmark-based autonomous navigation scheme is presented for pinpoint planetary landing. The dynamic model is built on the basis of measurements from Inertial Measurement Unit. Measurement models of landmarks with known coordinates and landmarks with unknown coordinates extracted from sequential descent images are developed and used to calculated the state corrections in Extend Kalman Filter, respectively. Then, the corrections are fused by a covariance intersection fusion algorithm to perform state updates. The tight coupling of the two types of landmark observations yields accurate and robust state estimates. Extensive simulations are performed, which confirm the validity of the proposed navigation scheme and analyze the effects of factors, such as the horizonal position errors and the densities of landmarks with known coordinates and the roughness of the landing surface, on the navigation accuracy.

English

An optimization method is proposed to perform high-accuracy terrain reconstruction of the landing area of Chang’e III. First, feature matching is conducted using geometric model constraints. Then, the initial terrain is obtained and the initial normal vector of each point is solved on the basis of the initial terrain. By changing the vector around the initial normal vector in small steps a set of new vectors is obtained. By combining these vectors with the direction of light and camera, the functions are set up on the basis of a surface reflection model. Then, a series of gray values is derived by solving the equations. The new optimized vector is recorded when the obtained gray value is closest to the corresponding pixel. Finally, the optimized terrain is obtained after iteration of the vector field. Experiments were conducted using the laboratory images and descent images of Chang’e III. The results showed that the performance of the proposed method was better than that of the classical feature matching method. It can provide a reference for terrain reconstruction of the landing area in subsequent moon exploration missions.

Relative pose estimation of a lander using crater detection and matching

Future space exploration missions require precise information about the lander pose during the descent and landing steps. An effective algorithm that utilizes crater detection and matching is presented to determine the lander pose with respect to the planetary surface. First, the projections of the crater circular rims in the descent image are detected and fitted into ellipses based on the geometric distance and coplanar circles constraint. Second, the detected craters are metrically rectified through a two-dimensional homography and matched with the crater database by similarity transformation. Finally, the lander pose is calculated by a norm-based optimization method. The algorithm is tested by synthetic and real trials. The experimental results show that our presented algorithm can determine the lander pose accurately and robustly.

Visual–inertial navigation for pinpoint planetary landing using scale-based landmark matching

Landing an autonomous spacecraft within 100 m of a mapped target is a navigation challenge in planetary exploration. Vision-based approaches attempt to pair 2D features detected in camera images with 3D mapped landmarks to reach the required precision. This paper presents a vision-aided inertial navigation system for pinpoint planetary landing called LION (Landing Inertial and Optical Navigation). It can fly over any type of terrain, whatever the topography. LION uses measurements from a novel image-to-map matcher in order to update through a tight data fusion scheme the state of an extended Kalman filter propagated with inertial data. The image processing uses the state and covariance predictions from the filter to determine the regions and extraction scales in which to search for non-ambiguous landmarks in the image. The image scale management process operates per landmark and greatly improves the repeatability rate between the map and descent images. A lunar-representative optical test bench called Visilab was also designed in order to test LION. The observability of absolute navigation performances in Visilab is evaluated with a model developed specifically for this purpose. Finally, the system performances are evaluated at a number of altitudes along with its robustness to off-nadir camera angle, illumination changes, a different map generation process and non-planar topography. The error converges to a mean of 4 m and a 3-RMS dispersion of 47 m at 3 km of altitude on the test setup at scale.

Descent image based landing area terrain reconstruction technology for lunar landing mission

In this paper, we present a descent image based extraterrestrial terrain reconstruction technology, which has been successfully applied in China's first soft landing Chang'E-3 mission on moon. When scouting on the moon surface, the lunar rover (Jade Rabbit rover) requires reliable terrain information for navigation and positioning tasks. The descent image sequence acts as a link between the orbital images and the surface images. The proposed approach collects multilevel descent images, as well as inertial information from an inertial measurement unit and surface stereo vision information, to generalise a consistent surface model. A data fusion strategy is proposed to track the camera trajectory, and refined plane sweep stereo matching and 3D alignment technologies are analysed in this article. The onboard result, as well as on-earth simulation experiments, has shown that the discussed techniques are reliable to reconstruct the terrain information for the lunar rover's localisation and navigation.

Terrain reconstruction based on descent images for the Chang ’ e III landing area

Terrain reconstruction based on descent images for the Chang ’ e III landing area

Improved best match search method in depth recovery with descent images

Searching for the best match is a key step in stereovision or 3D reconstruction with images. In our previous study, we recovered every pixel's depth in the lower image by searching for the highest correlation in its dimensional ZNCC correlation curve in the cubic correlation matrix as the best match, but sparse reconstruction errors such as holes or spikes existed in the result. After careful analysis we found that not all the highest-correlational positions are corresponding to their correct match due to noise or similarity. Therefore, a new best match search method based on best seed propagation first strategy is proposed by considering neighborhood disparity constraints. At first, some pixels are chosen as initial seeds and inserted into a seed queue by assessing their correlation curves. Their depths are determined by the layers in the cubic correlation matrix in which they get their highest correlation value. Second, the front seed is taken out of the queue and its neighbor points are propagated as new seeds under the propagation rules. The new propagated seeds will also be inserted into the seed queue, and their depth are accordingly decided. This operation is repeated till the seed queue is null. At last, there will be some points which are never propagated as seeds according to the propagation rules. Their depths are determined by their neighbor points depth information through post processing. The comparison experiments show that the new method can improve the accuracy of the matches and reduce the reconstruction error effectively.

High precision landing site mapping and rover localization for Chang’e-3 mission

This paper presents the comprehensive results of landing site topographic mapping and rover localization in Chang’e-3 mission. High-precision topographic products of the landing site with extremely high resolutions (up to 0.05 m) were generated from descent images and registered to CE-2 DOM. Local DEM and DOM with 0.02 m resolution were produced routinely at each waypoint along the rover traverse. The lander location was determined to be (19.51256°W, 44.11884°N, −2615.451 m) using a method of DOM matching. In order to reduce error accumulation caused by wheel slippage and IMU drift in dead reckoning, cross-site visual localization and DOM matching localization methods were developed to localize the rover at waypoints; the overall traveled distance from the lander is 114.8 m from cross-site visual localization and 111.2 m from DOM matching localization. The latter is of highest accuracy and has been verified using a LRO NAC image where the rover trajeactory is directly identifiable. During CE-3 mission operations, landing site mapping and rover localization products including DEMs and DOMs, traverse maps, vertical traverse profiles were generated timely to support teleoperation tasks such as obstacle avoidance and rover path planning.

Chang’E-3 system pinpoint landing localization based on descent image sequence

Realizing the pinpoint landing localization for the ChangE-3 system is a symbol of success for this mission, and it is also a basic requirement for the lunar Lander and Rover to carry out the following scientific exploration tasks. In this paper, jointly dealing with the descent image sequence and ChangE-2 DOM data, an approach based on the images processing techniques, such as SIFT feature matching, homography transportation and stereo vision measurement, is presented to realize the pinpoint localization of the ChangE-3 system. The localization result is proved to be reliable and accuracy and the result will be a critical component for the ChangE-3 mission planning and the completion of scientific exploration.

High Precision Topographic Mapping at Chang'E-3 Landing Site with Multi-Source Data

Abstract. Chang'e-3 (CE-3) is the first lander and rover of China following the success of Chang'e-1 and Chang'e-2 (CE-2) orbiters. High precision topographic mapping can provide detailed terrain information to ensure the safety of the rover as well as to support scientific investigations. In this research, multi-source data are co-registered into a uniform geographic framework for high precision topographic mapping at the CE-3 landing site. CE-2 CCD images with 7 m- and 1.5 m- resolutions are registered using selfcalibration bundle adjustment method with ground control points (GCPs) selected from LRO WAC mosaic map and LOLA DTM. The trajectory of CE-3 descent images are recovered using self-calibration free net bundle adjustment, and then the topographic data is rectified by absolute orientation with GCPs selected from the adjusted CE-2 DEM and DOM. Finally, these topographic data are integrated into the same geographic framework for unified, multi-scale, high precision mapping of the CE-3 landing site. Key technologies and the mapping products of this research have been used to support the surface operations of CE-3 mission.

Horizontal velocity estimation via downward looking descent images for lunar landing

In lunar landing missions, it is very important to closely estimate the horizontal velocity of a descending spacecraft for achieving a successful and safe lunar landing. The purpose of this paper is to present a novel, vision-aided approach for the accurate, efficient, and robust estimation of such horizontal velocity. Our algorithm processes images from a downward-looking camera, as well as attitude and altitude information from other sensors, to estimate horizontal velocities. During descent, images vary greatly in scale, orientation, and viewpoint. To begin, the scale-invariant feature transform (SIFT) algorithm copes with such shifts, so one is able to use extracted keypoints to establish correspondences between consecutive descent images. Then, matched SIFT keypoints are projected to the level ground plane according to the measurement of the camera state and the central projection imaging collinear equation. A 1-point random sample consensus (RANSAC) algorithm is adopted to remove mismatched keypoints. From each correctly matched keypoint pair, the algorithm derives a hypothesis for the spacecraft displacement relative to lunar ground, since those keypoints represent measurements of the same position on the lunar surface. From the bundle of displacement hypotheses, our algorithm robustly recovers the mode of the sample distribution. This final horizontal displacement estimate of the spacecraft is obtained by using the mean shift method to search for an appropriate answer among these hypotheses. In combination with the time interval between shots, the horizontal velocity is estimated. Additionally, a digital signal processor with field-programmable gate array architecture is also presented to implement velocity estimation in real time. We evaluate the performance of our algorithm based on numerous simulated image sequences and real flight images compared with the descent image motion estimation system approach and an extended Kalman filter monocular simultaneous localization and mapping method.

Fourier Lucas-Kanade Algorithm

In this paper, we propose a framework for both gradient descent image and object alignment in the Fourier domain. Our method centers upon the classical Lucas & Kanade (LK) algorithm where we represent the source and template/model in the complex 2D Fourier domain rather than in the spatial 2D domain. We refer to our approach as the Fourier LK (FLK) algorithm. The FLK formulation is advantageous when one preprocesses the source image and template/model with a bank of filters (e.g., oriented edges, Gabor, etc.) as 1) it can handle substantial illumination variations, 2) the inefficient preprocessing filter bank step can be subsumed within the FLK algorithm as a sparse diagonal weighting matrix, 3) unlike traditional LK, the computational cost is invariant to the number of filters and as a result is far more efficient, and 4) this approach can be extended to the Inverse Compositional (IC) form of the LK algorithm where nearly all steps (including Fourier transform and filter bank preprocessing) can be precomputed, leading to an extremely efficient and robust approach to gradient descent image matching. Further, these computational savings translate to nonrigid object alignment tasks that are considered extensions of the LK algorithm, such as those found in Active Appearance Models (AAMs).