On-board Autonomy Research Articles

On-board autonomy operations for OPS-SAT experiment

Upcoming space missions are requiring a higher degree of on-board autonomy operations to increase quality science return, to minimize closed-loop space-ground decision making, and to enable new scenarios. Artificial Intelligence technologies like Machine Learning and Automated Planning are becoming more and more popular as they can support data analytics conducted directly on-board as input for the on-board decision making system that generates plans or updates them while being executed. This paper describes the planning and execution architecture under development at the European Space Agency to target this need of autonomy for the ops-sat mission.

Mars curiosity rover mobility trends during the first 7 years

AbstractNASA's Mars Science Laboratory (MSL) Curiosity rover landed on Mars on August 6, 2012. In the 7 years between landing and August 6, 2019 (sol 2488), Curiosity has driven 21,318.5 m over a variety of terrain types and slopes, employing multiple drive modes with varying amounts of onboard autonomy. Curiosity's drive distances each sol have ranged from its shortest drive of 2.6 cm to its longest drive of 142.5 m, with an average drive distance of 28.9 m. Real‐time human intervention is not possible during Curiosity's drives due to the latency in uplinking commands and downlinking telemetry. Instead, the operations team relies on Curiosity's fault protection, autonomous navigation, and visual odometry software to keep the rover safe during drives. During its first 7 years on Mars, Curiosity has attempted 738 drives. While 622 drives ran to completion, 116 drives were prevented or stopped early by Curiosity's fault protection software. The primary risks to mobility success have been wheel damage, wheel entrapment, progressive wheel sinkage, and the potential for hardware or cable failures that result in an inability to command one or more steer or drive actuators. In this paper, we describe Curiosity's mobility subsystem, mobility trends over the first 21.3 km of the mission, operational aspects of mobility fault protection, risks to continued mobility success, and risk mitigation strategies.

Mapping Burned Areas of Mato Grosso State Brazilian Amazon Using Multisensor Datasets

Quantifying forest fires remain a challenging task for the implementation of public policies aimed to mitigate climate change. In this paper, we propose a new method to provide an annual burned area map of Mato Grosso State located in the Brazilian Amazon region, taking advantage of the high spatial and temporal resolution sensors. The method consists of generating the vegetation, soil, and shade fraction images by applying the Linear Spectral Mixing Model (LSMM) to the Landsat-8 OLI (Operational Land Imager), PROBA-V (Project for On-Board Autonomy–Vegetation), and Suomi NPP-VIIRS (National Polar-Orbiting Partnership-Visible Infrared Imaging Radiometer Suite) datasets. The shade fraction images highlight the burned areas, in which values are represented by low reflectance of ground targets, and the mapping was performed using an unsupervised classifier. Burned areas were evaluated in terms of land use and land cover classes over the Amazon, Cerrado and Pantanal biomes in the Mato Grosso State. Our results showed that most of the burned areas occurred in non-forested areas (66.57%) and old deforestation (21.54%). However, burned areas over forestlands (11.03%), causing forest degradation, reached more than double compared with burned areas identified in consolidated croplands (5.32%). The results obtained were validated using the Sentinel-2 data and compared with active fire data and existing global burned areas products, such as the MODIS (Moderate Resolution Imaging Spectroradiometer product) MCD64A1 and MCD45A1, and Fire CCI (ESA Climate Change Initiative) products. Although there is a good visual agreement among the analyzed products, the areas estimated were quite different. Our results presented correlation of 51% with Sentinel-2 and agreement of r2 = 0.31, r2 = 0.29, and r2 = 0.43 with MCD64A1, MCD45A1, and Fire CCI products, respectively. However, considering the active fire data, it was achieved the better performance between active fire presence and burn mapping (92%). The proposed method provided a general perspective about the patterns of fire in various biomes of Mato Grosso State, Brazil, that are important for the environmental studies, specially related to fire severity, regeneration, and greenhouse gas emissions.

Three-Dimensional Reconstructions of Coronal Wave Surfaces Using a New Mask-Fitting Method

Coronal waves are large-scale disturbances often driven by coronal mass ejections (CMEs). We investigate a spectacular wave event on 7 March 2012, which is associated with an X5.4 flare (SOL2012-03-07). By using a running center-median (RCM) filtering method for the detection of temporal variations in extreme ultraviolet (EUV) images, we enhance the EUV disturbance observed by the Atmospheric Imaging Assembly (AIA) onboard the Solar Dynamics Observatory (SDO) and the Sun Watcher using Active Pixel System detector and Image Processing (SWAP) onboard the PRoject for Onboard Autonomy 2 (PROBA2). In coronagraph images, a halo front is observed to be the upper counterpart of the EUV disturbance. Based on the EUV and coronagraph images observed from three different perspectives, we have made three-dimensional (3D) reconstructions of the wave surfaces using a new mask-fitting method. The reconstructions are compared with those obtained from forward-fitting methods. We show that the mask fitting method can reflect the inhomogeneous coronal medium by capturing the concave shape of the shock wave front. Subsequently, we trace the developing concave structure and derive the deprojected wave kinematics. The speed of the 3D-wave nose increases from a low value below a few hundred $\mathrm{km\,s^{-1}}$ to a maximum value of about 3800 $\mathrm{km\,s^{-1}}$, and then slowly decreases afterwards. The concave structure starts to decelerate earlier and has significantly lower speeds than those of the wave nose. We also find that the 3D-wave in the extended corona has a much higher speed than the speed of EUV disturbances across the solar disk.

Coordination of the in situ payload of Solar Orbiter

Solar Orbiter’s in situ coordination working group met frequently during the development of the mission with the goal of ensuring that its in situ payload has the necessary level of coordination to maximise science return. Here we present the results of that work, namely how the design of each of the in situ instruments (EPD, MAG, RPW, SWA) was guided by the need for coordination, the importance of time synchronisation, and how science operations will be conducted in a coordinated way. We discuss the mechanisms by which instrument sampling schemes are aligned such that complementary measurements will be made simultaneously by different instruments, and how burst modes are scheduled to allow a maximum overlap of burst intervals between the four instruments (telemetry constraints mean different instruments can spend different amounts of time in burst mode). We also explain how onboard autonomy, inter-instrument communication, and selective data downlink will be used to maximise the number of transient events that will be studied using high-resolution modes of all the instruments. Finally, we briefly address coordination between Solar Orbiter’s in situ payload and other missions.

Coordination within the remote sensing payload on the Solar Orbiter mission

Context.To meet the scientific objectives of the mission, the Solar Orbiter spacecraft carries a suite of in-situ (IS) and remote sensing (RS) instruments designed for joint operations with inter-instrument communication capabilities. Indeed, previous missions have shown that the Sun (imaged by the RS instruments) and the heliosphere (mainly sampled by the IS instruments) should be considered as an integrated system rather than separate entities. Many of the advances expected from Solar Orbiter rely on this synergistic approach between IS and RS measurements.Aims.Many aspects of hardware development, integration, testing, and operations are common to two or more RS instruments. In this paper, we describe the coordination effort initiated from the early mission phases by the Remote Sensing Working Group. We review the scientific goals and challenges, and give an overview of the technical solutions devised to successfully operate these instruments together.Methods.A major constraint for the RS instruments is the limited telemetry (TM) bandwidth of the Solar Orbiter deep-space mission compared to missions in Earth orbit. Hence, many of the strategies developed to maximise the scientific return from these instruments revolve around the optimisation of TM usage, relying for example on onboard autonomy for data processing, compression, and selection for downlink. The planning process itself has been optimised to alleviate the dynamic nature of the targets, and an inter-instrument communication scheme has been implemented which can be used to autonomously alter the observing modes. We also outline the plans for in-flight cross-calibration, which will be essential to the joint data reduction and analysis.Results.The RS instrument package on Solar Orbiter will carry out comprehensive measurements from the solar interior to the inner heliosphere. Thanks to the close coordination between the instrument teams and the European Space Agency, several challenges specific to the RS suite were identified and addressed in a timely manner.

Analysis of Large Deflections of Prominence–CME Events during the Rising Phase of Solar Cycle 24

Motivated by the need to improve the ability to forecast whether a certain coronal mass ejection (CME) is to impact Earth, and by the insufficiency of statistical studies that analyze the whole erupting system with the focus on the governing conditions under CME deflections, we performed a careful analysis of 13 events along a one-year time interval showing large deflections from their source region. We used telescopes imaging the solar corona at different heights and wavelengths on board the Project for Onboard Autonomy 2 (PROBA2), Solar Dynamics Observatory (SDO), Solar TErrestrial RElations Observatory (STEREO), Solar and Heliospheric Observatory (SOHO) spacecraft and from National Solar Observatory (NSO). By taking advantage of the quadrature position of these spacecraft from October 2010 until September 2011, we inspected the 3D trajectory of CMEs and their associated prominences with respect to their solar sources by means of a tie-pointing tool and a forward model. Considering the coronal magnetic fields as computed from a potential field source surface model, we investigate the roles of magnetic energy distribution and kinematic features in the non-radial propagation of both structures. The magnetic environment present during the eruption is found to be crucial in determining the trajectory of CMEs, in agreement with previous reports.

Quality Assessment of PROBA-V Surface Albedo V1 for the Continuity of the Copernicus Climate Change Service

The Copernicus Climate Change Service (C3S) includes estimates of Essential Climate Variables (ECVs) as a series of Climate Data Records (CDRs) derived from satellite data. The C3S Surface Albedo (SA) v1.0 CDR is composed of observations from National Oceanic and Atmospheric Administration (NOAA) Very High Resolution Radiometers (AVHRR) (1981–2005), and VEGETATION sensors onboard Satellites for the Observation of the Earth (SPOT/VGT) (1998–2014) and Project for Onboard Autonomy satellite (PROBA-V) (2014–2020), and will continue with Sentinel-3 (from 2020 onwards). The goal of this study is to assess the uncertainties associated with the C3S PROBA-V SA v1.0 product, with a focus on the transition from SPOT/VGT to PROBA-V. The methodology followed the good practices recommended by the Land Product Validation sub-group (LPV) of the Working Group on Calibration and Validation (WGCV) of the Committee on Earth Observing Satellites (CEOS) for the validation of satellite-derived global albedo products. Several performance criteria were evaluated, including an intercomparison with National Aeronautics and Space Agency (NASA) MCD43A3 C6 products. C3S PROBA-V SA v1.0 and MCD43A3 C6 showed similar completeness but had higher fractions of missing data than C3S SPOT/VGT SA v1.0. C3S PROBA-V SA v1.0 showed similar precision (~1%) to MCD43A3 C6, improving the results of SPOT/VGT SA v1.0 (2–3%), but C3S PROBA-V SA v1.0 provided residual noise in the near-infrared (NIR). Good spatio-temporal continuity between C3S PROBA-V and SPOT/VGT SA v1.0 products was found with a mean bias between ±2%. The comparison with MCD43A3 C6 showed positive mean biases (5%, 8%, and 12% for visible, NIR and total shortwave, respectively). The accuracy assessment with ground measurements showed a median error of 18.4% with systematic overestimation (positive bias of 11.5%). The percentage of PROBA-V retrievals complying with the C3S target requirements was 28.6%.

Investigation of the Middle Corona with SWAP and a Data-Driven Non-Potential Coronal Magnetic Field Model

The large field-of-view of the Sun Watcher using Active Pixel System detector and Image Processing (SWAP) instrument onboard the PRoject for Onboard Autonomy 2 (PROBA2) spacecraft provides a unique opportunity to study extended coronal structures observed in the EUV in conjunction with global coronal magnetic field simulations. A global non-potential magnetic field model is used to simulate the evolution of the global corona from 1 September 2014 to 31 March 2015, driven by newly emerging bipolar active regions determined from Helioseismic and Magnetic Imager (HMI) magnetograms. We compare the large-scale structure of the simulated magnetic field with structures seen off-limb in SWAP EUV observations. In particular, we investigate how successful the model is in reproducing regions of closed and open structures, the scale of structures, and compare the evolution of a coronal fan observed over several rotations. The model is found to accurately reproduce observed large-scale, off-limb structures. When discrepancies do arise they mainly occur off the east solar limb due to active regions emerging on the far side of the Sun, which cannot be incorporated into the model until they are observed on the Earth-facing side. When such “late” active region emergences are incorporated into the model, we find that the simulated corona self-corrects within a few days, so that simulated structures off the west limb more closely match what is observed. Where the model is less successful, we consider how this may be addressed, through model developments or additional observational products.

Neural network optimal control in astrodynamics: Application to the missed thrust problem

While high-efficiency propulsion techniques are enabling new mission concepts in deep space exploration, their limited thrust capabilities necessitate long thrusting arcs and make spacecraft more susceptible to missed thrust events. To correct for such mishaps, most spacecraft require updated trajectories that are relayed from Earth. While this solution is viable for spacecraft near Earth, in deep space, where one-way communication time is measured in hours, a delay in transmission may prolong the time of flight or result in a complete loss of mission. Such problems can be alleviated by increasing the spacecraft's onboard autonomy in guidance. This paper demonstrates how a computationally lightweight neural network can map the spacecraft's state to a near-optimal control action, autonomously guiding a spacecraft within different astrodynamic regimes and optimality criteria. The neural network is trained using supervised learning and datasets comprised of optimal state-action pairs, as determined through traditional direct and indirect methods. Additionally, the neural network-designed solutions retain optimality and time of flight corresponding to traditional trajectories. Finally, the same neural networks can autonomously correct for most missed thrust events encountered on long-duration low-thrust trajectories. The presented results provide a path for mitigating risks associated with the use of high-efficiency low-thrust propulsion techniques.

Magnetic survey and autonomous target reacquisition with a scalar magnetometer on a small AUV

AbstractA scalar magnetometer payload has been developed and integrated into a two‐man portable autonomous underwater vehicle (AUV) for geophysical and archeological surveys. The compact system collects data from a Geometrics microfabricated atomic magnetometer, a total‐field atomic magnetometer. Data from the sensor is both stored for post‐processing and made available to an onboard autonomy engine for real‐time sense and react behaviors. This system has been characterized both in controlled laboratory conditions and at sea to determine its performance limits. Methodologies for processing the magnetometer data to correct for interference and error introduced by the AUV platform were developed to improve sensing performance. When conducting seabed surveys, detection and characterization of targets of interest are performed in real‐time aboard the AUV. This system is used to drive both single‐ and multiple‐vehicle autonomous target reacquisition behaviors. The combination of on‐board target detection and autonomous reacquire capability is found to increase the effective survey coverage rate of the AUV‐based magnetic sensing system.

Long-Term Evolution of the Solar Corona Using PROBA2 Data

We use The Sun Watcher with Active Pixel System detector and Image Processing (SWAP) imager onboard the Project for Onboard Autonomy 2 (PROBA2) mission to study the evolution of large-scale EUV structures in the solar corona observed throughout Solar Cycle 24 (from 2010 to 2019). We discuss the evolution of the on-disk coronal features and at different heights above the solar surface based on EUV intensity changes. We also look at the evolution of the corona in equatorial and polar regions and compare them at different phases of the solar cycle, as well as with sunspot number evolution and with the PROBA2/Lyman-Alpha Radiometer (LYRA) signal. The main results are as follows: The three time series (SWAP on-disk average brightness, sunspot number and LYRA irradiance) are very well correlated, with correlation coefficients around 0.9. The average rotation rate of bright features at latitudes of +15, 0, and -15 degrees was around 15 degree/day throughout the period studied. A secondary peak in EUV averaged intensity at the Poles was observed on the descending phase of SC24. These peaks (at North and South poles respectively) seem to be associated with the start of the development of the (polar) coronal holes. Large-scale off-limb structures were visible from around March 2010 to around March 2016, meaning that they were absent at the minimum phase of solar activity. A fan at the North pole persisted for more than 11 Carrington rotations (February 2014 to March 2015), and it could be seen up to altitudes of 1.6 Rs.

MicroUSV: A low-cost platform for indoor marine swarm robotics research.

This article describes an open source Unmanned Surface Vehicle (USV) designed to operate in indoor laboratory environments. The microUSV is small (23cm long) and inexpensive (approximately $320 per unit for 10 vessels): an ideal hardware platform for algorithm validation and marine swarm robotics research. The primary design goal was to minimize the vehicle's size and cost while providing a stable and maneuverable platform with onboard autonomy. To that end the vehicle is built using 3D printed and off-the-shelf hobbyist electronic components and uses an overhead camera system to simulate sensor data to minimize the number of onboard sensors required. This article describes the context, design, and assembly procedures for a microUSV and demonstrates the platform's base-level functionality in the form of a waypoint following controller implementation for both single and multi-robot configurations.

Functional Phenology of a Texas Post Oak Savanna from a CHRIS PROBA Time Series

Remnant midwestern oak savannas in the USA have been altered by fire suppression and the encroachment of woody evergreen trees and shrubs. The Gus Engeling Wildlife Management Area (GEWMA) near Palestine, Texas represents a relatively intact southern example of thickening and evergreen encroachment in oak savannas. In this study, 18 images from the CHRIS/PROBA (Compact High-Resolution Imaging Spectrometer/Project for On-Board Autonomy) sensor were acquired between June 2009 and October 2010 and used to explore variation in canopy dynamics among deciduous and evergreen trees and shrubs, and savanna grassland in seasonal leaf-on and leaf-off conditions. Nadir CHRIS images from the 11 useable dates were processed to surface reflectance and a selection of vegetation indices (VIs) sensitive to pigments, photosynthetic efficiency, and canopy water content were calculated. An analysis of temporal VI phenology was undertaken using a fishnet polygon at 90 m resolution incorporating tree densities from a classified aerial photo and soil type polygons. The results showed that the major differences in spectral phenology were associated with deciduous tree density, the density of evergreen trees and shrubs—especially during deciduous leaf-off periods—broad vegetation types, and soil type interactions with elevation. The VIs were sensitive to high densities of evergreens during the leaf-off period and indicative of a photosynthetic advantage over deciduous trees. The largest differences in VI profiles were associated with high and low tree density, and soil types with the lowest and highest available soil water. The study showed how time series of hyperspectral data could be used to monitor the relative abundance and vigor of desirable and less desirable species in conservation lands.

MODIS and PROBA-V NDVI Products Differ when Compared with Observations from Phenological Towers at Four Tropical Dry Forests in the Americas

The Normalized Difference Vegetation Index (NDVI) is widely used to monitor vegetation phenology and productivity around the world. Over the last few decades, phenology monitoring at large scales has been possible due to the information and metrics derived from satellite sensors such as the Moderate Resolution Imaging Spectroradiometer (MODIS) or the Project for On-Board Autonomy–Vegetation (PROBA-V). However, due to their temporal and spatial resolution, adequate ground comparison is lacking. In this paper, we analyze how NDVI products from MODIS (Aqua and Terra) and PROBA-V predict vegetation phenology when compared with near-surface observations. We conduct this comparison at four tropical dry forests (TDFs) in the Americas. We undertake this study by comparing the following: (i) Dissimilarities of the standardized NDVI (NDVIS) using dynamic time warping, (ii) the differences of daily NDVIS between seasons and ENSO months using generalized linear models, and (iii) phenometrics derived from NDVI time series. Overall, our results suggest that NDVIS from satellite observations present DTW distances (dissimilarities) between 2.98 and 46.57 (18.91 ± 12.31) when compared with near-surface observations. Furthermore, NDVIS comparisons reveal that overall differences between satellite and near-surface observations are close to zero, but this tends to differ between seasons or when El Nino Southern Oscillation (ENSO) is present. Phenometrics comparisons show that metrics derived from satellite observations such as green-up, maturity, and start and end of the wet season strongly correlate with those from near-surface observations. In contrast, phenometrics that describe the day of the highest or lowest NDVI tend to be inconsistent with those from near-surface observations. All findings were observed independently of the NDVI source. Our results suggest that satellite-based NDVI products tend to be inconsistent descriptors of vegetation events on tropical deciduous forests in comparison with near-surface observations. These results reinforce the idea that satellite-based NDVI products should be used and interpreted with great caution and only in ecosystems with well-established knowledge of their vegetation phenology.

Enabling Technologies for Maritime Autonomous Surface Ships

This paper identifies key technological challenges associated with building and operating Maritime Autonomous Surface Ships (MASS) and describes how existing and emerging technologies from other domains can contribute to addressing these challenges. The objective is not to provide an exhaustive list of technologies that could be utilized in future MASS, but rather to consider a few key technologies, their limitations and how they can contribute to meeting requirements associated with designing, building and operating MASS. The paper focuses on two aspects of autonomous operations: 1) on-board autonomy; and 2) remote operations. The former considers primarily vessel safety and reliability concerns and the latter deals with issues related to the control of autonomous vessels at remote operations centres (ROC) using a telecommunications infrastructure.

A Combination of PROBA-V/MODIS-Based Products with Sentinel-1 SAR Data for Detecting Wet and Dry Snow Cover in Mountainous Areas

In the present study, we explore the value of employing both vegetation indexes as well as land surface temperature derived from Project for On-Board Autonomy—Vegetation (PROBA-V) and Moderate Resolution Imaging Spectroradiometer (MODIS) sensors, respectively, to support the detection of total (wet + dry) snow cover extent (SCE) based on a simple tuning machine learning approach and provide reliability maps for further analysis. We utilize Sentinel-1-based synthetic aperture radar (SAR) observations, including backscatter coefficient, interferometric coherence, and polarimetric parameters, and four topographical factors as well as vegetation and temperature information to detect the total SCE with a land cover-dependent random forest-based approach. Our results show that the overall accuracy and F-measure are over 90% with an ’Area Under the receiver operating characteristic Curve (ROC)’ (AUC) score of approximately 80% over five study areas located in different mountain ranges, continents, and hemispheres. These accuracies are also confirmed by a comprehensive validation approach with different data sources, attesting the robustness and global transferability. Additionally, based on the reliability maps, we find an inversely proportional relationship between classification reliability and vegetation density. In conclusion, comparing to a previous study only utilizing SAR-based observations, the method proposed in the present study provides a complementary approach to achieve a higher total SCE mapping accuracy while maintaining global applicability with reliable accuracy and corresponding uncertainty information.

A data-driven approximate dynamic programming approach based on association rule learning: Spacecraft autonomy as a case study

Dynamic programming (DP) and Markov Decision Process (MDP) offer powerful tools for formulating, modeling, and solving decision making problems under uncertainty. In real-world applications, the applicability of DP is limited by severe scalability issues. These issues can be addressed by Approximate Dynamic Programming (ADP) techniques. ADP methods are based on the assumption of having either a proper estimation of the underlying state transition probability distributions or a simulation mechanism with the capability of generating samples according to such probability distributions. In this paper, we present a data-driven ADP-based approach, which can offer an alternative in case such assumption cannot be guaranteed. In particular, when varying the set-up of the MDP state transition probability matrix, different policies can be calculated through exact DP or ADP methods. Such policies are then processed by an Apriori-based algorithm to find frequent association rules within them. A pruning procedure is used to select the most suitable association rules, and finally an Association Classifier infers the optimal policy in all the possible circumstances. We show a detailed application of the proposed approach for the calculation of a proper mission operations plan for spacecrafts with a high level of on-board autonomy.

A Survey on Model-Based Mission Planning and Execution for Autonomous Spacecraft

Different drivers are nowadays leading spacecraft toward an increased level of on-board autonomy. In this paper, we survey model-based techniques as a vehicle to implement highly autonomous on-board capabilities for spacecraft mission planning and execution. In this respect, spacecraft reconfiguration approaches based on Markovian Decision Process are explored, and then compared with other model-based alternatives. The integration of planning systems and dynamic reprogramming capabilities into the on-board software is presented. Finally, operational concepts for mission planning and execution in recent European space projects as well as the implementation of in-flight adaptive mission operations via on-board control procedures are also analyzed.

First Detection of Solar Flare Emission in Mid-ultraviolet Balmer Continuum

Abstract We present the first detection of solar flare emission at mid-ultraviolet wavelengths around 2000 Å by the channel 2 of the Large-Yield RAdiometer (LYRA) on board the PRoject for OnBoard Autonomy 2 mission. The flare (SOL20170906) was also observed in the channel 1 of LYRA centered at the H i Lyα line at 1216 Å, showing a clear non-thermal profile in both channels. The flare radiation in channel 2 is consistent with the hydrogen Balmer continuum emission produced by an optically thin chromospheric slab heated up to 10,000 K. Simultaneous observations in channels 1 and 2 allow the separation of the line emission (primarily from the Lyα line) from the Balmer continuum emission. Together with the recent detection of the Balmer continuum emission in the near-ultraviolet by the Interface Region Imaging Spectrometer, the LYRA observations strengthen the interpretation of broadband flare emission as the hydrogen recombination continua originating in the chromosphere.