Non-stationary Domains Research Articles

Constrained Meta-Reinforcement Learning for Adaptable Safety Guarantee with Differentiable Convex Programming

Despite remarkable achievements in artificial intelligence, the deployability of learning-enabled systems in high-stakes real-world environments still faces persistent challenges. For example, in safety-critical domains like autonomous driving, robotic manipulation, and healthcare, it is crucial not only to achieve high performance but also to comply with given constraints. Furthermore, adaptability becomes paramount in non-stationary domains, where environmental parameters are subject to change. While safety and adaptability are recognized as key qualities for the new generation of AI, current approaches have not demonstrated effective adaptable performance in constrained settings. Hence, this paper breaks new ground by studying the unique challenges of ensuring safety in nonstationary environments by solving constrained problems through the lens of the meta-learning approach (learning to learn). While unconstrained meta-learning already encounters complexities in end to end differentiation of the loss due to the bi-level nature, its constrained counterpart introduces an additional layer of difficulty, since the constraints imposed on task-level updates complicate the differentiation process. To address the issue, we first employ successive convex-constrained policy updates across multiple tasks with differentiable convex programming, which allows meta-learning in constrained scenarios by enabling end-to-end differentiation. This approach empowers the agent to rapidly adapt to new tasks under nonstationarity while ensuring compliance with safety constraints. We also provide a theoretical analysis demonstrating guaranteed monotonic improvement of our approach, justifying our algorithmic designs. Extensive simulations across diverse environments provide empirical validation with significant improvement over established benchmarks.

The bias caused by the String Effect in ordinary Kriging: risks and solutions

ABSTRACT Kriging relies on the assumption that the study area is within an infinite domain. In practice, Kriging is performed in a finite domain, generally defined by geology, and a string of samples along a drillhole is commonly truncated inside this domain. In the Kriging system, the string-end samples receive more weight as they are less screened. This feature is referred to as the ‘String Effect’ and becomes a concern in non-stationary domains in which the end and central samples are systematically different, such as (i) domains with grade enrichment or diminution towards the centre, or (ii) end samples are partially contaminated with material from adjacent domains due to sampling methods. Three methods available in the literature and one novel approach (Kriging with Adjusted Declustering [KAD]) are applied to continuous and categorical variables in three case studies. The KAD results in a significant improvement in the results of all case study models.

Answer set programming for non-stationary Markov decision processes

Non-stationary domains, where unforeseen changes happen, present a challenge for agents to find an optimal policy for a sequential decision making problem. This work investigates a solution to this problem that combines Markov Decision Processes (MDP) and Reinforcement Learning (RL) with Answer Set Programming (ASP) in a method we call ASP(RL). In this method, Answer Set Programming is used to find the possible trajectories of an MDP, from where Reinforcement Learning is applied to learn the optimal policy of the problem. Results show that ASP(RL) is capable of efficiently finding the optimal solution of an MDP representing non-stationary domains.

Learning Continuous Time Bayesian Networks in Non-stationary Domains

Non-stationary continuous time Bayesian networks are introduced. They allow the parents set of each node to change over continuous time. Three settings are developed for learning non-stationary continuous time Bayesian networks from data: known transition times, known number of epochs and unknown number of epochs. A score function for each setting is derived and the corresponding learning algorithm is developed. A set of numerical experiments on synthetic data is used to compare the effectiveness of non-stationary continuous time Bayesian networks to that of non-stationary dynamic Bayesian networks. Furthermore, the performance achieved by non-stationary continuous time Bayesian networks is compared to that achieved by state-of-the-art algorithms on four real-world datasets, namely drosophila, saccharomyces cerevisiae, songbird and macroeconomics.

An incremental learning algorithm based on the K-associated graph for non-stationary data classification

Non-stationary classification problems concern the changes on data distribution over a classifier lifetime. To face this problem, learning algorithms must conciliate essential, but difficult to gather, attributes like good classification performance, stability and low associated costs, like processing time and memory. This paper presents an extension of the K-associated optimal graph learning algorithm to cope with classification over non-stationary domains. The algorithm relies on a graph structure consisting of many disconnected components (subgraphs). Such graph enhances data representation by fitting locally groups of data according to a purity measure, which, in turn, quantifies the overlapping between vertices of different classes. As a result, the graph can be used to accurately estimate the probability of unlabeled data to belong to a given class. The proposed algorithm is benefited from the dynamical evolution of the graph by updating its set of components when new data is presented along time, by removing old components as new components arise. Experimental results on artificial and real domains and further statistical analysis show that the proposed algorithm is an effective solution to non-stationary classification problems.

A Computational Model of Cell Polarization and Motility Coupling Mechanics and Biochemistry

The motion of a eukaryotic cell presents a variety of interesting and challenging problems from both a modeling and a computational perspective. The processes span many spatial scales (from molecular to tissue) as well as disparate time scales, with reaction kinetics on the order of seconds, and the deformation and motion of the cell occurring on the order of minutes. The computational difficulty, even in 2D, resides in the fact that the problem is inherently one of deforming, non-stationary domains, bounded by an elastic perimeter, inside of which there is redistribution of biochemical signaling substances. Here we report the results of a computational scheme using the immersed boundary method to address this problem. We adopt a simple reaction-diffusion system that represents an internal regulatory mechanism controlling the polarization of a cell, and determining the strength of protrusion forces at the front of its elastic perimeter. Using this computational scheme we are able to study the effect of protrusive and elastic forces on cell shapes on their own, the distribution of the reaction-diffusion system in irregular domains on its own, and the coupled mechanical-chemical system. We find that this representation of cell crawling can recover important aspects of the spontaneous polarization and motion of certain types of crawling cells.

Trace expansions for elliptic cone operators with stationary domains

We analyze the behavior of the trace of the resolvent of an ellip- tic cone differential operator as the spectral parameter tends to infinity. The resolvent splits into two components, one associated with the minimal exten- sion of the operator, and another, of finite rank, depending on the particular choice of domain. We give a full asymptotic expansion of the first compo- nent and expand the component of finite rank in the case where the domain is stationary. The results make use, and develop further, our previous investi- gations on the analytic and geometric structure of the resolvent. The analysis of nonstationary domains, considerably more intricate, is pursued elsewhere.

Adapting Bayes network structures to non-stationary domains

When an incremental structural learning method gradually modifies a Bayesian network (BN) structure to fit a sequential stream of observations, we call the process structural adaptation. Structural adaptation is useful when the learner is set to work in an unknown environment, where a BN is gradually being constructed as observations of the environment are made. Existing algorithms for incremental learning assume that the samples in the database have been drawn from a single underlying distribution. In this paper we relax this assumption, so that the underlying distribution can change during the sampling of the database. The proposed method can thus be used in unknown environments, where it is not even known whether the dynamics of the environment are stable. We state formal correctness results for our method, and demonstrate its feasibility experimentally.

Modeling non‐Fickian transport in geological formations as a continuous time random walk

Non‐Fickian (or anomalous) transport of contaminants has been observed at field and laboratory scales in a wide variety of porous and fractured geological formations. Over many years a basic challenge to the hydrology community has been to develop a theoretical framework that quantitatively accounts for this widespread phenomenon. Recently, continuous time random walk (CTRW) formulations have been demonstrated to provide general and effective means to quantify non‐Fickian transport. We introduce and develop the CTRW framework from its conceptual picture of transport through its mathematical development to applications relevant to laboratory‐ and field‐scale systems. The CTRW approach contrasts with ones used extensively on the basis of the advection‐dispersion equation and use of upscaling, volume averaging, and homogenization. We examine the underlying assumptions, scope, and differences of these approaches, as well as stochastic formulations, relative to CTRW. We argue why these methods have not been successful in fitting actual measurements. The CTRW has now been developed within the framework of partial differential equations and has been generalized to apply to nonstationary domains and interactions with immobile states (matrix effects). We survey models based on multirate mass transfer (mobile‐immobile) and fractional derivatives and show their connection as subsets within the CTRW framework.

EVOLUTION OF COOPERATION IN A SPATIAL PRISONER'S DILEMMA

We investigate the spatial distribution and the global frequency of agents who can either cooperate or defect. The agent interaction is described by a deterministic, non-iterated prisoner's dilemma game, further each agent only locally interacts with his neighbors. Based on a detailed analysis of the local payoff structures we derive critical conditions for the invasion or the spatial coexistence of cooperators and defectors. These results are concluded in a phase diagram that allows us to identify five regimes, each characterized by a distinct spatiotemporal dynamics and a corresponding final spatial structure. In addition to the complete invasion of defectors, we find coexistence regimes with either a majority of cooperators in large spatial domains, or a minority of cooperators organized in small non-stationary domains or in small clusters. The analysis further allowed a verification of computer simulation results by Nowak and May (1993). Eventually, we present simulation results of a true 5-person game on a lattice. This modification leads to non-uniform spatial interactions that may even enhance the effect of cooperation.