Stochastic Path Research Articles

Mpemba-Like Sensory Withdrawal Effect

Biological sensors rely on the temporal dynamics of ligand concentration for signaling. The sensory performance is bounded by the distinguishability between the sensory state transition dynamics under different environmental protocols. This work presents a comprehensive theory to characterize arbitrary transient sensory dynamics of biological sensors. Here the sensory performance is quantified by the Kullback-Leibler (KL) divergence between the probability distributions of the sensor's stochastic paths. We introduce a novel benchmark to assess a sensor's transient sensory performance arbitrarily far from equilibrium. We identify a counterintuitive phenomenon in multistate sensors: while an initial exposure to high ligand concentration may hinder a sensor's sensitivity towards a future concentration up-shift, certain sensors may show a boost in sensitivity if the initial high concentration exposure is followed by a transient resetting at a low concentration environment. The boosted performance exceeds that of a sensor starting from an initially low concentration environment. This effect, reminiscent of a drug withdrawal effect, can be explained by the Markovian dynamics of the multistate sensor, similar to the Markovian Mpemba effect. Moreover, an exhaustive machine learning study of four-state sensors reveals a tight connection between the sensor's performance and the structure of the Markovian graph of its states. Published by the American Physical Society 2024

Volume Scattering Probability Guiding

Simulating the light transport of volumetric effects poses significant challenges and costs, especially in the presence of heterogeneous volumes. Generating stochastic paths for volume rendering involves multiple decisions, and previous works mainly focused on directional and distance sampling, where the volume scattering probability (VSP), i.e., the probability of scattering inside a volume, is indirectly determined as a byproduct of distance sampling. We demonstrate that direct control over the VSP can significantly improve efficiency and present an unbiased volume rendering algorithm based on an existing resampling framework for precise control over the VSP. Compared to previous state-of-the-art, which can only increase the VSP without guaranteeing to reach the desired value, our method also supports decreasing the VSP. We further present a data-driven guiding framework to efficiently learn and query an approximation of the optimal VSP everywhere in the scene without the need for user control. Our approach can easily be combined with existing path-guiding methods for directional sampling at minimal overhead and shows significant improvements over the state-of-the-art in various complex volumetric lighting scenarios.

Evaluation of transition rates from nonequilibrium instantons

Equilibrium rate theories play a crucial role in understanding rare, reactive events. However, they are inapplicable to a range of irreversible processes in systems driven far from thermodynamic equilibrium like active and biological matter. Here we develop an efficient numerical method to compute the rate constant of rare nonequilibrium events in the weak-noise limit based on an instanton approximation to the stochastic path integral and illustrate its wide range of application. We demonstrate excellent agreement of the instanton rates with numerically exact results for a particle under a nonconservative force. We also study phase transitions in an active field theory. We elucidate how activity alters the stability of the two phases and their rates of interconversion in a manner that can be well described by modifying classical nucleation theory. Published by the American Physical Society 2024

PCLOS based fractional-order sliding mode stochastic path following control for underactuated marine vehicles with multiple disturbances and constraints

This paper investigates the stochastic path following control of underactuated marine vehicles (UMVs) subject to multiple disturbances and constraints. Firstly, the complex marine environment in which UMVs navigate typically contains stochastic components, thus the multiple disturbances are categorized as slow-varying deterministic disturbances and stochastic disturbances. Secondly, a position-constrained line-of-sight (PCLOS) based fractional-order sliding mode stochastic (FSMS) control strategy is established to achieve path following control of UMVs. A PCLOS guidance law based on universal barrier Lyapunov function is proposed to ensure that the position errors remain within the constraint ranges, which is versatile for systems with symmetric constraints or without constraints. An FSMS controller based on fractional-order theory and sliding mode control is designed to improve the dynamic response speed of the system and effectively attenuate chattering phenomenon. A stochastic disturbance observer is developed to estimate the slow-varying deterministic disturbances in the stochastic system, and auxiliary dynamic compensators are used to mitigate the impact of input constraints. Lastly, theoretical analysis indicates that the closed-loop system is stable and the position constraint requirements are satisfied. Comparative simulations illustrate the effectiveness of the proposed control strategy.

Iterative algorithms for solving one-sided partially observable stochastic shortest path games

Real-world scenarios often involve dynamic interactions among competing agents, where decisions are made considering actions taken by others. These situations can be modeled as partially observable stochastic games (POSGs), with zero-sum variants capturing strictly competitive interactions (e.g., security scenarios). While such models address a broad range of problems, they commonly focus on infinite-horizon scenarios with discounted-sum objectives. Using the discounted-sum objective, however, can lead to suboptimal solutions in cases where the length of the interaction does not directly affect the gained rewards of the players.We thus focus on games with undiscounted objective and an indefinite horizon where every realization of the game is guaranteed to terminate after some unspecified number of turns. To manage the computational complexity of solving POSGs in general, we restrict to games with one-sided partial observability where only one player has imperfect information while their opponent is provided with full information about the current situation. We introduce two novel algorithms based on the heuristic search value iteration (HSVI) algorithm that iteratively solve sequences of easier-to-solve approximations of the game using fundamentally different approaches for constructing the sequences: (1) in GoalHorizon, the game approximations are based on a limited number of turns in which players can change their actions, (2) in GoalDiscount, the game approximations are constructed using an increasing discount factor. We provide theoretical qualitative guarantees for algorithms, and we also experimentally demonstrate that these algorithms are able to find near-optimal solutions on pursuit-evasion games and a game modeling privilege escalation problem from computer security.

From efficiency to equity: A multi-user paradigm in mobile route optimization

This study addresses the challenge of optimizing vehicle mobility in urban environments, which is significant for the advancement of smart city initiatives and spatial data analysis. We introduce a novel mobile recommendation system designed for multi-user scenarios, aiming to achieve a balance between effectiveness and fairness. The system prioritizes maximizing the profitability of vehicle service providers while ensuring an equitable distribution of recommended routes among users. Our approach features a redefined objective function that integrates a fairness criterion alongside path quality optimization. We further propose PSA-DLMA (Parallel Simulated Annealing with Deep Learning-Guided Move Adaptation), a stochastic path search method that leverages deep learning to guide move and strategy selection, alongside a dynamic termination mechanism and a parallel processing strategy. We validate our methodology using recent yellow taxi data from New York City and its surroundings, conducting comprehensive experiments to assess the performance of the system. The results demonstrate the superiority of PSA-DLMA over existing state-of-the-art solutions, offering significant contributions to improving urban vehicle mobility within the smart city framework.

Stochastic LPV MPC-based path following control for bevel-tip flexible needle with probabilistic constraints

This paper addresses the path-tracking problem for flexible needle control systems using a stochastic linear parameter varying (LPV) and model predictive control (MPC) strategy. Flexible needles operating in dynamic environments with non-uniform tissue density often deviate from ideal assumptions, resulting in non-standard models. The bicycle kinematics model for flexible needle motion control is transformed into an LPV model, improving accuracy and enabling more efficient control. The proposed stochastic LPV MPC approach aims to mitigate uncertainties arising from modelling errors and dynamic environmental factors, ensuring accurate trajectory tracking for the flexible needle. The sample and removal method is utilized to reformulate the probabilistic-constrained optimization problem for implementation. The contributions of this work lie in the application of stochastic LPV MPC to address the trajectory tracking problem in the presence of uncertainties. The simulation results illustrate the superior robustness of the stochastic LPV MPC approach, as evidenced by significantly smaller tracking errors across various scenarios.

Symmetry and Historicity in the Game of Life: Modifying Rules to Simulate Evolutionary Dynamics

The Game of Life (GoL) is a paradigmatic computer simulation that exhibits the emergence of complex properties of the whole from relatively simple sets of heuristic rules operating at lower organizational levels. Therefore, it is widely understood as a valuable tool for investigating global properties of evolutionary processes. Cognizant of refined concepts that emerged in recent debates on the central role of historical contingency (historicity) in evolution, we modify the original GoL rules by introducing an updating feedback loop and a probability factor that reflect the degree of historicity in pattern evolution as both stochastic path dependence and sensitivity to initial conditions. We examine this trait in simulations of the emergence and breaking of bilateral and radial symmetries commonly observed in the evolution of life, most prominently as evolving body plans. We show that the implementation of historicity parameters leads to a more realistic sequential and gradual alternating emergence and the breaking of new symmetries than the original set of rules. Apart from its more realistic representations of evolutionary processes, the new approach allows for easier exploration of the parameter space, demonstrates the sequential and gradual emergence and breaking of symmetries, and provides a publicly available and modular simulation tool. We discuss the exploratory potential of the modified GoL platform we designed as an extended thought experiment to study the parameter space defining the historicity of biological processes.

Evolutionary paths toward multi-level convergence of lactic acid bacteria in fructose-rich environments

Convergence provides clues to unveil the non-random nature of evolution. Intermediate paths toward convergence inform us of the stochasticity and the constraint of evolutionary processes. Although previous studies have suggested that substantial constraints exist in microevolutionary paths, it remains unclear whether macroevolutionary convergence follows stochastic or constrained paths. Here, we performed comparative genomics for hundreds of lactic acid bacteria (LAB) species, including clades showing a convergent gene repertoire and sharing fructose-rich habitats. By adopting phylogenetic comparative methods we showed that the genomic convergence of distinct fructophilic LAB (FLAB) lineages was caused by parallel losses of more than a hundred orthologs and the gene losses followed significantly similar orders. Our results further suggested that the loss of adhE, a key gene for phenotypic convergence to FLAB, follows a specific evolutionary path of domain architecture decay and amino acid substitutions in multiple LAB lineages sharing fructose-rich habitats. These findings unveiled the constrained evolutionary paths toward the convergence of free-living bacterial clades at the genomic and molecular levels.

Action formalism for geometric phases from self-closing quantum trajectories

When subject to measurements, quantum systems evolve along stochastic quantum trajectories that can be naturally equipped with a geometric phase observable via a post-selection in a final projective measurement. When post-selecting the trajectories to form a close loop, the geometric phase undergoes a topological transition driven by the measurement strength. Here, we study the geometric phase of a subset of self-closing trajectories induced by a continuous Gaussian measurement of a single qubit system. We utilize a stochastic path integral that enables the analysis of rare self-closing events using action methods and develop the formalism to incorporate the measurement-induced geometric phase therein. We show that the geometric phase of the most likely trajectories undergoes a topological transition for self-closing trajectories as a function of the measurement strength parameter. Moreover, the inclusion of Gaussian corrections in the vicinity of the most probable self-closing trajectory quantitatively changes the transition point in agreement with results from numerical simulations of the full set of quantum trajectories.

RealSinger: Ultra-realistic singing voice generation via stochastic differential equations

Synthesizing high-quality singing voice from music score is a challenging problem in music generation and has many practical applications. Samples generated by existing singing voice synthesis (SVS) systems can roughly reflect the lyrics, pitch and duration in a given score, but they fail to contain necessary details. In this paper, based on stochastic differential equations (SDE) we propose RealSinger to generate 22.05kHz ultra-realistic singing voice conditioned on a music score. Our RealSinger learns to find the stochastic process path from a source of white noise to the target singing voice manifold under the conditional music score, allowing to sing the music score while maintaining the local voice details of the target singer. During training, our model learns to accurately predict the direction of movement in the ambient Euclidean space onto the low-dimensional singing voice manifold. RealSinger’s framework is very flexible. It can either generate intermediate feature representations of the singing voice, such as mel-spectrogram, or directly generate the final waveform, as in the end-to-end style which rectify defects and accumulation errors introduced by two-stage connected singing synthesis systems. An extensive subjective and objective test on benchmark datasets shows significant gains in perceptual quality using RealSinger. The mean opinion scores (MOS) obtained with RealSinger are closer to those of the human singer’s original high-fidelity singing voice than to those obtained with any state-of-the-art method. Audio samples are available at https://realsinger.github.io/.

Intelligent Agents and Causal Inference: Enhancing Decision-Making through Causal Reasoning

This study examines the incorporation of causal inference methods into intelligent entities and examines the benefits of utilizing causal reasoning to improve decision-making procedures. This study entails conducting an experimental evaluation within a video game setting to evaluate the performance of three separate agent types: ExplorerBOT, GuardBOT, and CausalBOT. The ExplorerBOT utilizes a stochastic path selection technique for task completion, whereas the GuardBOT remains immobile yet exhibits exceptional proficiency in identifying and neutralizing other bots. On the other hand, the CausalBOT utilizes sophisticated causal inference methods to examine the underlying factors contributing to the failures noticed in the task completion of the ExplorerBOT. The aforementioned feature allows CausalBOT to make informed decisions by selecting paths that have a greater likelihood of achieving success. The main purpose of these experiments is to assess and compare the effectiveness of two distinct bots, namely ExplorerBOT and CausalBOT, in accomplishing their respective objectives. To facilitate comparison, two iterations of the ExplorerBOT are utilized. The initial iteration is predicated exclusively on stochastic path selection and necessitates a more profound understanding of the variables that impact the achievement of tasks. On the other hand, the second version integrates an algorithm for informed search. In contrast, CausalBOT employs causal inference techniques to discover the underlying causes of failures exhibited by ExplorerBOTs and collect pertinent data. Through the process of discerning the fundamental causal mechanisms, CausalBOT is able to make well-informed decisions by selecting pathways that maximize the probability of successfully completing a given job. The utilization of this approach greatly boosts the decision-making powers of CausalBOT, hence enabling it to effectively adapt and overcome problems in a more efficient manner when compared to alternative agents.

Stochastic ray tracing for Fresnel diffraction

We propose stochastic ray tracing for laser beam propagation in Fresnel diffraction to find the duality between wave and ray representations. We transform from the Maxwell equations to the Schrödinger equation for a monochromatic laser beam in the slowly varying envelope approximation. The stochastic ray tracing method interprets this Schrödinger equation as a stochastic process, of an analogy of Nelson’s stochastic mechanics. It can illustrate the stochastic paths and the wavefront of an optical beam. This ray tracing method includes Fresnel diffraction effects naturally. We show its general theoretical construction and numerical tests for a Gaussian laser beam with diffraction, that stochasticity realizes the beam waist around the Rayleigh range.

Stochastic path power and the Laplace transform

Transition probabilities for stochastic systems can be expressed in terms of a functional integral over paths taken by the system. Approximately evaluating this integral by the saddle point method in the weak-noise limit leads to a remarkable mapping between dominant stochastic paths through the potential V and conservative, Hamiltonian mechanics with an effective potential −|∇V|2 . The conserved ‘energy’ in this effective system has dimensions of power. We show that this power, H, can be identified with the Laplace parameter of the time-transformed dynamics. As H → 0, corresponding to the long-time limit, the equilibrium Boltzmann density is recovered. However, keeping H finite leads to insights into the non-equilibrium behaviour of the system. Moreover, it facilitates the explicit summation over families of trajectories, which is far harder in the time domain, and turns out to be essential for making contact with the long-time limit in some cases. We illustrate the validity of these results using simple examples that can be explicitly solved by other means.

Signature-based validation of real-world economic scenarios

AbstractMotivated by insurance applications, we propose a new approach for the validation of real-world economic scenarios. This approach is based on the statistical test developed by Chevyrev and Oberhauser ((2022)Journal of Machine Learning Research,23(176), 1–42.) and relies on the notions of signature and maximum mean distance. This test allows to check whether two samples of stochastic processes paths come from the same distribution. Our contribution is to apply this test to a variety of stochastic processes exhibiting different pathwise properties (Hölder regularity, autocorrelation, and regime switches) and which are relevant for the modelling of stock prices and stock volatility as well as of inflation in view of actuarial applications.

Stochastic Shortest Path Problem on Borel Space Considering Dead-ends and Undesired Terminal States

Stochastic Shortest Path Problem on Borel Space Considering Dead-ends and Undesired Terminal States

Characterization of Gauss–Markov stochastic sequences for mission analysis

In real scenarios, the spacecraft deviates from the intended paths owing to uncertainties in dynamics, navigation, and command actuation. Accurately quantifying these uncertainties is crucial for assessing the observability, collision risks, and mission viability. This issue is further magnified for CubeSats because they have limited control authority and thus require accurate dispersion estimates to avoid rejecting viable trajectories or selecting unviable ones. Trajectory uncertainties arise from random variables (e.g., measurement errors and drag coefficients) and processes (e.g., solar radiation pressure and low-thrust acceleration). Although random variables generally present minimal computational complexity, handling stochastic processes can be challenging because of their noisy dynamics. Nonetheless, accurately modeling these processes is essential, as they significantly influence the uncertain propagation of space trajectories, and an inadequate representation can result in either underestimation or overestimation of the stochastic characteristics associated with a given trajectory. This study addresses the gap in characterizing process uncertainties, represented as Gauss–Markov processes in mission analysis, by presenting models, evaluating derived quantities, and providing results on the impact of spacecraft trajectories. This study emphasizes the importance of accurately modeling random processes to properly characterize stochastic spacecraft paths.

Minimizing File Transfer Time in Opportunistic Spectrum Access Model

We study the file transfer problem in opportunistic spectrum access (OSA) model, which has been widely studied in throughput-oriented applications for max-throughput strategies and in delay-related works that commonly assume identical channel rates and fixed file sizes. Our work explicitly considers minimizing the file transfer time for a given file in a set of heterogeneous-rate Bernoulli channels, showing that max-throughput policy doesn't minimize file transfer time in general. We formulate a mathematical framework for static extend to dynamic policies by mapping our file transfer problem to a stochastic shortest path problem. We analyze the performance of our proposed static and dynamic optimal policies over the max-throughput policy. We propose a mixed-integer programming formulation as an efficient alternative way to obtain the dynamic optimal policy and show a huge reduction in computation time. Then, we propose a heuristic policy that takes into account the performance-complexity tradeoff and consider the online implementation with unknown channel parameters. Furthermore, we present numerical simulations to support our analytical results and discuss the effect of switching delay on different policies. Finally, we extend the file transfer problem to Markovian channels and demonstrate the impact of the correlation of each channel.

Simulating animal space use from fitted integrated Step‐Selection Functions (iSSF)

Abstract A standing challenge in the study of animal movement ecology is the capacity to predict where and when an individual animal might occur on the landscape, the so‐called, utilisation distribution (UD). Under certain assumptions, the steady‐state UD can be predicted from a fitted exponential habitat selection function. However, these assumptions are rarely met. Furthermore, there are many applications that require the estimation of transient dynamics rather than steady‐state UDs (e.g. when modelling migration or dispersal). Thus, there is a clear need for computational tools capable of predicting UDs based on observed animal movement data. Integrated Step‐Selection Analyses (iSSAs), which integrates movement of the animal into habitat selection analyses, are widely used to study habitat selection and movement of wild animals, and result in a fully parametrised individual‐based model of animal movement, which we refer to as an integrated Step Selection Function (iSSF). An iSSF can be used to generate stochastic animal paths based on random draws from a series of Markovian redistribution kernels, each consisting of a selection‐free, but possibly habitat‐influenced, movement kernel and a movement‐free selection function. The UD can be approximated by a sufficiently large set of such stochastic paths. Here, we present a set of functions in R to facilitate the simulation of animal space use from fitted iSSFs. Our goal is to provide a general purpose simulator that is easy to use and is part of an existing workflow for iSSAs (within the amt R package). We demonstrate through a series of applications how the simulator can be used to address a variety of questions in applied movement ecology. By providing functions in amt and coded examples, we hope to encourage ecologists using iSSFs to explore their predictions and model goodness‐of‐fit using simulations, and to further explore mechanistic approaches to modelling landscape connectivity.

Dynamic User-Scheduling and Power Allocation for SWIPT Aided Federated Learning: A Deep Learning Approach

Federated learning (FL) has been considered as a promising paradigm for enabling distributed machine learning (ML) in wireless networks. To address the limited energy capacity of wireless devices, we propose a simultaneous wireless information and power transfer (SWIPT) aided FL, in which one FL server (FLS) co-located at a cellular base station (BS) uses SWIPT to simultaneously broadcast the global model to wireless user-devices (UDs) and provide wireless power transfer to them. The UDs then use the harvested energy to train their local models and further transmit the local models to the FLS for aggregation. To improve the spectrum efficiency, we consider that the UDs form a non-orthogonal multiple access (NOMA) group for simultaneously sending their local models over the same spectrum channel. Taking the UDs' time-varying available energy and channel conditions into account, we propose a dynamic optimization of the UDs-scheduling, the BS's transmit-power allocation, and the UDs' power-splitting factors for SWIPT, with the objective of minimizing the long-term energy consumption while ensuring the FL convergence. The optimization problem, however, is challenging to solve since it is a finite-horizon dynamic programming problem but with an unknown stopping time, and moreover, the action space covers both discrete and continuous variables. To address these difficulties, we first execute a series of equivalent transformations to reduce the number of decision variables and then formulate the problem as a stochastic shortest path problem, based on which we propose an actor-critic deep reinforcement learning algorithm with the proximal policy optimization to efficiently learn the policy that dynamically adjusts the UDs-scheduling for FL as well as the BS's transmit-power for SWIPT. Numerical results validate the effectiveness and performance of our proposed algorithm. The results demonstrate that our proposed algorithm can effectively reduce the long-term energy consumption in comparison with two baseline algorithms.