Transition Function Research Articles

Testing MOND on Small Bodies in the Remote Solar System

Modified Newtonian dynamics (MOND), which postulates a breakdown of Newton's laws of gravity/dynamics below some critical acceleration threshold, can explain many otherwise puzzling observational phenomena on galactic scales. MOND competes with the hypothesis of dark matter, which successfully explains the cosmic microwave background and large-scale structure. Here we provide the first solar system test of MOND that probes the subcritical acceleration regime. Using the Bekenstein–Milgrom “aquadratic Lagrangian” (or AQUAL) formulation, we simulate the evolution of myriads of test particles (planetesimals or comets) born in the trans-Neptunian region and scattered by the giant planets over the lifetime of the Sun to heliocentric distances of 102–105 au. We include the effects of the Galactic tidal field and passing stars. While Newtonian simulations reproduce the distribution of binding energies of long-period and Oort-cloud comets detectable from Earth, MOND-based simulations do not. This conclusion is robust to plausible changes in the migration history of the planets, the migration history of the Sun, the MOND transition function, effects of the Sun's birth cluster, and the fading properties of long-period comets. For the most popular version of AQUAL, characterized by a gradual transition between the Newtonian and MOND regimes, our MOND-based simulations also fail to reproduce the orbital distribution of trans-Neptunian objects in the detached disk (perihelion q > 38 au). Our results do not rule out some MOND theories more elaborate than AQUAL, in which non-Newtonian effects are screened on small spatial scales, at small masses, or in external gravitational fields comparable in strength to the critical acceleration.

Enteric nervous system regeneration and functional cure of experimental digestive Chagas disease with trypanocidal chemotherapy

Digestive Chagas disease (DCD) is an enteric neuropathy caused by Trypanosoma cruzi infection. There is a lack of evidence on the mechanism of pathogenesis and rationales for treatment. We used a female C3H/HeN mouse model that recapitulates key clinical manifestations to study how infection dynamics shape DCD pathology and the impact of treatment with the front-line, anti-parasitic drug benznidazole. Curative treatment 6 weeks post-infection resulted in sustained recovery of gastrointestinal transit function, whereas treatment failure led to infection relapse and gradual return of DCD symptoms. Neuro/immune gene expression patterns shifted from chronic inflammation to a tissue repair profile after cure, accompanied by increased cellular proliferation, glial cell marker expression and recovery of neuronal density in the myenteric plexus. Delaying treatment until 24 weeks post-infection led to partial reversal of DCD, suggesting the accumulation of permanent tissue damage over the course of chronic infection. Our study shows that murine DCD pathogenesis is sustained by chronic T. cruzi infection and is not an inevitable consequence of acute stage denervation. The risk of irreversible enteric neuromuscular tissue damage and dysfunction developing highlights the importance of prompt diagnosis and treatment. These findings support the concept of treating asymptomatic, T. cruzi-infected individuals with benznidazole to prevent DCD development.

Evolutionary Probabilistic Analysis of Non-Smoothly Damped Stochastic Oscillator Under Modulated Random and Harmonic Excitations

This paper concerns the evolutionary transitional probability density functions (PDFs) and the mean upcrossing rates (MCRs) of responses of the non-smoothly damped stochastic oscillator under both modulated random and harmonic excitations. The damping expressed by a non-smooth function of velocity is considered in the stochastic dynamical systems. A procedure is presented by making the time-related weighted residue of the formulated Fokker–Planck–Kolmogorov (FPK) equation vanish in the weak sense. Thus, a group of stochastic ordinary differential equations regarding the undetermined parameters in the assumed solution can be formulated and solved numerically. Three cases are studied under different conditions. The achieved evolutionary transitional PDF and MCR solutions attest to the outstanding effectiveness of the presented procedure even in the tail ends of the PDF solutions. Moreover, the achieved solutions of the stochastic dynamical systems are found to have significant non-Gaussian behaviors due to the interaction of the random and harmonic excitations and the system nonlinearity. Notably, the presented procedure greatly improves the running efficiency when compared with Monte Carlo simulation.

Model-Free Nonstationary Reinforcement Learning: Near-Optimal Regret and Applications in Multiagent Reinforcement Learning and Inventory Control

We consider model-free reinforcement learning (RL) in nonstationary Markov decision processes. Both the reward functions and the state transition functions are allowed to vary arbitrarily over time as long as their cumulative variations do not exceed certain variation budgets. We propose Restarted Q-Learning with Upper Confidence Bounds (RestartQ-UCB), the first model-free algorithm for nonstationary RL, and show that it outperforms existing solutions in terms of dynamic regret. Specifically, RestartQ-UCB with Freedman-type bonus terms achieves a dynamic regret bound of [Formula: see text], where S and A are the numbers of states and actions, respectively, [Formula: see text] is the variation budget, H is the number of time steps per episode, and T is the total number of time steps. We further present a parameter-free algorithm named Double-Restart Q-UCB that does not require prior knowledge of the variation budget. We show that our algorithms are nearly optimal by establishing an information-theoretical lower bound of [Formula: see text], the first lower bound in nonstationary RL. Numerical experiments validate the advantages of RestartQ-UCB in terms of both cumulative rewards and computational efficiency. We demonstrate the power of our results in examples of multiagent RL and inventory control across related products. This paper was accepted by Omar Besbes, revenue management and market analytics. Funding: The research of D. Simchi-Levi and R. Zhu was supported by the MIT Data Science Laboratory. The research of W. Mao, K. Zhang, and T. Başar was supported in part by the U.S. Army Research Laboratory (ARL) Cooperative Agreement W911NF-17-2-0196, in part by the Office of Naval Research (ONR) [MURI Grant N00014-16-1-2710], and in part by the Air Force Office of Scientific Research (AFOSR) [Grant FA9550-19-1-0353]. K. Zhang also acknowledges support from U.S. Army Research Laboratory (ARL) [Grant W911NF-24-1-0085]. Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2022.02533 .

Unraveling two yellow fluorescence emission mechanisms and color-changing afterglow characteristics of Dy-doped Mg2SnO4 luminescent materials

In this work, Dy ion doped Mg2SnO4 (Mg2SnO4:Dy3+) with monochromatic yellow fluorescence and color-changing long afterglow properties are prepared by the high-temperature solid-phase method. Mg2SnO4:Dy3+ shows the best luminescence performance under the preparation conditions of 1550 °C and 0.02 Dy3+ concentration. The samples are able to achieve yellow fluorescence emission (λem = 575 nm) under 350 nm and 291 nm excitation, and the emission intensity of the samples under 350 nm excitation is much higher than that of 291 nm excitation. Under 254 nm excitation, the sample not only emits bright white fluorescence but also has the ability of obvious afterglow after excitation. Moreover, the afterglow also has the color change function of transition from bright white to yellow. The luminescence mechanism study shows that Mg2SnO4:Dy3+ contains two emission centers, Dy3+ and lattice defects. Combining the results of spectroscopic measurements with the Dy3+ energy level distribution data, two different fluorescence emission processes under 291 and 350 nm excitation as well as the afterglow discoloration mechanism due to 254 nm excitation are discussed in detail.

Sparse reconstruction in spin systems. I: iid spins

For a sequence of Boolean functions fn:{−1,1}Vn→{−1,1}\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${f_n}:{\\{ - 1,1\\} ^{{V_n}}} \ o \\{ - 1,1\\} $$\\end{document}, defined on increasing configuration spaces of random inputs, we say that there is sparse reconstruction if there is a sequence of subsets Un ⊆ Vn of the coordinates satisfying ∣Un∣ = o(∣Vn∣) such that knowing the coordinates in Un gives us a non-vanishing amount of information about the value of fn.We first show that, if the underlying measure is a product measure, then no sparse reconstruction is possible for any sequence of transitive functions. We discuss the question in different frameworks, measuring information content in L2 and with entropy. We also highlight some interesting connections with cooperative game theory. Beyond transitive functions, we show that the left-right crossing event for critical planar percolation on the square lattice does not admit sparse reconstruction either. Some of these results answer questions posed by Itai Benjamini.

Existence and uniqueness of quasi-stationary and quasi-ergodic measures for absorbing Markov chains: A Banach lattice approach

We establish the existence and uniqueness of quasi-stationary and quasi-ergodic measures for almost surely absorbed discrete-time Markov chains under weak conditions. We obtain our results by exploiting Banach lattice properties of transition functions under natural regularity assumptions.

Analytical approach on stress analysis in a functionally graded piezoelectric annular disk with varying compressibility and varying density under internal pressure

When a crystal is subjected to mechanical stress, it exhibits piezoelectricity, also known as the piezoelectric effect, which is the emergence of an electrical potential across the crystal's sides. A few examples of piezoelectric materials include quartz, Rochelle salt, lead zirconate titanate (PZT4) and zinc oxide (ZnO). Piezoelectricity has many applications in regards to electrical transducers and signal devices. In this paper, the analytical solution is obtained for elastic-plastic stresses in a thin rotating disc formed of functionally graded piezoelectric material with varying compressibility and varying density for distinct angular velocity with internal pressure using Seth's transition theory. The distribution of the mechanical and electrical components can be precisely controlled by applying functionally graded piezoelectric materials (FGPMs). Transition theory assumes that a mid-zone (transition state) exists between elastic state and plastic state. A nonlinear Governing equation of the problem is formulated using equilibrium equation by substituting the values of stresses. For evaluation of transitional stresses, transition function R is assumed in terms of radial stresses and equations are solved analytically by considering the transition (critical points) of the differential equation. The investigation focuses on how the rotational velocity and internal pressure affects the radial and circumferential distribution of stresses. With the help of graphs and mathematical calculations, it is observed that both variable density and variable compressibility have significant effect on the performance of a functionally graded piezoelectric materials. Therefore, it can be concluded that an annular disc made up of functionally graded piezoelectric materials is safer than that of homogeneous piezoelectric materials for engineering design purposes.

Aero-engine Degradation Evaluation Based on First-Order Transition Function and Gaussian Mapping Networks

Aero-engine Degradation Evaluation Based on First-Order Transition Function and Gaussian Mapping Networks

New Fascisms and the Crises of Empire: Lessons from the Americas

The 1970s witnessed debates about new fascistic forms of capitalist counterrevolution among militant intellectuals across the world. This article briefly maps some of the Latin American debates over the analytical and political pertinence of the category of fascism as applied to the military dictatorships of that time. It explores competing ideas of dependent fascism, fascism as project, and the counterinsurgent state, while homing in on two crucial lessons of the debate: (1) the centrality accorded to fascism as a historically and geographically specific product of imperialist crises, and (2) the notion that the fascization of the state may serve a transitional function in accelerating and consolidating processes of capitalist restructuring.

Current understanding of the pathogenesis of placenta accreta spectrum disorder with focus on mitochondrial function.

The refinement of assisted reproductive technology, including the development of cryopreservation techniques (vitrification) and ovarian stimulation protocols, makes frozen embryo transfer (FET) an alternative to fresh ET and has contributed to the success of assisted reproductive technology. Compared with fresh ET cycles, FET cycles were associated with better in vitro fertilization outcomes; however, the occurrence of pregnancy-induced hypertension, preeclampsia, and placenta accreta spectrum (PAS) was higher in FET cycles. PAS has been increasing steadily in incidence as a life-threatening condition along with cesarean rates worldwide. In this review, we summarize the current understanding of the pathogenesis of PAS and discuss future research directions. A literature search was performed in the PubMed and Google Scholar databases. Risk factors associated with PAS incidence include a primary defect of the decidua basalis or scar dehiscence, aberrant vascular remodeling, and abnormally invasive trophoblasts, or a combination thereof. Freezing, thawing, and hormone replacement manipulations have been shown to affect multiple cellular pathways, including cell proliferation, invasion, epithelial-to-mesenchymal transition (EMT), and mitochondrial function. Molecules involved in abnormal migration and EMT of extravillous trophoblast cells are beginning to be identified in PAS placentas. Many of these molecules were also found to be involved in mitochondrial biogenesis and dynamics. The etiology of PAS may be a multifactorial genesis with intrinsic predisposition (e.g., placental abnormalities) and certain environmental factors (e.g., defective decidua) as triggers for its development. A distinctive feature of this review is its focus on the potential factors linking mitochondrial function to PAS development.

Dynamic Tangled Derivative Logic of Metric Spaces

Dynamical systems are abstract models of interaction between space and time. They are often used in fields such as physics and engineering to understand complex processes, but due to their general nature, they have found applications for studying computational processes, interaction in multi-agent systems, machine learning algorithms and other computer science related phenomena. In the vast majority of applications, a dynamical system consists of the action of a continuous `transition function' on a metric space. In this work, we consider decidable formal systems for reasoning about such structures. Spatial logics can be traced back to the 1940's, but our work follows a more dynamic turn that these logics have taken due to two recent developments: the study of the topological mu-calculus, and the the integration of linear temporal logic with logics based on the Cantor derivative. In this paper, we combine dynamic topological logics based on the Cantor derivative and the `next point in time' operators with an expressively complete fixed point operator to produce a combination of the topological mu-calculus with linear temporal logic. We show that the resulting logics are decidable and have a natural axiomatisation. Moreover, we prove that these logics are complete for interpretations on the Cantor space, the rational numbers, and subspaces thereof.

Omega-Regular Decision Processes

Regular decision processes (RDPs) are a subclass of non-Markovian decision processes where the transition and reward functions are guarded by some regular property of the past (a lookback). While RDPs enable intuitive and succinct representation of non-Markovian decision processes, their expressive power coincides with finite-state Markov decision processes (MDPs). We introduce omega-regular decision processes (ODPs) where the non-Markovian aspect of the transition and reward functions are extended to an omega-regular lookahead over the system evolution. Semantically, these lookaheads can be considered as promises made by the decision maker or the learning agent about her future behavior. In particular, we assume that, if the promised lookaheads are not met, then the payoff to the decision maker is falsum (least desirable payoff), overriding any rewards collected by the decision maker. We enable optimization and learning for ODPs under the discounted-reward objective by reducing them to lexicographic optimization and learning over finite MDPs. We present experimental results demonstrating the effectiveness of the proposed reduction.

Online Markov Decision Processes Configuration with Continuous Decision Space

In this paper, we investigate the optimal online configuration of episodic Markov decision processes when the space of the possible configurations is continuous. Specifically, we study the interaction between a learner (referred to as the configurator) and an agent with a fixed, unknown policy, when the learner aims to minimize her losses by choosing transition functions in online fashion. The losses may be unrelated to the agent's rewards. This problem applies to many real-world scenarios where the learner seeks to manipulate the Markov decision process to her advantage. We study both deterministic and stochastic settings, where the losses are either fixed or sampled from an unknown probability distribution. We design two algorithms whose peculiarity is to rely on occupancy measures to explore with optimism the continuous space of transition functions, achieving constant regret in deterministic settings and sublinear regret in stochastic settings, respectively. Moreover, we prove that the regret bound is tight with respect to any constant factor in deterministic settings. Finally, we compare the empiric performance of our algorithms with a baseline in synthetic experiments.

Imagine, Initialize, and Explore: An Effective Exploration Method in Multi-Agent Reinforcement Learning

Effective exploration is crucial to discovering optimal strategies for multi-agent reinforcement learning (MARL) in complex coordination tasks. Existing methods mainly utilize intrinsic rewards to enable committed exploration or use role-based learning for decomposing joint action spaces instead of directly conducting a collective search in the entire action-observation space. However, they often face challenges obtaining specific joint action sequences to reach successful states in long-horizon tasks. To address this limitation, we propose Imagine, Initialize, and Explore (IIE), a novel method that offers a promising solution for efficient multi-agent exploration in complex scenarios. IIE employs a transformer model to imagine how the agents reach a critical state that can influence each other's transition functions. Then, we initialize the environment at this state using a simulator before the exploration phase. We formulate the imagination as a sequence modeling problem, where the states, observations, prompts, actions, and rewards are predicted autoregressively. The prompt consists of timestep-to-go, return-to-go, influence value, and one-shot demonstration, specifying the desired state and trajectory as well as guiding the action generation. By initializing agents at the critical states, IIE significantly increases the likelihood of discovering potentially important under-explored regions. Despite its simplicity, empirical results demonstrate that our method outperforms multi-agent exploration baselines on the StarCraft Multi-Agent Challenge (SMAC) and SMACv2 environments. Particularly, IIE shows improved performance in the sparse-reward SMAC tasks and produces more effective curricula over the initialized states than other generative methods, such as CVAE-GAN and diffusion models.

Dynamic Regret of Adversarial MDPs with Unknown Transition and Linear Function Approximation

We study reinforcement learning (RL) in episodic MDPs with adversarial full-information losses and the unknown transition. Instead of the classical static regret, we adopt dynamic regret as the performance measure which benchmarks the learner's performance with changing policies, making it more suitable for non-stationary environments. The primary challenge is to handle the uncertainties of unknown transition and unknown non-stationarity of environments simultaneously. We propose a general framework to decouple the two sources of uncertainties and show the dynamic regret bound naturally decomposes into two terms, one due to constructing confidence sets to handle the unknown transition and the other due to choosing sub-optimal policies under the unknown non-stationarity. To this end, we first employ the two-layer online ensemble structure to handle the adaptation error due to the unknown non-stationarity, which is model-agnostic. Subsequently, we instantiate the framework to three fundamental MDP models, including tabular MDPs, linear MDPs and linear mixture MDPs, and present corresponding approaches to control the exploration error due to the unknown transition. We provide dynamic regret guarantees respectively and show they are optimal in terms of the number of episodes K and the non-stationarity P̄ᴋ by establishing matching lower bounds. To the best of our knowledge, this is the first work that achieves the dynamic regret exhibiting optimal dependence on K and P̄ᴋ without prior knowledge about the non-stationarity for adversarial MDPs with unknown transition.

Abstract and Explore: A Novel Behavioral Metric with Cyclic Dynamics in Reinforcement Learning

Intrinsic motivation lies at the heart of the exploration of reinforcement learning, which is primarily driven by the agent's inherent satisfaction rather than external feedback from the environment. However, in recent more challenging procedurally-generated environments with high stochasticity and uninformative extrinsic rewards, we identify two significant issues of applying intrinsic motivation. (1) State representation collapse: In existing methods, the learned representations within intrinsic motivation have a high probability to neglect the distinction among different states and be distracted by the task-irrelevant information brought by the stochasticity. (2) Insufficient interrelation among dynamics: Unsuccessful guidance provided by the uninformative extrinsic reward makes the dynamics learning in intrinsic motivation less effective. In light of the above observations, a novel Behavioral metric with Cyclic Dynamics (BCD) is proposed, which considers both cumulative and immediate effects and facilitates the abstraction and exploration of the agent. For the behavioral metric, the successor feature is utilized to reveal the expected future rewards and alleviate the heavy reliance of previous methods on extrinsic rewards. Moreover, the latent variable and vector quantization techniques are employed to enable an accurate measurement of the transition function in a discrete and interpretable manner. In addition, cyclic dynamics is established to capture the interrelations between state and action, thereby providing a thorough awareness of environmental dynamics. Extensive experiments conducted on procedurally-generated environments demonstrate the state-of-the-art performance of our proposed BCD.

Long-Term Safe Reinforcement Learning with Binary Feedback

Safety is an indispensable requirement for applying reinforcement learning (RL) to real problems. Although there has been a surge of safe RL algorithms proposed in recent years, most existing work typically 1) relies on receiving numeric safety feedback; 2) does not guarantee safety during the learning process; 3) limits the problem to a priori known, deterministic transition dynamics; and/or 4) assume the existence of a known safe policy for any states. Addressing the issues mentioned above, we thus propose Long-term Binary-feedback Safe RL (LoBiSaRL), a safe RL algorithm for constrained Markov decision processes (CMDPs) with binary safety feedback and an unknown, stochastic state transition function. LoBiSaRL optimizes a policy to maximize rewards while guaranteeing long-term safety that an agent executes only safe state-action pairs throughout each episode with high probability. Specifically, LoBiSaRL models the binary safety function via a generalized linear model (GLM) and conservatively takes only a safe action at every time step while inferring its effect on future safety under proper assumptions. Our theoretical results show that LoBiSaRL guarantees the long-term safety constraint, with high probability. Finally, our empirical results demonstrate that our algorithm is safer than existing methods without significantly compromising performance in terms of reward.

Robust Active Measuring under Model Uncertainty

Partial observability and uncertainty are common problems in sequential decision-making that particularly impede the use of formal models such as Markov decision processes (MDPs). However, in practice, agents may be able to employ costly sensors to measure their environment and resolve partial observability by gathering information. Moreover, imprecise transition functions can capture model uncertainty. We combine these concepts and extend MDPs to robust active-measuring MDPs (RAM-MDPs). We present an active-measure heuristic to solve RAM-MDPs efficiently and show that model uncertainty can, counterintuitively, let agents take fewer measurements. We propose a method to counteract this behavior while only incurring a bounded additional cost. We empirically compare our methods to several baselines and show their superior scalability and performance.

Regular decision processes

We introduce and study Regular Decision Processes (RDPs), a new, compact model for domains with non-Markovian dynamics and rewards, in which the dependence on the past is regular, in the language theoretic sense. RDPs are an intermediate model between MDPs and POMDPs. They generalize k-order MDPs and can be viewed as a POMDP in which the hidden state is a regular function of the entire history. In factored RDPs, transition and reward functions are specified using formulas in linear temporal logics over finite traces, or using regular expressions. This allows specifying complex dependence on the past using intuitive and compact formulas, and building models of partially observable domains without specifying an underlying state space.