Infinite Setting Research Articles

Off-Policy Confidence Interval Estimation with Confounded Markov Decision Process

This article is concerned with constructing a confidence interval for a target policy’s value offline based on a pre-collected observational data in infinite horizon settings. Most of the existing works assume no unmeasured variables exist that confound the observed actions. This assumption, however, is likely to be violated in real applications such as healthcare and technological industries. In this article, we show that with some auxiliary variables that mediate the effect of actions on the system dynamics, the target policy’s value is identifiable in a confounded Markov decision process. Based on this result, we develop an efficient off-policy value estimator that is robust to potential model misspecification and provide rigorous uncertainty quantification. Our method is justified by theoretical results, simulated and real datasets obtained from ridesharing companies. A Python implementation of the proposed procedure is available at https://github.com/Mamba413/cope.

Distributionally robust modeling of optimal control

The aim of this paper is to formulate several questions related to distributionally robust Stochastic Optimal Control modeling. As an example the distributionally robust counterpart of the classical inventory model is discussed in details. Finite and infinite horizon stationary settings are considered.

Robust Portfolio Choice with Sticky Wages

We present a robust version of the life-cycle optimal portfolio choice problem in the presence of labor income, as introduced in Biffis, Gozzi, and Prosdocimi [SIAM J. Control Optim., 58 (2020), pp. 1906--1938] and Dybvig and Liu [J. Econom. Theory, 145 (2010), pp. 885--907]. In particular, in Biffis, Gozzi, and Prosdocimi the influence of past wages on the future ones is modeled linearly in the evolution equation of labor income, through a given weight function. The optimization relies on the resolution of an infinite dimensional HJB equation. We improve the state of art in three ways. First, we allow the weight to be a Radon measure. This accommodates for more realistic weighting of the sticky wages, e.g., on a discrete temporal grid according to some periodic income. Second, there is a general correlation structure between labor income and stock markets. This naturally affects the optimal hedging demand, which may increase or decrease according to the correlation sign. Third, we allow the weight to change with time, possibly lacking perfect identification. The uncertainty is specified by a given set of Radon measures K, in which the weight process takes values. This renders the inevitable uncertainty on how the past affects the future, and includes the standard case of error bounds on a specific estimate for the weight. Under uncertainty averse preferences, the decision maker takes a maxmin approach to the problem. Our analysis confirms the intuition: in the infinite dimensional setting, the optimal policy remains the best investment strategy under the worst case weight.

Korovkin-type theorems and local approximation problems

Of concern are local approximation problems for sequences of positive linear operators acting on linear subspaces of functions defined on a metric space. A Korovkin-type theorem is established in such a framework together with several consequences related to one dimensional, multidimensional and infinite dimensional settings (Hilbert spaces).Furthermore, some applications are discussed which concern classical sequences of positive linear operators including (one dimensional and multidimensional) Bernstein operators, Kantorovich operators, Szász–Mirakyan operators, Gauss–Weierstrass operators and Bernstein–Schnabl operators on convex subsets of Hilbert spaces.Finally the paper ends with a reassessment of a result of Korovkin concerning subspaces of bounded 2π− periodic functions on R and with an application related to sequences of convolution operators generated by positive approximate identities.

Two Singularity Subtraction Schemes for a Class of Nonlinear Weakly Singular Integral Equations

Singularity subtraction for linear weakly singular Fredholm integral equations of the second kind is generalized to nonlinear integral equations. Two approaches are presented: The Classical Approach discretizes the nonlinear problem, and uses some finite dimensional linearization process to solve numerically the discrete problem. Its convergence is proved under mild hypotheses on the nonlinearity and the quadrature rule of the singularity subtraction scheme. The New Approach is based on linearization of the problem in its infinite dimensional setting, and discretization of the sequence of linear problems by singularity subtraction. It is more efficient than the former, as three numerical experiments confirm.

Design-based estimators of the distribution function in ranked set sampling with an application

Empirical distribution functions (EDFs) based on ranked set sampling (RSS) and its modifications have been examined by many authors. In these studies, the proposed estimators have been investigated for infinite population settings. However, developing EDF estimators in finite population settings would be more valuable for areas such as environmental, ecological, agricultural, biological, medical, etc. This paper introduces new EDF estimators based on level-0, level-1 and level-2 sampling designs in RSS. Asymptotic properties of the new EDF estimators have been established. Numerical results have been obtained for the case when ranking is imperfect under different distribution functions. It has been observed that level-2 sampling design provides a more efficient EDF estimator than its counterparts of level-0, level-1 and simple random sampling. In real data application, we consider a pointwise estimate of distribution function and estimation of the median of sheep's weights at 7 months using RSS based on level-2 sampling design.

Determination of evaporation rate of warm water placed inside a partially-filled top cooled enclosure

Evaporation is an important step in clean water production technologies such as solar desalination. Here, a warm body of impure water evaporates in a differentially heated cavity, the vapour condenses over a cooler surface and the condensate is collected as salt-free water. While buoyancy-driven convection is set-up in air, the water body is subjected to evaporative cooling and buoyancy-driven flow is setup in the water volume as well. Evaporation-induced buoyant flow in water is imaged in the present study using a Mach-Zehnder interferometer in the infinite and wedge fringe settings. The corresponding interfacial heat flux, that includes the contribution from evaporation, is evaluated by analyzing the interferograms. From a modeling perspective, the evaporation rate is the central quantity of importance. It is determined in the present study using an ab initio model for an air-water system that solves mass, momentum, energy, and moisture transport equations with convective flow and jump conditions in the heat flux at the interface. The instantaneous mass and energy fluxes thus determined have also been compared with the kinetic theory model and Dunkle's correlation that is frequently adopted in the design of solar stills. Among the three models, the ab initio approach shows close agreement with experiments. The match with Dunkle's correlation is less satisfactory. The ab initio model closely follows the flow transitions in the air-phase and provides a justification for a good match with the experimentally derived instantaneous evaporative heat and mass fluxes. The ab initio approach is further adopted to parametrically study the effect of initial water temperature, top surface temperature and water depth on the evaporation rate.

Existence of doubly strong equilibria in generalized games and quasi-Ky Fan minimax inequalities

Using new sufficient conditions for the existence of solutions to quasi-Ky Fan minimax inequalities in infinite dimensional setting, we prove the existence of strategy profiles, called doubly strong equilibria, which maximize players' preferences in generalized games. Our setting allows discontinuous and non-ordered preferences. The doubly strong equilibrium is a refinement of the strong equilibrium by Aumann. We show that our conditions do not imply the previous ones that guarantee the existence of strong equilibria.

Stochastic extensions of symbols in Wiener spaces and heat operator

The construction, in [1], of a pseudodifferential calculus analogous to the Weyl calculus, in an infinite dimensional setting, required the introduction of convenient symbol classes. The symbols are functions defined on an infinite dimensional Hilbert space H, which compels us to investigate, in particular, their stochastic extensions to a Wiener space linked with H.

The Split and Approximate Split Property in 2D Systems: Stability and Absence of Superselection Sectors

The split property of a pure state for a certain cut of a quantum spin system can be understood as the entanglement between the two subsystems being weak. From this point of view, we may say that if it is not possible to transform a state omega via sufficiently local automorphisms (in a sense that we will make precise) into a state satisfying the split property, then the state omega has a long-range entanglement. It is well known that in 1D, gapped ground states have the split property with respect to cutting the system into left and right half-chains. In 2D, however, the split property fails to hold for interesting models such as Kitaev’s toric code. Here we show that this failure is the reason that anyons can exist in that model. There is a folklore saying that the existence of anyons, like in the toric code model, implies long-range entanglement of the state. In this paper, we prove this folklore in an infinite dimensional setting. More precisely, we show that long-range entanglement, in a way that we will define precisely, is a necessary condition to have non-trivial superselection sectors. Anyons in particular give rise to such non-trivial sectors. States with the split property for cones, on the other hand, do not admit non-trivial sectors. A key technical ingredient of our proof is that under suitable assumptions on locality, the automorphisms generated by local interactions can be “approximately factorized.” That is, they can be written as the tensor product of automorphisms localized in a cone and its complement respectively, followed by an automorphism acting near the “boundary” of Lambda , and conjugation with a unitary. This result may be of independent interest. This technique also allows us to prove that the approximate split property, a weaker version of the split property that is satisfied in e.g. the toric code, is stable under applying such automorphisms.

Studies enhancement of transient stability by single machine infinite bus system and setting purpose genetic algorithm

Maintaining network synchronization is important to customer service. Low fluctuations cause voltage instability, non-synchronization in the power system or the problems in the electrical system disturbances, harmonics current and voltages inflation and contraction voltage. Proper tunning of the parameters of stabilizer is prime for validation of stabilizer. To overcome instability issues and get reinforcement found a lot of the techniques are developed to overcome instability problems and improve performance of power system. Genetic algorithm was applied to optimize parameters and suppress oscillation. The simulation of the robust composite capacitance system of an infinite single-machine bus was studied using MATLAB was used for optimization purpose. The critical time is an indication of the maximum possible time during which the error can pass in the system to obtain stability through the simulation. The effectiveness improvement has been shown in the system.

Diffusion tensor regularization with metric double integrals

AbstractIn this paper, we propose a variational regularization method for denoising and inpainting of diffusion tensor magnetic resonance images. We consider these images as manifold-valued Sobolev functions, i.e. in an infinite dimensional setting, which are defined appropriately. The regularization functionals are defined as double integrals, which are equivalent to Sobolev semi-norms in the Euclidean setting. We extend the analysis of [14] concerning stability and convergence of the variational regularization methods by a uniqueness result, apply them to diffusion tensor processing, and validate our model in numerical examples with synthetic and real data.

The Spectra of Banach Algebras of Holomorphic Functions on Polydisk-Type Domains

Aron et al. (Math Ann 353:293–303, 2012) proved that the Cluster Value Theorem in the infinite dimensional Banach space setting holds for the Banach algebra $$\mathcal {H}^\infty (B_{c_0})$$ . On the other hand, Cole and Gamelin (Proc Lond Math Soc 53:112–142, 1986) showed that $$\mathcal {H}^\infty (\ell _2 \cap B_{c_0})$$ is isometrically isomorphic to $$\mathcal {H}^\infty (B_{c_0})$$ in the sense of an algebra. Motivated by this work, we are interested in a class of open subsets U of a Banach space X for which $$\mathcal {H}^\infty (U)$$ is isometrically isomorphic to $$\mathcal {H}^\infty (B_{c_0})$$ . We prove that there exist polydisk-type domains U of any infinite dimensional Banach space X with a Schauder basis such that $$\mathcal {H}^\infty (U)$$ is isometrically isomorphic to $$\mathcal {H}^\infty (B_{c_0})$$ , which generalizes the result by Cole and Gamelin (Proc Lond Math Soc 53:112–142, 1986). Furthermore, we study the analytic and algebraic structure of the spectrum of $$\mathcal {H}^\infty (U)$$ and show that the Cluster Value Theorem is true for $$\mathcal {H}^\infty (U)$$ .

An Augmented Lagrangian Deep Learning Method for Variational Problems with Essential Boundary Conditions

This paper is concerned with a novel deep learning method for variational problems with essential boundary conditions. To this end, we first reformulate the original problem into a minimax problem corresponding to a feasible augmented Lagrangian, which can be solved by the augmented Lagrangian method in an infinite dimensional setting. Based on this, by expressing the primal and dual variables with two individual deep neural network functions, we present an augmented Lagrangian deep learning method for which the parameters are trained by the stochastic optimization method together with a projection technique. Compared to the traditional penalty method, the new method admits two main advantages: i) the choice of the penalty parameter is flexible and robust, and ii) the numerical solution is more accurate in the same magnitude of computational cost. As typical applications, we apply the new approach to solve elliptic problems and (nonlinear) eigenvalue problems with essential boundary conditions, and numerical experiments are presented to show the effectiveness of the new method.

Incremental intervention effects in studies with dropout and many timepoints#

Abstract Modern longitudinal studies collect feature data at many timepoints, often of the same order of sample size. Such studies are typically affected by dropout and positivity violations. We tackle these problems by generalizing effects of recent incremental interventions (which shift propensity scores rather than set treatment values deterministically) to accommodate multiple outcomes and subject dropout. We give an identifying expression for incremental intervention effects when dropout is conditionally ignorable (without requiring treatment positivity) and derive the nonparametric efficiency bound for estimating such effects. Then we present efficient nonparametric estimators, showing that they converge at fast parametric rates and yield uniform inferential guarantees, even when nuisance functions are estimated flexibly at slower rates. We also study the variance ratio of incremental intervention effects relative to more conventional deterministic effects in a novel infinite time horizon setting, where the number of timepoints can grow with sample size and show that incremental intervention effects yield near-exponential gains in statistical precision in this setup. Finally, we conclude with simulations and apply our methods in a study of the effect of low-dose aspirin on pregnancy outcomes.

Envelope theorems for static optimization and Calculus of Variations

We establish differentiability properties of the value function of problems of Static Optimization in an abstract infinite dimensional setting and we apply that to problems of Calculus of Variations. We lighten the assumptions of existing results, notably by using Gâteaux and Hadamard differentials. Moreover we use recently established Multipliers Rules.

Statistical Inference of the Value Function for Reinforcement Learning in Infinite-Horizon Settings

AbstractReinforcement learning is a general technique that allows an agent to learn an optimal policy and interact with an environment in sequential decision-making problems. The goodness of a policy is measured by its value function starting from some initial state. The focus of this paper was to construct confidence intervals (CIs) for a policy’s value in infinite horizon settings where the number of decision points diverges to infinity. We propose to model the action-value state function (Q-function) associated with a policy based on series/sieve method to derive its confidence interval. When the target policy depends on the observed data as well, we propose a SequentiAl Value Evaluation (SAVE) method to recursively update the estimated policy and its value estimator. As long as either the number of trajectories or the number of decision points diverges to infinity, we show that the proposed CI achieves nominal coverage even in cases where the optimal policy is not unique. Simulation studies are conducted to back up our theoretical findings. We apply the proposed method to a dataset from mobile health studies and find that reinforcement learning algorithms could help improve patient’s health status. A Python implementation of the proposed procedure is available at https://github.com/shengzhang37/SAVE.

A novel interferometric method for simultaneous measurement of film thickness and film interface temperature for a horizontal tube falling film evaporator for MED systems

Horizontal tube falling film evaporation spans a lot of applications, especially in multi-effect desalination application. Hence, the improvement of its gain output ratio is of prime importance, and for that, the studies on the characteristics associated with the flow, mainly falling film thickness and falling film interface temperature, are inevitable. As the falling film flow is a delicate fluid structure, non-intrusive measurement techniques have to be preferred for the experiment. In this work, we introduce a novel interferometric method to calculate film interface temperature and a shadowgraph approach to estimate film thickness simultaneously. A Mach-Zehnder interferometer with infinite fringe setting was used to capture the thermal fluctuations around the horizontal tube falling film evaporator. Interferometric method implemented in this work also serves as an image visualization technique and does the purpose of qualitative as well as quantitative analysis. An in house designed experimental setup was fabricated for producing a thin and uniform falling film with a fully wetted flow over the horizontal copper tube. A heater controller setup was incorporated into it for accurate temperature control. This method holds the advantage of scanning falling film thickness over circumferential angle for the range 5∘≤θ≤175∘, which is an improvement to state of the art. Furthermore, dynamic film thickness and instantaneous film interface temperature studies were also incorporated to prove the efficacy of the technique and to test the robustness of the algorithms implemented. CFD analysis was also carried out for the entire process by means of VOF scheme and was found to be in good agreement with the experimental results, which proves our technique to be a good one. A standard error of mean analysis was performed on every data set, which validates the reproducibility and the dynamic measurement capability of the technique used.

Compressible gas density measurement by means of Fourier analysis of interferograms

This paper describes a method for nonintrusive compressible gas density measurement by means of automated analysis of interferograms using FFT (Fast Fourier Transform), and its implementation using DFT (Discrete Fourier Transform), that does make this measurement technique a fairly valuable and accessible experimental method. The presented approach makes it possible to use the finite fringe setting of the interferometer, thus reducing adjustment time. In the process of analysis, the errors originating from the imperfections of optical elements are reduced, and a digital image of a virtual infinite setting of the instrument is generated. As described here, the method is extremely beneficial, but not limited to, flows with shock wave – boundary layer interactions. A result of density measured for a flow with shock wave is presented.

Strategic behavior under tradeable driving permits and congestion tolls: A political economy model

This paper develops a political economy model to study strategic behavior related to the introduction of congestion policies. Our main focus is on tradeable permits, although we also study results under a congestion toll. We first study a two-period model where, in the first period, a majority decides whether or not to introduce a permit system in the next period. The model setup is such that tolls and permits can be made perfectly equivalent, both in terms of welfare effects and voting outcomes. We find that anticipatory behavior after the policy is announced but prior to its introduction induces people to drive more; the overall effect may be welfare reducing. Moreover, drivers oppose the policies even when they receive all permits for free, or toll revenues are distributed to drivers only. Consequently, strategic behavior makes it more difficult to get a political majority to support these congestion policies. We further show that, in an infinite horizon setting, tradable permits and congestion tolls are no longer equivalent. Permits are superior to congestion tolls in that they avoid strategic behavior once the system is implemented. In contrast, with congestion tolls the steady-state equilibrium implies continuing strategic behavior. One consequence is that it is easier to get a political majority for permits than for tolls: we find that drivers will always oppose congestion tolls, but they support permits if they receive a sufficient share of the permits for free.