Model-free Setting Research Articles

Risk measures based on weak optimal transport

In this paper, we study convex risk measures with weak optimal transport penalties. In a first step, we show that these risk measures allow for an explicit representation via a nonlinear transform of the loss function. In a second step, we discuss computational aspects related to the nonlinear transform as well as approximations of the risk measures using, for example, neural networks. Our setup comprises a variety of examples, such as classical optimal transport penalties, parametric families of models, divergence risk measures, uncertainty on path spaces, moment constraints, and martingale constraints. In a last step, we show how to use the theoretical results for the numerical computation of worst-case losses in an insurance context and no-arbitrage prices of European contingent claims after quoted maturities in a model-free setting.

Test and Measure for Partial Mean Dependence Based on Machine Learning Methods

It is of importance to investigate the significance of a subset of covariates W for the response Y given covariates Z in regression modeling. To this end, we propose a significance test for the partial mean independence problem based on machine learning methods and data splitting. The test statistic converges to the standard Chi-squared distribution under the null hypothesis while it converges to a normal distribution under the fixed alternative hypothesis. Power enhancement and algorithm stability are also discussed. If the null hypothesis is rejected, we propose a partial Generalized Measure of Correlation (pGMC) to measure the partial mean dependence of Y given W after controlling for the nonlinear effect of Z. We present the appealing theoretical properties of the pGMC and establish the asymptotic normality of its estimator with the optimal root-N convergence rate. Furthermore, the valid confidence interval for the pGMC is also derived. As an important special case when there are no conditional covariates Z, we introduce a new test of overall significance of covariates for the response in a model-free setting. Numerical studies and real data analysis are also conducted to compare with existing approaches and to demonstrate the validity and flexibility of our proposed procedures. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

A zero-estimator approach for estimating the signal level in a high-dimensional model-free setting

We study a high-dimensional regression setting under the assumption of known covariate distribution. We aim at estimating the amount of explained variation in the response by the best linear function of the covariates (the signal level). In our setting, neither sparsity of the coefficient vector, nor normality of the covariates or linearity of the conditional expectation are assumed. We present an unbiased and consistent estimator and then improve it by using a zero-estimator approach, where a zero-estimator is a statistic whose expected value is zero. More generally, we present an algorithm based on the zero estimator approach that in principle can improve any given estimator. We study some asymptotic properties of the proposed estimators and demonstrate their finite sample performance in a simulation study.

UAV Multi-Dynamic Target Interception: A Hybrid Intelligent Method Using Deep Reinforcement Learning and Fuzzy Logic

With the rapid development of Artificial Intelligence, AI-enabled Uncrewed Aerial Vehicles have garnered extensive attention since they offer an accessible and cost-effective solution for executing tasks in unknown or complex environments. However, developing secure and effective AI-based algorithms that empower agents to learn, adapt, and make precise decisions in dynamic situations continues to be an intriguing area of study. This paper proposes a hybrid intelligent control framework that integrates an enhanced Soft Actor–Critic method with a fuzzy inference system, incorporating pre-defined expert experience to streamline the learning process. Additionally, several practical algorithms and approaches within this control system are developed. With the synergy of these innovations, the proposed method achieves effective real-time path planning in unpredictable environments under a model-free setting. Crucially, it addresses two significant challenges in RL: dynamic-environment problems and multi-target problems. Diverse scenarios incorporating actual UAV dynamics were designed and simulated to validate the performance in tracking multiple mobile intruder aircraft. A comprehensive analysis and comparison of methods relying solely on RL and other influencing factors, as well as a controller feasibility assessment for real-world flight tests, are conducted, highlighting the advantages of the proposed hybrid architecture. Overall, this research advances the development of AI-driven approaches for UAV safe autonomous navigation under demanding airspace conditions and provides a viable learning-based control solution for different types of robots.

Reset-free data-driven gain estimation: Power iteration using reversed-circulant matrices

A direct data-driven iterative algorithm is developed to accurately estimate the H∞ norm of a linear time-invariant system from continuous operation, i.e., without resetting the system. The main technical step involves a reversed-circulant matrix that can be evaluated in a model-free setting by performing experiments on the real system.

Structured Output Feedback Control for Linear Quadratic Regulator Using Policy Gradient Method

We consider the static output feedback control for Linear Quadratic Regulator problems with structured constraints under the assumption that system parameters are unknown. To solve the problem in the model free setting, we propose the policy gradient algorithm based on the gradient projection method and show its global convergence to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\varepsilon$</tex-math></inline-formula> -stationary points. In addition, we introduce a variance reduction technique and show both theoretically and numerically that it significantly reduces the variance in the gradient estimation. We also show in the numerical experiments that the model free approach efficiently solves the problem.

Benchmarks for the benchmark approach to valuing long-term insurance liabilities: comment on Fergusson &amp; Platen (2023)

AbstractThis article comments on the paper “Less-expensive long-term annuities linked to mortality, cash and equity” by Kevin Fergusson and Eckard Platen, appearing in this issue of theAnnals of Actuarial Science. It adds two perspectives to their thought-provoking contribution. The first is a similarity to some recent work in quantitative finance on “deep hedging” that leverages machine learning models to find the cheapest replication strategy for a derivative payoff in a largely model-free setting. The second perspective engages with some of the interesting implications of their approach and draws parallels to literature in asset pricing and macro-finance. These perspectives point to the potential need for more fundamental shifts than the authors of the paper are advertising.

Revisiting PGD Attacks for Stability Analysis of High-Dimensional Nonlinear Systems and Perception-Based Control

Many existing region-of-attraction (ROA) analysis tools find difficulty in addressing feedback systems with large-scale neural network (NN) policies and/or high-dimensional sensing modalities such as cameras. In this letter, we tailor the projected gradient descent (PGD) attack method as a general-purpose ROA analysis tool for high-dimensional nonlinear systems and end-to-end perception-based control. We show that the ROA analysis can be approximated as a constrained maximization problem such that PGD-based iterative methods can be directly applied. In the model-based setting, we show that the PGD updates can be efficiently performed using back-propagation. In the model-free setting, we propose a finite-difference PGD estimate which is general and only requires a black-box simulator for generating the trajectories of the closed-loop system given any initial state. Finally, we demonstrate the scalability and generality of our analysis tool on several numerical examples with large state dimensions or complex image observations.

Policy Gradient Learning Methods for Stochastic Control with Exit Time and Applications to Share Repurchase Pricing

ABSTRACT We develop policy gradients methods for stochastic control with exit time in a model-free setting. We propose two types of algorithms for learning either directly the optimal policy or by learning alternately the value function (critic) and the optimal control (actor). The use of randomized policies is crucial for overcoming notably the issue related to the exit time in the gradient computation. We demonstrate the effectiveness of our approach by implementing our numerical schemes in the application to the problem of share repurchase pricing. Our results show that the proposed policy gradient methods outperform PDE or other neural networks techniques in a model-based setting. Furthermore, our algorithms are flexible enough to incorporate realistic market conditions like, e.g., price impact or transaction costs.

Model-free prediction test with application to genomics data

Testing the significance of predictors in a regression model is one of the most important topics in statistics. This problem is especially difficult without any parametric assumptions on the data. This paper aims to test the null hypothesis that given confounding variables Z, X does not significantly contribute to the prediction of Y under the model-free setting, where X and Z are possibly high dimensional. We propose a general framework that first fits nonparametric machine learning regression algorithms on [Formula: see text] and [Formula: see text], then compares the prediction power of the two models. The proposed method allows us to leverage the strength of the most powerful regression algorithms developed in the modern machine learning community. The P value for the test can be easily obtained by permutation. In simulations, we find that the proposed method is more powerful compared to existing methods. The proposed method allows us to draw biologically meaningful conclusions from two gene expression data analyses without strong distributional assumptions: 1) testing the prediction power of sequencing RNA for the proteins in cellular indexing of transcriptomes and epitopes by sequencing data and 2) identification of spatially variable genes in spatially resolved transcriptomics data.

Decentralized safe reinforcement learning for inverter-based voltage control

Inverter-based distributed energy resources provide the possibility for fast time-scale voltage control by quickly adjusting their reactive power. The power-electronic interfaces allow these resources to realize almost arbitrary control law, but designing these decentralized controllers is nontrivial. Reinforcement learning (RL) approaches are becoming increasingly popular to search for policy parameterized by neural networks. It is difficult, however, to enforce that the learned controllers are safe, in the sense that they may introduce instabilities into the system.This paper proposes a safe learning approach for voltage control. We prove that the system is guaranteed to be exponentially stable if each controller satisfies certain Lipschitz constraints. The set of Lipschitz bound is optimized to enlarge the search space for neural network controllers. We explicitly engineer the structure of neural network controllers such that they satisfy the Lipschitz constraints by design. A decentralized RL framework is constructed to train local neural network controller at each bus in a model-free setting.

Testing the Effects of High-Dimensional Covariates via Aggregating Cumulative Covariances

In this article, we test for the effects of high-dimensional covariates on the response. In many applications, different components of covariates usually exhibit various levels of variation, which is ubiquitous in high-dimensional data. To simultaneously accommodate such heteroscedasticity and high dimensionality, we propose a novel test based on an aggregation of the marginal cumulative covariances, requiring no prior information on the specific form of regression models. Our proposed test statistic is scale-invariance, tuning-free and convenient to implement. The asymptotic normality of the proposed statistic is established under the null hypothesis. We further study the asymptotic relative efficiency of our proposed test with respect to the state-of-art universal tests in two different settings: one is designed for high-dimensional linear model and the other is introduced in a completely model-free setting. A remarkable finding reveals that, thanks to the scale-invariance property, even under the high-dimensional linear models, our proposed test is asymptotically much more powerful than existing competitors for the covariates with heterogeneous variances while maintaining high efficiency for the homoscedastic ones. Supplementary materials for this article are available online.

Data-Driven Optimal Control of Affine Systems: A Linear Programming Perspective

In this letter, we discuss the problem of optimal control for affine systems in the context of data-driven linear programming. First, we introduce a unified framework for the fixed point characterization of the value function, Q-function and relaxed Bellman operators. Then, in a model-free setting, we show how to synthesize and estimate Bellman inequalities from a small but sufficiently rich dataset. To guarantee exploration richness, we complete the extension of Willems&#x2019; fundamental lemma to affine systems.

Proper closed-loop specifications for data-driven model-reference control

In control applications where finding a model of the plant is costly and time consuming, direct data-driven approaches represent a valid alternative for the design of model reference controllers. However, the selection of a proper reference model within a model-free setting is known to be a critical task, as such a model typically plays the role of a hyperparameter. In this work, we extend the existing theory so as to compute both a reference model and the corresponding optimal controller parameters from data to satisfy given behavioral bounds on the desired closed-loop performance. The effectiveness of the proposed approach is illustrated on a benchmark simulation example.

Revisiting the predictive power of kernel principal components

In this short note, recent results on the predictive power of kernel principal component in a regression setting are extended in two ways: (1) in the model-free setting, we relax a conditional independence model assumption to obtain a stronger result; and (2) the model-free setting is also extended in the infinite-dimensional setting.

Data-Driven Attack Detection for Linear Systems

This letter studies the attack detection problem in a data-driven and model-free setting, for deterministic systems with linear and time-invariant dynamics. Differently from existing studies that leverage knowledge of the system dynamics to derive security bounds and monitoring schemes, we treat the cases where the system dynamics, as well as the attack strategy and attack location, are unknown. We derive fundamental security limitations as a function of only the observed data and without estimating the system dynamics (in fact, no assumption is made on the identifiability of the system). In particular, (i) we derive detection limitations as a function of the informativity and length of the observation window, (ii) provide a data-driven characterization of undetectable attacks, and (iii) construct a data-driven detection monitor. Surprisingly, our results show that while data-driven monitoring requires a larger observation window to attain attack detection capability, once attained it shares the same limitations as model-based monitoring.

Resampling-based confidence intervals for model-free robust inference on optimal treatment regimes.

We propose a new procedure for inference on optimal treatment regimes in the model-free setting, which does not require to specify an outcome regression model. Existing model-free estimators for optimal treatment regimes are usually not suitable for the purpose of inference, because they either have nonstandard asymptotic distributions or do not necessarily guarantee consistent estimation of the parameter indexing the Bayes rule due to the use of surrogate loss. We first study a smoothed robust estimator that directly targets the parameter corresponding to the Bayes decision rule for optimal treatment regimes estimation. This estimator is shown to have an asymptotic normal distribution. Furthermore, we verify that a resampling procedure provides asymptotically accurate inference for both the parameter indexing the optimal treatment regime and the optimal value function. A new algorithm is developed to calculate the proposed estimator with substantially improved speed and stability. Numerical results demonstrate the satisfactory performance of the newmethods.

Numerical Method for Model-free Pricing of Exotic Derivatives in Discrete Time Using Rough Path Signatures

ABSTRACT We estimate prices of exotic options in a discrete-time model-free setting when the trader has access to market prices of a rich enough class of exotic and vanilla options. This is achieved by estimating an unobservable quantity called ‘implied expected signature’ from such market prices, which are used to price other exotic derivatives. The implied expected signature is an object that characterizes the market dynamics.

Optimal Multi-Step-Ahead Prediction of ARCH/GARCH Models and NoVaS Transformation

This paper gives a computer‘intensive approach to multi-step-ahead prediction of volatility in financial returns series under an ARCH/GARCH model and also under a model-free setting, namely employing the NoVaS transformation. Our model-based approach only assumes i . i . d innovations without requiring knowledge/assumption of the error distribution and is computationally straightforward. The model-free approach is formally quite similar, albeit a GARCH model is not assumed. We conducted a number of simulations to show that the proposed approach works well for both point prediction (under L 1 and/or L 2 measures) and prediction intervals that were constructed using bootstrapping. The performance of GARCH models and the model-free approach for multi-step ahead prediction was also compared under different data generating processes.

Sevan methodologies revisited: Random line processes

The paper studies random line processes Z that are translation invariant in probability distribution, and whose first and second order moment measures possess continuous densities. The purpose is to review the analytical apparatus based on the concept of horizontal or vertical windows and corresponding Palm-type probability distributions that are now proved below to exist. That apparatus enables to study the relation between the quantities p k (l, α) and π k (l, α), where p k (l, α) = probability to have k hits by the lines from Z on a “test segment” of length l and direction α, while π k (l, α) = conditional probability of the same event, the condition being that the test segment lies on one of the lines from Z. Palm equations for horizontal windows have been known since long, but for vertical windows they were first put down in the last chapter of the book [4], under stronger condition of Euclideanmotions invariance of Z. The paper considers two different models that imply Poissonity of the probabilities p k (l, α). Translation invariant line processes can be viewed as stationary states of random dynamical arrays of countably many particles each moving with constant speed along the test line, and these models are of special interest in that context. In a model-free setting, the paper presents a formula for calculation of the conditional intensity Λ(α) = l −1Σ k kπ k (l, α). That formula includes quantities depending on the distribution of the typical vertex shape. “Sevan metodologies” have been the topic of authors plenary report at the Rasht (Iran) meeting in 2011. This usage is motivated in a special historical section below; another section is devoted to detailed description of Sevan methodologies themselves.