Smoothness Assumptions Research Articles

Inexact SARAH algorithm for stochastic optimization

We develop and analyse a variant of the SARAH algorithm, which does not require computation of the exact gradient. Thus this new method can be applied to general expectation minimization problems rather than only finite sum problems. While the original SARAH algorithm, as well as its predecessor, SVRG, requires an exact gradient computation on each outer iteration, the inexact variant of SARAH (iSARAH), which we develop here, requires only stochastic gradient computed on a mini-batch of sufficient size. The proposed method combines variance reduction via sample size selection and iterative stochastic gradient updates. We analyse the convergence rate of the algorithms for strongly convex and non-strongly convex cases, under smooth assumption with appropriate mini-batch size selected for each case. We show that with an additional, reasonable, assumption iSARAH achieves the best-known complexity among stochastic methods in the case of non-strongly convex stochastic functions.

Optimal Leader–Follower Consensus for Constrained-Input Multiagent Systems With Completely Unknown Dynamics

In this article, we investigate a novel reinforcement-learning (RL)-based scheme to address the optimal leader–follower consensus issue for constrained-input continuous-time multiagent systems. First, as for input-constrained problems, the fundamental smooth assumption is not satisfied for the value function. To deal with this problem, we employ the method of vanishing viscosity solutions to relax this smooth assumption to a continuity assumption for the value function, which broadens the scope of RL applications. Second, the control cost functions take a more general form which guarantees the continuity of the optimal control policy instead of the specific integrand form used in previous input-constrained RL-based schemes. Based on these results, we introduce a novel identifier–critic–actor structure to extend the conventional critic–actor RL framework into a distributed model-free one, where the learning of the identifier, critic, and actor is online and simultaneous. We provide the simulation examples to validate the effectiveness of the proposed scheme.

On the generalized Burgers-Huxley equation: Existence, uniqueness, regularity, global attractors and numerical studies

<p style='text-indent:20px;'>In this work, we consider the forced generalized Burgers-Huxley equation and establish the existence and uniqueness of a global weak solution using a Faedo-Galerkin approximation method. Under smoothness assumptions on the initial data and external forcing, we also obtain further regularity results of weak solutions. Taking external forcing to be zero, a positivity result as well as a bound on the classical solution are also established. Furthermore, we examine the long-term behavior of solutions of the generalized Burgers-Huxley equations. We first establish the existence of absorbing balls in appropriate spaces for the semigroup associated with the solutions and then show the existence of a global attractor for the system. The inviscid limits of the Burgers-Huxley equations to the Burgers as well as Huxley equations are also discussed. Next, we consider the stationary Burgers-Huxley equation and establish the existence and uniqueness of weak solution by using a Faedo-Galerkin approximation technique and compactness arguments. Then, we discuss about the exponential stability of stationary solutions. Concerning numerical studies, we first derive error estimates for the semidiscrete Galerkin approximation. Finally, we present two computational examples to show the convergence numerically.

Biharmonic Problems and their Application in Engineering and Medicine

In the present paper we study some properties of solutions of the Steklov and Neumann boundary value problems for the biharmonic equation. For solving these biharmonic problems with application in engineering and medicine, we need to solve boundary value problems Dirichlet and Cauchy for the Poisson equation using the scattering model. In order to select suitable solutions, we solve the Poisson equation with the corresponding boundary conditions Dirichlet and Cauchy, that is, some criterion function is minimized in the Sobolev norms. Under appropriate smoothness assumptions, these problems may be reformulated as boundary value problems for the biharmonic equation.MSC: 35J15; 35J35; 35J40; 58J50; 58J90.

Stability of a Subcritical Fluid Model for Fair Bandwidth Sharing with General File Size Distributions

This work concerns the asymptotic behavior of solutions to a (strictly) subcritical fluid model for a data communication network, where file sizes are generally distributed and the network operates under a fair bandwidth-sharing policy. Here we consider fair bandwidth-sharing policies that are a slight generalization of the [Formula: see text]-fair policies introduced by Mo and Walrand [Mo J, Walrand J (2000) Fair end-to-end window-based congestion control. IEEE/ACM Trans. Networks 8(5):556–567.]. Since the year 2000, it has been a standing problem to prove stability of the data communications network model of Massoulié and Roberts [Massoulié L, Roberts J (2000) Bandwidth sharing and admission control for elastic traffic. Telecommunication Systems 15(1):185–201.], with general file sizes and operating under fair bandwidth sharing policies, when the offered load is less than capacity (subcritical conditions). A crucial step in an approach to this problem is to prove stability of subcritical fluid model solutions. In 2012, Paganini et al. [Paganini F, Tang A, Ferragut A, Andrew LLH (2012) Network stability under alpha fair bandwidth allocation with general file size distribution. IEEE Trans. Automatic Control 57(3):579–591.] introduced a Lyapunov function for this purpose and gave an argument, assuming that fluid model solutions are sufficiently smooth in time and space that they are strong solutions of a partial differential equation and assuming that no fluid level on any route touches zero before all route levels reach zero. The aim of the current paper is to prove stability of the subcritical fluid model without these strong assumptions. Starting with a slight generalization of the Lyapunov function proposed by Paganini et al., assuming that each component of the initial state of a measure-valued fluid model solution, as well as the file size distributions, have no atoms and have finite first moments, we prove absolute continuity in time of the composition of the Lyapunov function with any subcritical fluid model solution and describe the associated density. We use this to prove that the Lyapunov function composed with such a subcritical fluid model solution converges to zero as time goes to infinity. This implies that each component of the measure-valued fluid model solution converges vaguely on [Formula: see text] to the zero measure as time goes to infinity. Under the further assumption that the file size distributions have finite pth moments for some p &gt; 1 and that each component of the initial state of the fluid model solution has finite pth moment, it is proved that the fluid model solution reaches the measure with all components equal to the zero measure in finite time and that the time to reach this zero state has a uniform bound for all fluid model solutions having a uniform bound on the initial total mass and the pth moment of each component of the initial state. In contrast to the analysis of Paganini et al., we do not need their strong smoothness assumptions on fluid model solutions and we rigorously treat the realistic, but singular situation, where the fluid level on some routes becomes zero, whereas other route levels remain positive.

LTSA-LE: A local tangent space alignment label enhancement algorithm

According to smoothness assumption, local topological structure can be shared between feature and label manifolds. This study proposes a new algorithm based on Local Tangent Space Alignment (LTSA) to implement the label enhancement process. In general, we first establish a learning model for feature extraction in label space and use a feature extraction method of LTSA to guide the reconstruction of label manifolds. Then, we establish an unconstrained optimization model based on the optimal theory presented in this paper. The model is suitable for solving problems with a large number of sample points. Finally, the experiment results show that the algorithm can effectively improve the training speed and multilabel dataset prediction accuracy.

Some asymptotic properties of kernel regression estimators of the mode for stationary and ergodic continuous time processes.

In the present paper, we consider the nonparametric regression model with random design based on (mathbf{X}_mathrm{t},mathbf{Y}_mathrm{t})_{mathrm{t}ge 0} a mathbb {R}^{d}times mathbb {R}^{q}-valued strictly stationary and ergodic continuous time process, where the regression function is given by m(mathbf{x},psi ) = mathbb {E}(psi (mathbf{Y}) mid mathbf{X} = mathbf{x})), for a measurable function psi : mathbb {R}^{q} rightarrow mathbb {R}. We focus on the estimation of the location {varvec{Theta }} (mode) of a unique maximum of m(cdot , psi ) by the location widehat{{varvec{Theta }}}_mathrm{T} of a maximum of the Nadaraya–Watson kernel estimator widehat{m}_mathrm{T}(cdot , psi ) for the curve m(cdot , psi ). Within this context, we obtain the consistency with rate and the asymptotic normality results for widehat{{varvec{Theta }}}_mathrm{T} under mild local smoothness assumptions on m(cdot , psi ) and the design density f(cdot ) of mathbf{X}. Beyond ergodicity, any other assumption is imposed on the data. This paper extends the scope of some previous results established under the mixing condition. The usefulness of our results will be illustrated in the construction of confidence regions.

Estimating scale-invariant directed dependence of bivariate distributions

Asymmetry of dependence is an inherent property of bivariate probability distributions. Being symmetric, commonly used dependence measures such as Pearson’s r or Spearman’s ρ mask asymmetry and implicitly assume that a random variable Y is equally dependent on a random variable X as vice versa. A copula-based, hence scale-invariant dependence measure called ζ1 overcoming the just mentioned problem was introduced in 2011. ζ1 attains values in [0,1], it is 0 if, and only if X and Y are independent, and 1 if, and only if Y is a measurable function of X. Working with so-called empirical checkerboard copulas allows to construct an estimator ζ1n for ζ1 which is strongly consistent in full generality, i.e., without any smoothness assumptions on the underlying copula. The R-package qad (short for quantification of asymmetric dependence) containing the estimator ζ1n is used both, to perform a simulation study illustrating the small sample performance of the estimator as well as to estimate the directed dependence between some global climate variables as well as between world development indicators.

GEOMETRY-BASED REGULARISATION FOR DENSE IMAGE MATCHING VIA UNCERTAINTY-DRIVEN DEPTH PROPAGATION

Abstract. In the present work, an uncertainty-driven geometry-based regularisation for the task of dense stereo matching is presented. The objective of the regularisation is the reduction of ambiguities in the depth reconstruction process, which exist due to the ill-posed nature of this task. Based on cost and uncertainty information computed beforehand, pixels are selected, whose depth information can be determined correctly with a high probability. This depth information assumed to be of high confidence is initially used to construct a triangle mesh, which is interpreted as surface approximation of the imaged scene and allows to propagate the confident depth information of the triangle vertices within local neighbourhoods. The proposed method further computes confidence scores for propagated depth estimates, which are used to fuse this depth information with the previously computed cost information, introducing a regularisation into the data term of global optimisation methods. Furthermore, based on the propagated depth information the local smoothness assumption of global optimisation methods is adjusted. Instead of fronto-parallel planes, the method presumes planes, which are parallel to the propagated depth information. The performance of the proposed regularisation approach is evaluated in combination with a global optimisation method. For a quantitative and qualitative evaluation two commonly employed and well-established stereo datasets are used. The proposed method shows significant improvements in accuracy on both datasets and for two different cost computation methods. Especially in unstructured areas, artefacts in the disparity maps are reduced.

On Nearly Assumption-Free Tests of Nominal Confidence Interval Coverage for Causal Parameters Estimated by Machine Learning

For many causal effect parameters of interest, doubly robust machine learning (DRML) estimators $\hat{\psi}_{1}$ are the state-of-the-art, incorporating the good prediction performance of machine learning; the decreased bias of doubly robust estimators; and the analytic tractability and bias reduction of sample splitting with cross-fitting. Nonetheless, even in the absence of confounding by unmeasured factors, the nominal $(1-\alpha)$ Wald confidence interval $\hat{\psi}_{1}\pm z_{\alpha/2}\widehat{\mathsf{s.e.}}[\hat{\psi}_{1}]$ may still undercover even in large samples, because the bias of $\hat{\psi}_{1}$ may be of the same or even larger order than its standard error of order $n^{-1/2}$. In this paper, we introduce essentially assumption-free tests that (i) can falsify the null hypothesis that the bias of $\hat{\psi}_{1}$ is of smaller order than its standard error, (ii) can provide a upper confidence bound on the true coverage of the Wald interval, and (iii) are valid under the null under no smoothness/sparsity assumptions on the nuisance parameters. The tests, which we refer to as Assumption Free Empirical Coverage Tests (AFECTs), are based on a U-statistic that estimates part of the bias of $\hat{\psi}_{1}$. Our claims need to be tempered in several important ways. First no test, including ours, of the null hypothesis that the ratio of the bias to its standard error is smaller than some threshold $\delta$ can be consistent [without additional assumptions (e.g., smoothness or sparsity) that may be incorrect]. Second, the above claims only apply to certain parameters in a particular class. For most of the others, our results are unavoidably less sharp. In particular, for these parameters, we cannot directly test whether the nominal Wald interval $\hat{\psi}_{1}\pm z_{\alpha/2}\widehat{\mathsf{s.e.}}[\hat{\psi}_{1}]$ undercovers. However, we can often test the validity of the smoothness and/or sparsity assumptions used by an analyst to justify a claim that the reported Wald interval’s actual coverage is no less than nominal. Third, in the main text, with the exception of the simulation study in Section 1, we assume we are in the semisupervised data setting (wherein there is a much larger dataset with information only on the covariates), allowing us to regard the covariance matrix of the covariates as known. In the simulation in Section 1, we consider the setting in which estimation of the covariance matrix is required. In the simulation, we used a data adaptive estimator which performs very well in our simulations, but the estimator’s theoretical sampling behavior remains unknown.

Generating Adversarial Surfaces via Band‐Limited Perturbations

AbstractAdversarial attacks have demonstrated remarkable efficacy in altering the output of a learning model by applying a minimal perturbation to the input data. While increasing attention has been placed on the image domain, however, the study of adversarial perturbations for geometric data has been notably lagging behind. In this paper, we show that effective adversarial attacks can be concocted for surfaces embedded in 3D, under weak smoothness assumptions on the perceptibility of the attack. We address the case of deformable 3D shapes in particular, and introduce a general model that is not tailored to any specific surface representation, nor does it assume access to a parametric description of the 3D object. In this context, we consider targeted and untargeted variants of the attack, demonstrating compelling results in either case. We further show how discovering adversarial examples, and then using them for adversarial training, leads to an increase in both robustness and accuracy. Our findings are confirmed empirically over multiple datasets spanning different semantic classes and deformations.

New extremal principles with applications to stochastic and semi-infinite programming

This paper develops new extremal principles of variational analysis that are motivated by applications to constrained problems of stochastic programming and semi-infinite programming without smoothness and/or convexity assumptions. These extremal principles concern measurable set-valued mappings/multifunctions with values in finite-dimensional spaces and are established in both approximate and exact forms. The obtained principles are instrumental to derive via variational approaches integral representations and upper estimates of regular and limiting normals cones to essential intersections of sets defined by measurable multifunctions, which are in turn crucial for novel applications to stochastic and semi-infinite programming.

New regularity results for Hamilton–Jacobi equations and long time behavior of pathwise (stochastic) viscosity solutions

We present two new sharp regularity results (regularizing effect and propagation of regularity) for viscosity solutions of uniformly convex homogeneous (space independent) Hamilton–Jacobi equations. The estimates do not depend on the convexity constants of the Hamiltonians. The sharp propagation of regularity result holds in dimension larger than one without additional smoothness assumptions on the data if and only if the Hamiltonians is quadratic in the gradient, a very surprising fact in the theory of Hamilton–Jacobi equations. In turn, the estimates yield new intermittent stochastic regularization results for pathwise (stochastic) viscosity solutions of Hamilton–Jacobi equations with uniformly convex Hamiltonians and rough multiplicative time dependence. Finally, the intermittent estimates allow for the study of the long time behavior of the pathwise (stochastic) viscosity solutions.

Superconvergence Analysis of the Runge–Kutta Discontinuous Galerkin Methods for a Linear Hyperbolic Equation

In this paper, we shall establish the superconvergence property of the Runge–Kutta discontinuous Galerkin (RKDG) method for solving a linear constant-coefficient hyperbolic equation. The RKDG method is made of the discontinuous Galerkin (DG) scheme with upwind-biased numerical fluxes coupled with the explicit Runge–Kutta algorithm of arbitrary orders and stages. Superconvergence results for the numerical flux, cell averages as well as the solution and derivative at some special points are shown, which are based on a systematical study of the $$\hbox {L}^2$$ -norm stability for the RKDG method and the incomplete correction techniques for the well-defined reference functions at each time stage. The result demonstrates that the superconvergence property of the semi-discrete DG method is preserved, and the optimal order in time is provided under the smoothness assumption that is independent of the number of stages. As a byproduct of the above superconvergence study, the expected order of the post-processed solution is obtained when a special initial solution is used. Some numerical experiments are also given.

Accelerated Convergence Algorithm for Distributed Constrained Optimization under Time-Varying General Directed Graphs

This paper studies a class of distributed convex optimization problems by a set of agents in which each agent only has access to its own local convex objective function and the estimate of each agent is restricted to both coupling linear constraint and individual box constraints. Our focus is to devise a distributed primal-dual gradient algorithm for working out the problem over a sequence of time-varying general directed graphs. The communications among agents are assumed to be uniformly strongly connected. A column-stochastic mixing matrix and a fixed step-size are applied in the algorithm which exactly steers all the agents to asymptotically converge to a global optimal solution. Based on the standard strong convexity and the smoothness assumptions of the objective functions, we show that the distributed algorithm is capable of driving the whole network to geometrically converge to an optimal solution of the convex optimization problem only if the step-size does not exceed some upper bound. We also give an explicit analysis for the convergence rate of the proposed optimization algorithm. Simulations on economic dispatch problems and demand response problems in power systems are performed to illustrate the effectiveness of the proposed optimization algorithm.

An eigenvalue estimate for a Robin $p$-Laplacian in $C^1$ domains

Let $\Omega\subset \mathbb{R}^n$ be a bounded $C^1$ domain and $p>1$. For $\alpha>0$, define the quantity \[ \Lambda(\alpha)=\inf_{u\in W^{1,p}(\Omega),\, u\not\equiv 0} \Big(\int_\Omega |\nabla u|^p\,\mathrm{d}x - \alpha \int_{\partial\Omega} |u|^p \,\mathrm{d} s\Big)\Big/ \int_\Omega |u|^p\,\mathrm{d} x \] with $\mathrm{d} s$ being the hypersurface measure, which is the lowest eigenvalue of the $p$-laplacian in $\Omega$ with a non-linear $\alpha$-dependent Robin boundary condition. We show the asymptotics $\Lambda(\alpha) =(1-p)\alpha^{p/(p-1)}+o(\alpha^{p/(p-1)})$ as $\alpha$ tends to $+\infty$. The result was only known for the linear case $p=2$ or under stronger smoothness assumptions. Our proof is much shorter and is based on completely different and elementary arguments, and it allows for an improved remainder estimate for $C^{1,\lambda}$ domains.

Robust optimal feedback control design for uncertain systems based on artificial neural network approximation of the Bellman’s value function

In this study, a local approximated solution for the Hamilton–Jacobi–Bellman equation based on differential neural networks is proposed. The approximated Value function is used to obtain a feedback control law for a class of uncertain dynamical systems. The approach to deal with the uncertainties of the dynamical system is a min–max method that yields obtaining a robust-like solution for an optimal control problem. The proposed cost functional is presented in the Bolza form and the necessary and sufficient conditions for getting the min–max optimal solution with the designed robust version of the dynamic programming approach are provided. Moreover, the effect of the neural network approximation is studied. All the analysis considers a smoothness assumption regarding the Value function. The practical stability of the system with the neural network feedback optimal control is formally demonstrated by means of the Lyapunov method. Finally, the performance of the control law is compared in simulation with a classical optimal controller. The proposed controller overcomes the results produced by the classical controller, which is confirmed by the functional evaluation.

Convergence Analysis of a Numerical Scheme for the Porous Medium Equation by an Energetic Variational Approach

The porous medium equation (PME) is a typical nonlinear degenerate parabolic equation. We have studied numerical methods for PME by an energetic variational approach in [C. Duan et al, J. Comput. Phys., 385 (2019) 13-32], where the trajectory equation can be obtained and two numerical schemes have been developed based on different dissipative energy laws. It is also proved that the nonlinear scheme, based on $f \log f$ as the total energy form of the dissipative law, is uniquely solvable on an admissible convex set and preserves the corresponding discrete dissipation law. Moreover, under certain smoothness assumption, we have also obtained the second order convergence in space and the first order convergence in time for the scheme. In this paper, we provide a rigorous proof of the error estimate by a careful higher order asymptotic expansion and two step error estimates. The latter technique contains a rough estimate to control the highly nonlinear term in a discrete $W^{1,\infty}$ norm, and a refined estimate is applied to derive the optimal error order.

Establishment and evaluation of a multicenter collaborative prediction model construction framework supporting model generalization and continuous improvement: A pilot study

Background and ObjectiveIn recent years, an increasing number of clinical prediction models have been developed to serve clinical care. Establishing a data-driven prediction model based on large-scale electronic health record (EHR) data can provide a more empirical basis for clinical decision making. However, research on model generalization and continuous improvement is insufficiently focused, which also hinders the application and evaluation of prediction models in real clinical environments. Therefore, this study proposes a multicenter collaborative prediction model construction framework to build a prediction model with greater generalizability and continuous improvement capabilities while preserving patient data security and privacy. Materials and MethodsBased on a multicenter collaborative research network, such as the Observational Health Data Sciences and Informatics (OHDSI), a multicenter collaborative prediction model construction framework is proposed. Based on the idea of multi-source transfer learning, in each source hospital, a base classifier was trained according to the model research setting. Then, in the target hospital with missing calibration data, a prediction model was established through weighted integration of base classifiers from source hospitals based on the smoothness assumption. Moreover, a passive-aggressive online learning algorithm was used for continuous improvement of the prediction model, which can help to maintain a high predictive performance to provide reliable clinical decision-making abilities. To evaluate the proposed prediction model construction framework, a prototype system for colorectal cancer prognosis prediction was developed. To evaluate the performance of models, 70,906 patients were screened, including 70,090 from 5 US hospital-specific datasets and 816 from a Chinese hospital-specific dataset. The area under the receiver operating characteristic curve (AUC) and the estimated calibration index (ECI) were used to evaluate the discrimination and calibration of models. ResultsRegarding the colorectal cancer prognosis prediction in our prototype system, compared with the reference models, our model achieved a better performance in model calibration (ECI = 9.294 [9.146, 9.441]) and a similar ability in model discrimination (AUC = 0.783 [0.780, 0.786]). Furthermore, the online learning process provided in this study can continuously improve the performance of the prediction model when patient data with specified labels arrive (the AUC value increased from 0.709 to 0.715 and the ECI value decreased from 13.013 to 9.634 after 650 patient instances with specified labels from the Chinese hospital arrived), enabling the prediction model to maintain a good predictive performance during clinical application. ConclusionsThis study proposes and evaluates a multicenter collaborative prediction model construction framework that can support the construction of prediction models with better generalizability and continuous improvement capabilities without the need to aggregate multicenter patient-level data.

On initial conditions for fractional delay differential equations

Derivatives of fractional order are introduced in different ways: as left-inverse of the fractional integral or by generalizing the limit of the difference quotient defining integer-order derivatives. Although the two approaches lead (under standard smoothness assumptions) to equivalent operators, the first one does not involve the function at the left of the initial point where, instead, the latter forces the function to assume selected values. With fractional delay differential equations new problems arise: the presence of the delay imposes to assign the solution not just at the initial point but on an entire interval. Due to the freedom in the choice of the initial function, some inconsistencies with the values forced by the fractional derivative are possible and the operators may no longer be equivalent. In this paper we discuss the initialization of fractional delay differential equations, we investigate the effects of the initial condition not only on the solution but also on the fractional operator as well and we study the difference between solutions obtained by incorporating or not the initial function in the memory of the fractional derivative. The exact solution of a family of linear equations is obtained by the Laplace transform whilst numerical methods are used to solve nonlinear problems; the different results are therefore shown and commented.