Nonregular Problems Research Articles

Necessary and Sufficient Optimality Conditions for Non-regular Problems

We derive new necessary and sufficient optimality conditions for optimization problems with multi-equality and inequality constraints through the Dubovitskii-Milyutin formalism, characterizing the feasible and tangent directions cones in a neighborhood of a non-regular point. We also establish conditions of 2-regularity under which necessary optimality conditions are non-degenerate. These conditions apply when the phenomenon of non-regularity (or abnormality) take place. In addition examples that illustrate our results are presented.

Solution of the nonregular problem for a parabolic equation with the time derivative in the boundary condition

There is studied the Hölder space solution u ε of the problem for parabolic equation with the time derivative ε ∂ t u ε | Σ in the boundary condition, where ε > 0 is a small parameter. The unique solvability of the perturbed problem and estimates of it’s solution are obtained. The convergence of u ε as ε → 0 to the solution of the unperturbed problem is proved. Boundary layer is not appeared.

Nonregular and minimax estimation of individualized thresholds in high dimension with binary responses

Given a large number of covariates Z, we consider the estimation of a high-dimensional parameter θ in an individualized linear threshold θTZ for a continuous variable X, which minimizes the disagreement between sign(X−θTZ) and a binary response Y. While the problem can be formulated into the M-estimation framework, minimizing the corresponding empirical risk function is computationally intractable due to discontinuity of the sign function. Moreover, estimating θ even in the fixed-dimensional setting is known as a nonregular problem leading to nonstandard asymptotic theory. To tackle the computational and theoretical challenges in the estimation of the high-dimensional parameter θ, we propose an empirical risk minimization approach based on a regularized smoothed non-convex loss function. The Fisher consistency of the proposed method is guaranteed as the bandwidth of the smoothed loss is shrunk to 0. Statistically, we show that the finite sample error bound for estimating θ in ℓ2 norm is (slogd/n)β/(2β+1), where d is the dimension of θ, s is the sparsity level, n is the sample size and β is the smoothness of the conditional density of X given the response Y and the covariates Z. The convergence rate is nonstandard and slower than that in the classical Lasso problems. Furthermore, we prove that the resulting estimator is minimax rate optimal up to a logarithmic factor. The Lepski’s method is developed to achieve the adaption to the unknown sparsity s or smoothness β. Computationally, an efficient path-following algorithm is proposed to compute the solution path. We show that this algorithm achieves geometric rate of convergence for computing the whole path. Finally, we evaluate the finite sample performance of the proposed estimator in simulation studies and a real data analysis from the ChAMP (Chondral Lesions And Meniscus Procedures) Trial.

Transformation-Invariant Learning of Optimal Individualized Decision Rules with Time-to-Event Outcomes

In many important applications of precision medicine, the outcome of interest is time to an event (e.g., death, relapse of disease) and the primary goal is to identify the optimal individualized decision rule (IDR) to prolong survival time. Existing work in this area have been mostly focused on estimating the optimal IDR to maximize the restricted mean survival time in the population. We propose a new robust framework for estimating an optimal static or dynamic IDR with time-to-event outcomes based on an easy-to-interpret quantile criterion. The new method does not need to specify an outcome regression model and is robust for heavy-tailed distribution. The estimation problem corresponds to a nonregular M-estimation problem with both finite and infinite-dimensional nuisance parameters. Employing advanced empirical process techniques, we establish the statistical theory of the estimated parameter indexing the optimal IDR. Furthermore, we prove a novel result that the proposed approach can consistently estimate the optimal value function under mild conditions even when the optimal IDR is nonunique, which happens in the challenging setting of exceptional laws. We also propose a smoothed resampling procedure for inference. The proposed methods are implemented in the R-package QTOCen. We demonstrate the performance of the proposed new methods via extensive Monte Carlo studies and a real data application. Supplementary materials for this article are available online.

On optimal designs for nonregular models

Classically, Fisher information is the relevant object in defining optimal experimental designs. However, for models that lack certain regularity, the Fisher information does not exist and, hence, there is no notion of design optimality available in the literature. This article seeks to fill the gap by proposing a so-called Hellinger information, which generalizes Fisher information in the sense that the two measures agree in regular problems, but the former also exists for certain types of non-regular problems. We derive a Hellinger information inequality, showing that Hellinger information defines a lower bound on the local minimax risk of estimators. This provides a connection between features of the underlying model---in particular, the design---and the performance of estimators, motivating the use of this new Hellinger information for non-regular optimal design problems. Hellinger optimal designs are derived for several non-regular regression problems, with numerical results empirically demonstrating the efficiency of these designs compared to alternatives.

Optimality Conditions for Nonregular Optimal Control Problems and Duality

ABSTRACTWe define a new class of optimal control problems and show that this class is the largest one of control problems where every admissible process that satisfies the Extended Pontryaguin Maximum Principle is an optimal solution of nonregular optimal control problems. In this class of problems the local and global minimum coincide. A dual problem is also proposed, which may be seen as a generalization of the Mond–Weir-type dual problem, and it is shown that the 2-invexity notion is a necessary and sufficient condition to establish weak, strong, and converse duality results between a nonregular optimal control problem and its dual problem. We also present an example to illustrate our results.

Smoothing the payoff for efficient computation of Basket option prices

We consider the problem of pricing basket options in a multivariate Black–Scholes or Variance-Gamma model. From a numerical point of view, pricing such options corresponds to moderate and high-dimensional numerical integration problems with non-smooth integrands. Due to this lack of regularity, higher order numerical integration techniques may not be directly available, requiring the use of methods like Monte Carlo specifically designed to work for non-regular problems. We propose to use the inherent smoothing property of the density of the underlying in the above models to mollify the payoff function by means of an exact conditional expectation. The resulting conditional expectation is unbiased and yields a smooth integrand, which is amenable to the efficient use of adaptive sparse-grid cubature. Numerical examples indicate that the high-order method may perform orders of magnitude faster than Monte Carlo or Quasi Monte Carlo methods in dimensions up to 35.

Generalized Single-Index Models: The EFM Approach

Generalized single-index models are natural extensions of linear models and circumvent the so-called curse of dimensionality. They are becoming increasingly popular in many scientific fields including biostatistics, medicine, economics and finan- cial econometrics. Estimating and testing the model index coefficients beta is one of the most important objectives in the statistical analysis. However, the commonly used assumption on the index coefficients, beta = 1, represents a non-regular problem: the true index is on the boundary of the unit ball. In this paper we introduce the EFM ap- proach, a method of estimating functions, to study the generalized single-index model. The procedure is to first relax the equality constraint to one with (d - 1) components of beta lying in an open unit ball, and then to construct the associated (d - 1) estimating functions by projecting the score function to the linear space spanned by the residuals with the unknown link being estimated by kernel estimating functions. The root-n consistency and asymptotic normality for the estimator obtained from solving the re- sulting estimating equations is achieved, and a Wilk's type theorem for testing the index is demonstrated. A noticeable result we obtain is that our estimator for beta has smaller or equal limiting variance than the estimator of Carroll et al. (1997). A fixed point iterative scheme for computing this estimator is proposed. This algorithm only involves one-dimensional nonparametric smoothers, thereby avoiding the data sparsity problem caused by high model dimensionality. Numerical studies based on simulation and on applications suggest that this new estimating system is quite powerful and easy to implement.

Reachability analysis of reversal-bounded automata on series\u2013parallel graphs

Extensions to finite-state automata on strings, such as multi-head automata or multi-counter automata, have been successfully used to encode many infinite-state non-regular verification problems. In this paper, we consider a generalization of automata-theoretic infinite-state verification from strings to labelled series–parallel graphs. We define a model of non-deterministic, 2-way, concurrent automata working on series–parallel graphs and communicating through shared registers on the nodes of the graph. We consider the following verification problem: given a family of series–parallel graphs described by a context-free graph transformation system (GTS), and a concurrent automaton over series–parallel graphs, is some graph generated by the GTS accepted by the automaton? The general problem is undecidable already for (one-way) multi-head automata over strings. We show that a bounded version, where the automata make a fixed number of reversals along the graph and use a fixed number of shared registers is decidable, even though there is no bound on the sizes of series–parallel graphs generated by the GTS. Our decidability result is based on establishing that the number of context switches can be bounded and on an encoding of the computation of bounded concurrent automata that allows us to reduce the reachability problem to the emptiness problem for pushdown automata.

New air traffic management approach based on expert system and using real-time scheduling algorithms

The air traffic management operations are considered as complex problems. In practice, they can formulate as a constrained optimisation problem that needs to be solved in a real-time environment, also they are non-regular problems, so we can use the expert system to schedule the air traffic management operations. Expert systems using extracted cognitive data, inspiring from the human expertise and its best practices. They are based on the machine performance and its ability to carry out a very large number of complex iterations. Almost all air traffic management operations are classified as scheduling operations. The goal of this paper is to present a new intelligent system with an evolving knowledge base and a creative inference engine. This system allows choosing the optimal way based on real time algorithms to schedule all air traffic management operations such as aircrafts' landing scheduling.

Stochastically optimal bootstrap sample size for shrinkage-type statistics

In nonregular problems where the conventional $$n$$n out of $$n$$n bootstrap is inconsistent, the $$m$$m out of $$n$$n bootstrap provides a useful remedy to restore consistency. Conventionally, opt...

Marginal Inferential Models: Prior-Free Probabilistic Inference on Interest Parameters

The inferential models (IM) framework provides prior-free, frequency-calibrated, and posterior probabilistic inference. The key is the use of random sets to predict unobservable auxiliary variables connected to the observable data and unknown parameters. When nuisance parameters are present, a marginalization step can reduce the dimension of the auxiliary variable which, in turn, leads to more efficient inference. For regular problems, exact marginalization can be achieved, and we give conditions for marginal IM validity. We show that our approach provides exact and efficient marginal inference in several challenging problems, including a many-normal-means problem. In nonregular problems, we propose a generalized marginalization technique and prove its validity. Details are given for two benchmark examples, namely, the Behrens–Fisher and gamma mean problems.

The Non-Regular CEO Problem

We consider the CEO problem for non-regular source distributions (such as uniform or truncated Gaussian). A group of agents observe independently corrupted versions of data and transmit coded versions over rate-limited links to a CEO. The CEO then estimates the underlying data based on the received coded observations. Agents are not allowed to convene before transmitting their observations. This formulation is motivated by the practical problem of a firm’s CEO estimating (non-regular) beliefs about a sequence of events, before acting on them. Agents’ observations are modeled as jointly distributed with the underlying data through a given conditional probability density function. We study the asymptotic behavior of the minimum achievable mean squared error distortion at the CEO in the limit when the number of agents $L$ and the sum rate $R$ tend to infinity. We establish a $1/R^{2}$ convergence of the distortion, an intermediate regime of performance between the exponential behavior in discrete CEO problems [Berger, Zhang, and Viswanathan (1996)], and the $1/R$ behavior in Gaussian CEO problems [Viswanathan and Berger (1997)]. Achievability is proved by a layered architecture with scalar quantization, distributed entropy coding, and midrange estimation. The converse is proved using the Bayesian Chazan–Zakai–Ziv bound.

Nonlinear normal modes of a two degrees-of-freedom piecewise linear system

A study of the Nonlinear Normal Modes (NNMs) of a two degrees of freedom mechanical system with a bilateral elastic stop for one of them is considered. The issue related to the non-smoothness of the impact force is handled through a regularization technique. The Harmonic Balance Method (HBM) with a large number of harmonics, combined with the Asymptotic Numerical Method (ANM), is used to solve the regularized problem. The results are validated from periodic orbits obtained analytically in the time domain by direct integration of the non-regular problem. The first NNM shows an elaborate dynamics with the occurrence of multiple impacts per period, internal resonance and instabilities. On the other hand, the second NNM presents a more simple, almost linear, dynamics. The two NNMs converge asymptotically (for an infinite energy) toward two other Linear Normal Modes, corresponding to the system with a gap equal to zero.

A Case of Regularity of Solutions to Nonregular Problems

We consider the regularity of solutions to the problem of minimizing $\int_{\Omega} [ L(\nabla v(x)) + g(x,v(x))]dx \mbox { on } u^0+W^{1,2}_0(\Omega),$ where $L$ is not necessarily differentiable at the origin.

Stochastically optimal bootstrap sample size for shrinkage-type statistics

In nonregular problems where the conventional $$n$$n out of $$n$$n bootstrap is inconsistent, the $$m$$m out of $$n$$n bootstrap provides a useful remedy to restore consistency. Conventionally, optimal choice of the bootstrap sample size $$m$$m is taken to be the minimiser of a frequentist error measure, estimation of which has posed a major difficulty hindering practical application of the $$m$$m out of $$n$$n bootstrap method. Relatively little attention has been paid to a stronger, stochastic, version of the optimal bootstrap sample size, defined as the minimiser of an error measure calculated directly from the observed sample. Motivated by this stronger notion of optimality, we develop procedures for calculating the stochastically optimal value of $$m$$m. Our procedures are shown to work under special forms of Edgeworth-type expansions which are typically satisfied by statistics of the shrinkage type. Theoretical and empirical properties of our methods are illustrated with three examples, namely the James---Stein estimator, the ridge regression estimator and the post-model-selection regression estimator.

Estimation and inference in parametric deterministic frontier models

In this paper we consider parametric deterministic frontier models. For example, the production frontier may be linear in the inputs, and the error is purely one-sided, with a known distribution such as exponential or half-normal. The literature contains many negative results for this model. Schmidt (Rev Econ Stat 58:238–239, 1976) showed that the Aigner and Chu (Am Econ Rev 58:826–839, 1968) linear programming estimator was the exponential MLE, but that this was a non-regular problem in which the statistical properties of the MLE were uncertain. Richmond (Int Econ Rev 15:515–521, 1974) and Greene (J Econom 13:27–56, 1980) showed how the model could be estimated by two different versions of corrected OLS, but this did not lead to methods of inference for the inefficiencies. Greene (J Econom 13:27–56, 1980) considered conditions on the distribution of inefficiency that make this a regular estimation problem, but many distributions that would be assumed do not satisfy these conditions. In this paper we show that exact (finite sample) inference is possible when the frontier and the distribution of the one-sided error are known up to the values of some parameters. We give a number of analytical results for the case of intercept only with exponential errors. In other cases that include regressors or error distributions other than exponential, exact inference is still possible but simulation is needed to calculate the critical values. We also discuss the case that the distribution of the error is unknown. In this case asymptotically valid inference is possible using subsampling methods.

Generalized convexity for non-regular optimization problems with conic constraints

In non-regular problems the classical optimality conditions are totally inapplicable. Meaningful results were obtained for problems with conic constraints by Izmailov and Solodov (SIAM J Control Optim 40(4):1280–1295, 2001). They are based on the so-called 2-regularity condition of the constraints at a feasible point. It is well known that generalized convexity notions play a very important role in optimization for establishing optimality conditions. In this paper we give the concept of Karush–Kuhn–Tucker point to rewrite the necessary optimality condition given in Izmailov and Solodov (SIAM J Control Optim 40(4):1280–1295, 2001) and the appropriate generalized convexity notions to show that the optimality condition is both necessary and sufficient to characterize optimal solutions set for non-regular problems with conic constraints. The results that exist in the literature up to now, even for the regular case, are particular instances of the ones presented here.

A sufficient optimality condition for nonregular problems via a nonlinear Lagrangian

A reformulation of a standard smooth mathematical program in terms of a nonlinear Lagrangian is used in conjunction with the calculus of subhessians to derive a set of sufficient optimality conditions that are applicable to some nonregular problems. These conditions are cast solely in terms of the first-- and second--order derivatives of the constituent functions and generalize standard second--order sufficiency conditions to a wide class of potentially nonregular problems.

The EFM approach for single-index models

Single-index models are natural extensions of linear models and circumvent the so-called curse of dimensionality. They are becoming increasingly popular in many scientific fields including biostatistics, medicine, economics and financial econometrics. Estimating and testing the model index coefficients $\bolds{\beta}$ is one of the most important objectives in the statistical analysis. However, the commonly used assumption on the index coefficients, $\|\bolds{\beta}\|=1$, represents a nonregular problem: the true index is on the boundary of the unit ball. In this paper we introduce the EFM approach, a method of estimating functions, to study the single-index model. The procedure is to first relax the equality constraint to one with (d-1) components of $\bolds{\beta}$ lying in an open unit ball, and then to construct the associated (d-1) estimating functions by projecting the score function to the linear space spanned by the residuals with the unknown link being estimated by kernel estimating functions. The root-n consistency and asymptotic normality for the estimator obtained from solving the resulting estimating equations are achieved, and a Wilks type theorem for testing the index is demonstrated. A noticeable result we obtain is that our estimator for $\bolds{\beta}$ has smaller or equal limiting variance than the estimator of Carroll et al. [J. Amer. Statist. Assoc. 92 (1997) 447-489]. A fixed-point iterative scheme for computing this estimator is proposed. This algorithm only involves one-dimensional nonparametric smoothers, thereby avoiding the data sparsity problem caused by high model dimensionality. Numerical studies based on simulation and on applications suggest that this new estimating system is quite powerful and easy to implement.