High-dimensional Situations Research Articles

Portfolio Value at Risk Based on Independent Components Analysis

Risk management technology applied to high dimensional portfolios needs simple and fast methods for calculation of Value-at-Risk (VaR). The multivariate normal framework provides a simple off-the-shelf methodology but lacks the heavy tailed distributional properties that are observed in data. A principle component based method (tied closely to the elliptical structure of the distribution) is therefore expected to be unsatisfactory. Here we propose and analyze a technology that is based on Independent Component Analysis (ICA). We study the proposed ICVaR methodology in an extensive simulation study and apply it to a high dimensional portfolio situation. Our analysis yields very accurate VaRs.

Power and sample size estimation in high dimensional biology

Genomic scientists often test thousands of hypotheses in a single experiment. One example is a microarray experiment that seeks to determine differential gene expression among experimental groups. Planning such experiments involves a determination of sample size that will allow meaningful interpretations. Traditional power analysis methods may not be well suited to this task when thousands of hypotheses are tested in a discovery oriented basic research. We introduce the concept of expected discovery rate (EDR) and an approach that combines parametric mixture modelling with parametric bootstrapping to estimate the sample size needed for a desired accuracy of results. While the examples included are derived from microarray studies, the methods, herein, are ‘extraparadigmatic’ in the approach to study design and are applicable to most high dimensional biological situations. Pilot data from three different microarray experiments are used to extrapolate EDR as well as the related false discovery rate at different sample sizes and thresholds.

Three-dimensional flops and noncommutative rings

For $Y,Y^+$ three-dimensional smooth varieties related by a flop, Bondal and Orlov conjectured that the derived categories $D^b({\rm coh}(Y))$ and $D^b({\rm coh}(Y^+))$ are equivalent. This conjecture was recently proved by Bridgeland. Our aim in this paper is to give a partially new proof of Bridgeland's result using noncommutative rings. The new proof also covers some mild singular and higher-dimensional situations (including those occuring in the recent paper by Chen [11]).

Asymptotic Results of a High Dimensional MANOVA Test and Power Comparison When the Dimension is Large Compared to the Sample Size

This paper is concerned with Dempster trace criterion for multivariate linear hypothesis which was proposed for high dimensional situation. First we derive asymptotic null and nonnull distributions of Dempster trace criterion when both the sample size and the dimension tend to infinity. Our approximations are examined through some numerical experiments. Next we compare the power of Dempster trace criterion with the ones of three classical criteria; likelihood ratio criterion, Lawley-Hotelling trace criterion, and Bartlett-Nanda-Pillai trace criterion when the dimension is large compared to the sample size.

Abstract variational problems with volume constraints

Existence results for a class of one-dimensional abstract variational problems with volume constraints are established. The main assumptions on their energy are additivity, translation invariance and solvability of a transition problem. These general results yield existence results for nonconvex problems. A counterexample shows that a naive extension to higher dimensional situations in general fails.

On accessing data in high-dimensional spaces: A comparative study of three space partitioning strategies

While experience shows that contemporary multi-dimensional access methods perform poorly in high-dimensional spaces, little is known about the underlying causes of this important problem. One of the factors that has a profound effect on the performance of a multi-dimensional structure in high-dimensional situations is its space partitioning strategy. This paper investigates the partitioning strategies of KDB-trees, the Pyramid Technique, and a new point access method called the Θ s Technique. The paper reveals important dimensionality problems associated with these strategies and shows how each strategy affects the retrieval performance across a range of spaces with varying dimensionalities. The Pyramid Technique, which is frequently regarded as the state-of-the-art access method for high-dimensional data, suffers from numerous problems that become particularly severe with highly skewed data in heavily sparse spaces. Although the partitioning strategy of KDB-trees incurs several problems in high-dimensional spaces, it exhibits a remarkable adaptability to the changing data distributions. However, the experimental evidence gathered on both simulated and real data sets shows that the Θ s Technique generally outperforms the other two schemes in high-dimensional spaces, usually by a significant margin.

Speed of stochastic locally contractive systems

The auto-regressive model on $\RR ^{d}$ defined by the recurrence equation $ Y^{y}_{n}=a_{n}Y^{y}_{n-1}+B_{n} $, where $ \{ (a_{n},B_{n})\} _{n} $\vspace*{-0.5pt} is a sequence of i.i.d. random variables in $ \RR ^{*}_{+}\times \RR ^{d} $, has, in the critical case $ \esp {\log a_{1}}=0 $,\vspace*{-0.5pt} a local contraction property, that is, when $ Y^{y}_{n} $ is in a compact set the distance $ | Y^{y}_{n}-Y^{x}_{n}| $ converges almost surely to 0. We determine the speed of this convergence and we use this asymptotic estimate to deal with some higher-dimensional situations. In particular, we prove the recurrence and the local contraction property with speed for an autoregressive model whose linear part is given by triangular matrices with first Lyapounov exponent equal to 0. We extend the previous results to a Markov chain on a nilpotent Lie group induced by a random walk on a solvable Lie group of $ \mathcal{NA} $ type.

A direct proof of a theorem of Blaschke and Lebesgue

The Blaschke-Lebesgue Theorem states that among all planar convex domains of given constant width B the Reuleaux triangle has minimal area. It is the purpose of this article to give a direct proof of this theorem by analyzing the underlying variational problem. The advantages of the proof are that it shows uniqueness (modulo rigid deformations such as rotation and translation) and leads analytically to the shape of the area-minimizing domain. Most previous proofs have relied on foreknowledge of the minimizing domain. Key parts of the analysis extend to the higher-dimensional situation, where the convex body of given constant width and minimal volume is unknown.

Data-Driven Model Choice in Multivariate Nonparametric Regression

The problem of choosing between different regression models, is investigated. A completely automatic model selection procedure is defined and asymptotic optimality results are shown. These results are stated in a general framework including as well nested as unnested situations, including as well i.i.d. as dependent samples, and without specifying any class of non-parametric smoothers. Because of the well-known curse of dimensionality, model selection is of particular interest in multivariate regression settings, and it will be discussed how the previous general model choice approach can be used in several different highdimensional situations.

Globally Constrained Local Function Approximations via Hierarchical Modeling; A Framework for System Modeling Under Partial Information

Local function approximations concern fitting low order models to weighted data in neighborhoods of the points where the approximations are desired. Despite their generality and convenience of use, local models typically suffer, among others, from difficulties arising in physical interpretation of the parameters and data sparsity in high dimensional situations (or the so called curse of dimensionality). While estimation in parametric globed models, on the other hand, may eliminate the majority of these problems, it generally raises other important issues such as trade off between selecting an appropriate model structure and preserving good local properties. This paper presents a new approach for system modeling under partial (global) information (or the so called gray-box modeling) that seeks to preserve the benefits of the globed as well as local methodologies within a unified framework. While the proposed technique relies on local approximations, constraints are introduced to ensure the conformity of the estimates to a given global structure. Hierarchical models are then utilized as a tool to accommodate global model uncertainties via parametric variabilities within this structure. The globed parameters and their associated uncertainties are estimated simultaneously with the (local estimates of) function values. The approach is applied to modeling of a linear time varying dynamic system under prior linear time invariant structure where local regression fails as a result of high dimensionality

A model of degenerate and singular oscillatory integral operators

The purpose of this paper is to determine the optimal order of decrease of the L 2 norm for the following degenerate and singular oscillatory integral operators defined by formula here for f ∈ C ∞ 0 (ℝ), where ψ( x , y ) is a smooth compactly supported function on ℝ 2 , S ( x , y ) is a homogeneous polynomial of degree n formula here and K ( x , y ) is a distribution kernel which satisfies the following hypotheses: K is a C 2 function away from the diagonal and the estimates formula here hold for 0 i =1, or 2. This paper basically uses the methods of [ 6 ], which are very general and can be widely used. The main idea of these methods is that the sizes of operator norms after the decomposition are estimated in different ways and the decomposition is then summed back by balancing the estimates of two types. The first type estimate takes into account the support sizes of the integrand after the decomposition. The second type estimate is delicate, depending on how to keep track of the distance to the singular varieties of phase function. In our case, we have to include the singular variety arising from distribution kernel as well. The analytic tool of the second type estimate is the operator version of the Van der Corput lemma [ 7 ]. We notice that the crux of matters is to locate the singular varieties of phase function and kernel. For higher dimensional situations, the problem is very difficult. One expects that the powerful theory of algebraic geometry may play a significant role. Here we would like to remark that Mather's results [ 2 ] may be useful although the geometrical shapes of singular varieties are basically undecidable.

On the route to a better understanding of the complex nonlinear dynamics of a plasma

In many cases, the analytical understanding of the complex nonlinear behaviour of plasmas is simplified by the fact that the plasma dynamics is low-dimensional. Then, quite universal descriptions, which can predict quite sophisticated phenomena, are available. This aspect, and the corresponding appearance of spatially coherent structures, are first discussed for the single-mode dynamics in a weakly unstable (or marginal) regime. The procedure is then generalized to a few modes. The following examples are used to point out the benefits from a low-dimensional description: ionization wave instability, Pierce instability, interchange instability and electron drift instability. Special emphasis is put on the following aspects. Low-dimensional dynamics is quite rich and allows one to apply modern control mechanisms. Coherent structures are present in the dynamics and may contribute to the understanding of transport scalings. The overview is concluded by pointing out some similarities to high-dimensional situations. In the latter, structures (singularities) also have a signature in the dynamical behaviour. They allow us to understand more deeply certain scaling behaviour.

Multivariate tests based on left-spherically distributed linear scores

In this paper, a method for multivariate testing based on low-dimensional, data-dependent, linear scores is proposed. The new approach reduces the dimensionality of observations and increases the stability of the solutions. The method is reliable, even if there are many redundant variables. As a key feature, the score coefficients are chosen such that a left-spherical distribution of the scores is reached under the null hypothesis. Therefore, well-known tests become applicable in high-dimensional situations, too. The presented strategy is an alternative to least squares and maximum likelihood approaches. In a natural way, standard problems of multivariate analysis thus induce the occurrence of left-spherical, nonnormal distributions. Hence, new fields of application are opened up to the generalized multivariate analysis. The proposed methodology is not restricted to normally distributed data, but can also be extended to any left-spherically distributed observations.

Exact Classification with Two-Layer Neural Nets

This paper considers the classification properties of two-layer networks of McCulloch–Pitts units from a theoretical point of view. In particular we consider their ability to realise exactly, as opposed to approximate, bounded decision regions in R2. The main result shows that a two-layer network can realise exactly any finite union of bounded polyhedra in R2whose bounding lines lie in general position, except for some well-characterised exceptions. The exceptions are those unions whose boundaries contain a line which is “inconsistent,” as described in the text. Some of the results are valid for Rn,n⩾2, and the problem of generalising the main result to higher-dimensional situations is discussed.

On matrix probabilities in nonarchimedean decision theory

In an earlier paper, we axiomatized a lexicographic expected utility model for preference in decision under uncertainty that is patterned on the models of Ramsey and Savage but omits their Archimedean axioms. Our model has the unusual feature that subjective probabilities are matrices that premultiply utility vectors in the lexicographic representation of preference between acts. Our purpose here is to analyze the model in relation to the Ramsey-Savage theory along with other models that have a lexicographic feature. A point of departure is Savage's postulate P4, whose purpose is to weakly order hisis more probable than relation on events. This postulate does not hold in our model and we therefore encounter incomparability between events. The paper explores the nature of incomparability, which can be widespread in high-dimensional situations. We include special cases of our model that retain a lexicographic component but also satisfy P4.