High-dimensional Situations Research Articles

Efficient Estimation and Response Variable Selection in Sparse Partial Envelope Model

In this paper, we propose a sparse partial envelope model that performs response variable selection efficiently under the partial envelope model. We discuss its theoretical properties including consistency, an oracle property and the asymptotic distribution of the sparse partial envelope estimator. A large-sample situation and high-dimensional situation are both considered. Numerical experiments demonstrate that the sparse partial envelope estimator has excellent response variable selection performance both in the large-sample situation and the high-dimensional situation. Moreover, simulation studies and real data analysis suggest that the sparse partial envelope estimator has a much more competitive performance than the standard estimator, the oracle partial envelope estimator, the active partial envelope estimator and the sparse envelope estimator, whether it is in the large-sample situation or the high-dimensional situation.

Feature screening filter for high dimensional survival data in the reproducing kernel Hilbert space

We consider the issue of screening uninformative variables associate with survival outcome in ultra high-dimensional situation. specifically, we focus on Hilbert-Schmits Independent criteria (HSIC) statistic defined in the Reproducing kernel Hilbert Space(RKHS) and use the HSIC as a feature screening approach to avoid from difficulty occurred from the ultra high dimensionality. Generally, it is known that the HSIC can account for the flexible relationship between the outcome and predictor variables. Compared with the typical approaches such as regularization and model free screening approaches, our approach does not require the complex non parametric computation or the complicated numerical optimization, Thereby, our proposed approach effectively and flexibly filters out the uninformative predictor. We will show the advantages of our method in numerical simulation studies and illustrate the usefulness of it by applying to diffuse large-B-cell lymphoma data.

Gaussian model for closed curves

In the case of image processing or understanding, one of the common important tasks is to fit closed curves (e.g., circles, ellipses, etc.) to the underlying image. In higher-dimensional situations, the problem of modeling clusters as closed curves remains even more challenging.To deal with this problem we introduce a new probability distribution, which models complicated closed curves with the use of Fourier series. Then, a mixture of such distributions is constructed, leading to our model MCEC. It is shown that MCEC can be effectively trained in the case of closed curves in Rn.MCEC was evaluated in clustering, curve fitting, and image segmentation tasks. We compare it in particular to classical GMM, Hough transform, and Snakes algorithm. In each task, we consistently obtain or outperform the current SOTA.

A Novel High-Dimensional Kernel Joint Non-Negative Matrix Factorization With Multimodal Information for Lung Cancer Study.

Judging and identifying biological activities and biomarkers inside tissues from imaging features of diseases is challenging, so correlating pathological image data with genes inside organisms is of great significance for clinical diagnosis. This paper proposes a high-dimensional kernel non-negative matrix factorization (NMF) method based on muti-modal information fusion. This algorithm can project RNA gene expression data and pathological images (WSI) into a common feature space, where the heterogeneous variables with the largest coefficient in the same projection direction form a co-module. In addition, the miRNA-mRNA and miRNA-lncRNA interaction networks in the ceRNA network are added to the algorithm as a priori information to explore the relationship between the images and the internal activities of the gene. Furthermore, the radial basis kernel function is used to calculate the feature proportion between different kinds of genes mapped in the high-dimensional feature space and projected into the common feature space to explore the gene interaction in the high-dimensional situation. The original feature matrix is regularized to improve biological correlation, and the feature factors are sparse by orthogonal constraints to reduce redundancy. Experimental results show that the proposed NMF method is better than the traditional NMF method in stability, decomposition accuracy, and robustness. Through data analysis applied to lung cancer, genes related to tissue morphology are found, such as COL7A1, CENPF and BIRC5. In addition, gene pairs with a correlation degree exceeding 0.8 are found, and potential biomarkers of significant correlation with survival are obtained such as CAPN8. It has potential application value for the clinical diagnosis of lung cancer.

Application of fused graphical lasso to statistical inference for multiple sparse precision matrices.

In this paper, the fused graphical lasso (FGL) method is used to estimate multiple precision matrices from multiple populations simultaneously. The lasso penalty in the FGL model is a restraint on sparsity of precision matrices, and a moderate penalty on the two precision matrices from distinct groups restrains the similar structure across multiple groups. In high-dimensional settings, an oracle inequality is provided for FGL estimators, which is necessary to establish the central limit law. We not only focus on point estimation of a precision matrix, but also work on hypothesis testing for a linear combination of the entries of multiple precision matrices. We apply a de-biasing technology, which is used to obtain a new consistent estimator with known distribution for implementing the statistical inference, and extend the statistical inference problem to multiple populations. The corresponding de-biasing FGL estimator and its asymptotic theory are provided. A simulation study and an application of the diffuse large B-cell lymphoma data show that the proposed test works well in high-dimensional situation.

Deep Bayesian experimental design for quantum many-body systems

Bayesian experimental design is a technique that allows to efficiently select measurements to characterize a physical system by maximizing the expected information gain. Recent developments in deep neural networks and normalizing flows allow for a more efficient approximation of the posterior and thus the extension of this technique to complex high-dimensional situations. In this paper, we show how this approach holds promise for adaptive measurement strategies to characterize present-day quantum technology platforms. In particular, we focus on arrays of coupled cavities and qubit arrays. Both represent model systems of high relevance for modern applications, like quantum simulations and computing, and both have been realized in platforms where measurement and control can be exploited to characterize and counteract unavoidable disorder. Thus, they represent ideal targets for applications of Bayesian experimental design.

Sparse Linear Discriminant Analysis Based on lq Regularization

Linear discriminant analysis plays an important role in feature extraction, data dimensionality reduction, and classification. With the progress of science and technology, the data that need to be processed are becoming increasingly large. However, in high-dimensional situations, linear discriminant analysis faces two problems: the lack of interpretability of the projected data since they all involve all p features, which are linear combinations of all features, as well as the singularity problem of the within-class covariance matrix. There are three different arguments for linear discriminant analysis: multivariate Gaussian model, Fisher discrimination problem, and optimal scoring problem. To solve these two problems, this article establishes a model for solving the kth discriminant component, which first transforms the original model of Fisher discriminant problem in linear discriminant analysis by using a diagonal estimated matrix for the within-class variance in place of the original within-class covariance matrix, which overcomes the singularity problem of the matrix and projects it to an orthogonal projection space to remove its orthogonal constraints, and subsequently an lq norm regularization term is added to enhance its interpretability for the purpose of dimensionality reduction and classification. Finally, an iterative algorithm for solving the model and a convergence analysis are given, and it is proved that the sequence generated by the algorithm is descended and converges to a local minimum of the problem for any initial value.

Computation Offloading for Tasks With Bound Constraints in Multiaccess Edge Computing

Multi-Access Edge Computing (MEC) provides task offloading services to facilitate the integration of idle resources with the network and bring cloud services closer to the end user. By selecting suitable servers and properly managing resources, task offloading can reduce task completion latency while maintaining the Quality of Service (QoS). Prior research, however, has primarily focused on tasks with strict time constraints, ignoring the possibility that tasks with soft constraints may exceed the bound limits and failing to analyze this complex task constraint issue. Furthermore, considering additional constraint features makes convergent optimization algorithms challenging when dealing with such complex and high-dimensional situations. In this paper, we propose a new computational offloading decision framework by minimizing the long-term payment of computational tasks with mixed bound constraints. In addition, redundant experiences are gotten rid of before the training of the algorithm. The most advantageous transitions in the experience pool are used for training in order to improve the learning efficiency and convergence speed of the algorithm as well as increase the accuracy of offloading decisions. The findings of our experiments indicate that the method we have presented is capable of achieving fast convergence rates while also reducing sample redundancy.

Large portfolio allocation based on high-dimensional regression and Kendall’s Tau

In financial investments, portfolio allocation is always one of the most fundamental and challenging tasks. This paper proposes a robust portfolio optimization approach, extending the application of classical mean-variance (M-VAR) method for high-dimensional situations. The yielded assets return of the proposed method can be enhanced to some extent. It is called Kendall’s tau unconstrained shrinkage regression for M-VAR method (KUSR-MV). By some representative empirical studies, it is shown to have a more robust estimation of high-dimensional portfolio allocation compared to its competitors. Besides, its Sharpe ratio can be improved while the risk constraint can be well remained.

Analysis of the application methods of film and television media and images in the era of big data cloud

Abstract In order to give full play to the application of big data in film and television media and imaging in the cloud era, this study proposes a communication-efficient distributed deep neural network training method based on the DANE algorithm framework. The DANE algorithm is an approximate Newtonian method that has been widely used in communication-efficient distributed machine learning. It has the advantages of fast convergence and no need to calculate the inverse of the Hessian matrix, which can significantly reduce the communication and computational overhead in high-dimensional situations. In order to further improve the computational efficiency, it is necessary to study how to speed up the local optimization of DANE. It is a feasible method to choose to use the most popular adaptive gradient optimization algorithm Adam to replace the commonly used stochastic gradient descent method to solve the local single-machine suboptimization problem of DANE. Experiments show that Adam-based optimization can converge significantly faster than the original SGD-based implementation with little sacrifice in model generalization performance. With the increase of sampling rate, DANE-Adam significantly outperforms the DANE method in terms of convergence speed, and at the same time, the accuracy can be kept almost unchanged, which are 0.96, 0.88 and 0.75, respectively. This shows that Adam-based optimization can converge significantly faster than the original SGD-based implementation with little sacrifice in model generalization performance, with significant potential value.

Stein variational gradient descent with learned direction

Recently, as a particle-based Bayesian inference method, Stein variational gradient descent (SVGD) has attracted much attention due to its strong approximation power and efficiency compared to traditional variational inference and Markov chain Monte Carlo sampling methods. However, the original SVGD method assumes the descent direction to be within an reproducing kernel Hilbert space (RKHS), which on one hand limits its expressive power, and on the other hand suffers from the curse of dimensionality. To address this issue, we propose to parameterize the descent direction using a flexible neural network. With this parameterization strategy, the descent direction can be possibly optimized in each step for updating the particles, such that the target distribution can be expected to be better approximated. Experiments on both synthetic and real datasets demonstrate the effectiveness and efficiency of the proposed method, especially in the high-dimensional situations, as compared with the original SVGD.

Foliations, solvability, and global injectivity

Let $F: \mathbb{R}^n\to\mathbb{R}^n$ be a $C^{\infty}$ map such that $DF(x)$ is invertible for every $x\in\mathbb{R}^n$. Although being a local diffeomorphism, $F$ is not necessarily globally injective if $n\geq2$. Finding additional assumptions implying the global injectivity of $F$ for $n\geq 2$ is object of intense study in several areas of Mathematics. In this paper we revisit some assumptions and relations between them in the bidimensional case and discuss the natural higher dimensional situation.

Fast and scalable learning of sparse changes in high-dimensional graphical model structure

We focus on the problem of estimating the change in the dependency structures of two p-dimensional Gaussian Graphical models (GGMs). Previous studies for sparse change estimation in GGMs involve expensive and difficult non-smooth optimization. We propose a novel method, DIFFEE for estimating DIFFerential networks via an Elementary Estimator under a high-dimensional situation. DIFFEE is solved through a faster and closed-form solution that enables it to work in large-scale settings. Notice that GGM assumes data are generated from a Gaussian distribution. However, the Gaussian assumption is too strict and can not be satisfied with all the real-world data generated from a complex process. Therefore, we further extend DIFFEE to NPN-DIFFEE by assuming that data are drawn from the nonparanormal distribution (a large family of distributions) instead of a multivariate Gaussian distribution. Thus, NPN-DIFFEE is applicable to more general conditions. We conduct a rigorous statistical analysis showing that surprisingly DIFFEE achieves the same asymptotic convergence rates as the state-of-the-art estimators that are much more difficult to compute. Our experimental results on multiple synthetic datasets and one real-world data about brain connectivity show strong performance improvements over baselines, as well as significant computational benefits.

Robust Variable Selection Based on Penalized Composite Quantile Regression for High-Dimensional Single-Index Models

The single-index model is an intuitive extension of the linear regression model. It has been increasingly popular due to its flexibility in modeling. In this work, we focus on the estimators of the parameters and the unknown link function for the single-index model in a high-dimensional situation. The SCAD and Laplace error penalty (LEP)-based penalized composite quantile regression estimators, which could realize variable selection and estimation simultaneously, are proposed; a practical iterative algorithm is introduced to obtain the efficient and robust estimators. The choices of the tuning parameters, the bandwidth, and the initial values are also discussed. Furthermore, under some mild conditions, we show the large sample properties and oracle property of the SCAD and Laplace penalized composite quantile regression estimators. Finally, we evaluated the performances of the proposed estimators by two numerical simulations and a real data application.

New estimation approaches for graphical models with elastic net penalty

In the context of undirected Gaussian graphical models, three estimators based on elastic net penalty for the underlying dependence graph are introduced. The aim is to estimate a sparse precision matrix, from which to retrieve both the underlying conditional dependence graph and the partial correlations. The first estimator is derived from the direct penalization of the precision matrix in the likelihood function, while the second uses penalized regressions to estimate the precision matrix. Finally, the third estimator relies on a two stage procedure that estimates the edge set first and then the precision matrix elements. Through simulations the performances of the proposed methods are investigated on a set of well-known network structures. Results on simulated data show that in high-dimensional situations the second estimator performs relatively well, while in low-dimensional settings the two stage procedure outperforms most estimators as the sample size grows. Nonetheless, there are situations where the first estimator is also a good choice. Mixed results suggest that the elastic net penalty is not always the best choice when compared to the LASSO penalty, i.e. pure ℓ1 penalty, even if elastic net penalty tends to outperform LASSO in presence of highly correlated data from the cluster structure. Finally, using real-world data on U.S. economic sectors, dependencies are estimated and the impact of Covid-19 pandemic on the network strength is studied.

Gradient-induced Model-free Variable Selection Based on Composite Quantile Regression in Reproducing Kernel Hilbert Space

Variable selection plays an important role to identify truly informative variables in high-dimensional data analysis. In his paper, we propose a variable selection method with composite quantile regression in reproducing kernel Hilbert space (RKHS), which has two main advantages. The first is that our method requires no special model structure assumption and no independence of error term. It is suitable for general non-parametric models and even heteroscedastic models. The second is that the calculation is simple and fast. So, it can also work in high-dimensional situations. Finally, the numerical experiments and real data analysis demonstrate its superior performance in variable selection.

A new measure for assessment of clustering based on kernel density estimation

A new clustering accuracy measure is proposed to determine the unknown number of clusters and to assess the quality of clustering of a data set given in any dimensional space. Our validity index applies the classical nonparametric univariate kernel density estimation method to the interpoint distances computed between the members of data. Being based on interpoint distances, it is free of the curse of dimensionality and therefore efficiently computable for high-dimensional situations where the number of study variables can be larger than the sample size. The proposed measure is compatible with any clustering algorithm and with every kind of data set where the interpoint distance measure can be defined to have a density function. Our conducted simulation study proves its superiority over widely used cluster validity indices like the average silhouette width and the Dunn index, whereas its applicability is shown with respect to a high-dimensional Biostatistical study of Alon data set and a large Astrostatistical application of time series with light curves of new variable stars.

Adjusting systematic bias in high dimensional principal component scores

Principal component analysis continues to be a powerful tool in dimension reduction of high dimensional data. We assume a variance-diverging model and use the high-dimension, low-sample-size asymptotics to show that even though the principal component directions are not consistent, the sample and prediction principal component scores can be useful in revealing the population structure. We further show that these scores are biased, and the bias is asymptotically decomposed into rotation and scaling parts. We propose methods of bias-adjustment that are shown to be consistent and work well in the high dimensional situations with small sample sizes. The potential advantage of bias-adjustment is demonstrated in a classification setting.

Spatial rank-based high-dimensional change point detection via random integration

Detecting change points is an important task to identify an abrupt and significant change in the data generating process. Traditional change point detection methods are not applicable in high-dimensional situations due to many obstacles, such as the requirement of normality or the estimation of the covariance matrix. This paper presented a novel nonparametric method to overcome such issues by using the random integration with spatial ranks, which is tailored to high-dimensional change point detection problems. The proposed method is a unified framework that includes and extends many existing methods and can effectively handle high-dimensional non-normal data, whose asymptotic properties are established under mild conditions. In addition, we developed a computationally efficient algorithm to calculate the rejection thresholds and an effective post-signal diagnostic procedure to identify the potential directions. Finally, numerical studies together with real data examples demonstrated that the proposed method can identify the change point efficiently.

Monitoring Forest Health Using Hyperspectral Imagery: Does Feature Selection Improve the Performance of Machine-Learning Techniques?

This study analyzed highly correlated, feature-rich datasets from hyperspectral remote sensing data using multiple statistical and machine-learning methods. The effect of filter-based feature selection methods on predictive performance was compared. In addition, the effect of multiple expert-based and data-driven feature sets, derived from the reflectance data, was investigated. Defoliation of trees (%), derived from in situ measurements from fall 2016, was modeled as a function of reflectance. Variable importance was assessed using permutation-based feature importance. Overall, the support vector machine (SVM) outperformed other algorithms, such as random forest (RF), extreme gradient boosting (XGBoost), and lasso (L1) and ridge (L2) regressions by at least three percentage points. The combination of certain feature sets showed small increases in predictive performance, while no substantial differences between individual feature sets were observed. For some combinations of learners and feature sets, filter methods achieved better predictive performances than using no feature selection. Ensemble filters did not have a substantial impact on performance. The most important features were located around the red edge. Additional features in the near-infrared region (800–1000 nm) were also essential to achieve the overall best performances. Filter methods have the potential to be helpful in high-dimensional situations and are able to improve the interpretation of feature effects in fitted models, which is an essential constraint in environmental modeling studies. Nevertheless, more training data and replication in similar benchmarking studies are needed to be able to generalize the results.