Dimensional Data Analysis Research Articles

An improved diversity visualization system for multivariate data

Exploring and analyzing data is becoming increasingly difficult due to the growth of data. Visual analytics tools can be an attractive solution to support the process to derive insights from data. Currently, there are many visual representation methods to visualize the diversity in multivariate data sets. However, most of these applications focus on visual representation problems, and these solutions support limited interactive components for users to effectively explore and analyze data on screen. In this paper, the adaptive diversity table (ADT) is proposed to solve the visual representation problems (occlusion and technique interference). Furthermore, it integrates the mantra techniques to support users to accomplish seven important tasks (i.e. overview, zoom, filter, details-on-demand, relate, history, and extract) that are useful for high dimensional data exploration and data analysis. Experimental results show that the proposed ADT is a better visual representation as compared to other prior techniques. Majority of the respondents prefers to use the proposed ADT over the other visual representation methods. User studies also show that the proposed ADT is more useful as it enables the respondents to be more efficient in analyzing the data sets provided.

High Dimensional Correlation Matrices: The Central Limit Theorem and Its Applications

Summary Statistical inferences for sample correlation matrices are important in high dimensional data analysis. Motivated by this, the paper establishes a new central limit theorem for a linear spectral statistic of high dimensional sample correlation matrices for the case where the dimension p and the sample size n are comparable. This result is of independent interest in large dimensional random-matrix theory. We also further investigate the sample correlation matrices of a high dimensional vector whose elements have a special correlated structure and the corresponding central limit theorem is developed. Meanwhile, we apply the linear spectral statistic to an independence test for p random variables, and then an equivalence test for p factor loadings and n factors in a factor model. The finite sample performance of the test proposed shows its applicability and effectiveness in practice. An empirical application to test the independence of household incomes from various cities in China is also conducted.

Universal Nonlinear Regression on High Dimensional Data Using Adaptive Hierarchical Trees

We study online sequential regression with nonlinearity and time varying statistical distribution when the regressors lie in a high dimensional space. We escape the curse of dimensionality by tracking the subspace of the underlying manifold using a hierarchical tree structure. We use the projections of the original high dimensional regressor space onto the underlying manifold as the modified regressor vectors for modeling of the nonlinear system. By using the proposed algorithm, we reduce the computational complexity to the order of the depth of the tree and the memory requirement to only linear in the intrinsic dimension of the manifold. The proposed techniques are specifically applicable to high dimensional streaming data analysis in a time varying environment. We demonstrate the significant performance gains in terms of mean square error over the other state of the art techniques through simulated as well as real data.

대학 행정의 정보통합 및 통계분석을 위한 다차원 BI 시스템의 설계 및 구현

As the number of legacy database systems and the size of data to manipulate have been vastly increased, it has become more difficult and complex to analyze characteristics of data. To improve the efficiency of data analysis and help administrators to make decisions in business life, BI(Business Intelligence) system is used. To construct data warehouse and cube from legacy database systems makes it easy and fast to transform raw data into integrated and categorized meaningful information. In this paper, we built a BI system for an University administration. Several source system databases were integrated to data warehouse to build data cubes. The implemented BI system shows much faster data analysis and reporting ability than the manipulation in legacy systems. It is especially efficient in multi dimensional data analysis, nonetheless in single dimensional analysis.

A Model for Implementing an Evidence-Based Practice in Student Fieldwork Placements: Barriers and Facilitators to the Use of “SBIRT”

ABSTRACTThis article reports on focus group data analysis that was used to understand initial, largely positive outcomes from a university-based initiative to disseminate and implement an evidence-based practice (EBP)—Screening, Brief Intervention, and Referral to Treatment for Substance Use (SBIRT)—into student fieldwork placements, which are usual care social work settings. Focus groups were conducted with three groups of stakeholders involved in the ongoing project: social work department faculty (n = 10), bachelor- and master-level social work students (n = 8), and social work fieldwork instructors (n = 6). Dimensional analysis of the focus group data yielded results indicating that dissemination and implementation of SBIRT was influenced by agency- and school-level factors and perceived fit between the EBP and individual professional identity, intrapersonal characteristics, and timing. The resulting model, developed through the focus group analysis, is offered and shows how these factors interacted and affected training, supervision, and use of the EBP. The model provides social work educational programs and agencies a working tool for diagnosing and proactively addressing barriers and breakdowns in the EBP implementation process. Future research that tests the model as a diagnostic tool and generates knowledge about its influence in developing competent evidence-based practitioners is indicated. Future focus groups in relation to this initiative are needed to better understand these barriers and facilitators in the EBP implementation process and their critical roles in the process of translating SBIRT into standard social work practice.

Semi-supervised convex nonnegative matrix factorizations with graph regularized for image representation

Non-negative matrix factorization (NMF) is a very effective method for high dimensional data analysis, which has been widely used in computer vision. It can capture the underlying structure of image in the low dimensional space using its parts-based representations. However, nonnegative entries are usually required for the data matrix in NMF, which limits its application. Besides, it is actually an unsupervised method without making use of prior information of data. In this paper, we propose a novel method called Pairwise constrained Graph Regularized Convex Nonnegative Matrix Factorization (PGCNMF), which not only allows the processing of mixed-sign data matrix but also incorporates pairwise constraints generated among all labeled data into Convex NMF framework. We expect that images which have the same class label will have very similar representations in the low dimensional space as much as possible, while images with different class labels will have dissimilar representations as much as possible. Clustering experiments on nonnegative and mixed-sign real-world image datasets are conducted to demonstrate the effectiveness of the proposed method.

Vigilance in the laboratory predicts avoidance in the real world: A dimensional analysis of neural, behavioral, and ecological momentary data in anxious youth

Vigilance and avoidance of threat are observed in anxious adults during laboratory tasks, and are posited to have real-world clinical relevance, but data are mixed in anxious youth. We propose that vigilance-avoidance patterns will become evident in anxious youth through a focus on individual differences and real-world strategic avoidance. Decreased functional connectivity between the amygdala and prefrontal cortex (PFC) could play a mechanistic role in this link. 78 clinically anxious youth completed a dot-probe task to assess vigilance to threat while undergoing fMRI. Real-world avoidance was assessed using Ecological Momentary Assessment (EMA) of self-reported suppression and distraction during negative life events. Vigilance toward threat was positively associated with EMA distraction and suppression. Functional connectivity between a right amygdala seed region and dorsomedial and right dorsolateral PFC regions was inversely related to EMA distraction. Dorsolateral PFC-amygdalar connectivity statistically mediated the relationship between attentional vigilance and real-world distraction. Findings suggest anxious youth showing attentional vigilance toward threat are more likely to use suppression and distraction to regulate negative emotions. Reduced PFC control over limbic reactivity is a possible neural substrate of this pattern. These findings lend ecological validity to laboratory vigilance assessments and suggest PFC-amygdalar connectivity is a neural mechanism bridging laboratory and naturalistic contexts.

Novel technologies and emerging biomarkers for personalized cancer immunotherapy.

The culmination of over a century’s work to understand the role of the immune system in tumor control has led to the recent advances in cancer immunotherapies that have resulted in durable clinical responses in patients with a variety of malignancies. Cancer immunotherapies are rapidly changing traditional treatment paradigms and expanding the therapeutic landscape for cancer patients. However, despite the current success of these therapies, not all patients respond to immunotherapy and even those that do often experience toxicities. Thus, there is a growing need to identify predictive and prognostic biomarkers that enhance our understanding of the mechanisms underlying the complex interactions between the immune system and cancer. Therefore, the Society for Immunotherapy of Cancer (SITC) reconvened an Immune Biomarkers Task Force to review state of the art technologies, identify current hurdlers, and make recommendations for the field. As a product of this task force, Working Group 2 (WG2), consisting of international experts from academia and industry, assembled to identify and discuss promising technologies for biomarker discovery and validation. Thus, this WG2 consensus paper will focus on the current status of emerging biomarkers for immune checkpoint blockade therapy and discuss novel technologies as well as high dimensional data analysis platforms that will be pivotal for future biomarker research. In addition, this paper will include a brief overview of the current challenges with recommendations for future biomarker discovery.

Recursive Random Lasso (RRLasso) for Identifying Anti-Cancer Drug Targets.

Uncovering driver genes is crucial for understanding heterogeneity in cancer. L 1-type regularization approaches have been widely used for uncovering cancer driver genes based on genome-scale data. Although the existing methods have been widely applied in the field of bioinformatics, they possess several drawbacks: subset size limitations, erroneous estimation results, multicollinearity, and heavy time consumption. We introduce a novel statistical strategy, called a Recursive Random Lasso (RRLasso), for high dimensional genomic data analysis and investigation of driver genes. For time-effective analysis, we consider a recursive bootstrap procedure in line with the random lasso. Furthermore, we introduce a parametric statistical test for driver gene selection based on bootstrap regression modeling results. The proposed RRLasso is not only rapid but performs well for high dimensional genomic data analysis. Monte Carlo simulations and analysis of the “Sanger Genomics of Drug Sensitivity in Cancer dataset from the Cancer Genome Project” show that the proposed RRLasso is an effective tool for high dimensional genomic data analysis. The proposed methods provide reliable and biologically relevant results for cancer driver gene selection.

A-Optimal Non-negative Projection with Hessian regularization

With the demand of high dimensional data analysis, data representation (or feature learning) has attracted more and more attention and becomes a central problem in pattern recognition and data mining. Non-negative Matrix Factorization (NMF) which is a useful data representation method makes great contribution in finding the latent structure of the data and leading to a parts-based representation by decomposing the data matrix into a few bases and encodings with the non-negative constraint. Considering the learned encodings from a statistical view by modeling the data points via ridge regression and minimizing the variance of the parameter, A-Optimal Non-negative Projection (ANP) improves the performance of NMF. However, it neglects the intrinsic geometric structure of the data. We introduce Hessian regularization and propose a novel method called A-Optimal Non-negative Projection with Hessian regularization (AHNP) to address this problem. Therefore, AHNP not only leads to parts-based and precise representations but preserves the intrinsic geometrical structure of the obtained subspace. We demonstrate the effectiveness of this novel algorithm through a set of evaluations on real world applications.

Principal component selection via adaptive regularization method and generalized information criterion

The principal component analysis has been widely used in various fields of research (e.g., bioinformatics, medical statistics, etc.), especially high dimensional data analysis. Although crucial components selection is a vital matter in principal components analysis, relatively little attention was paid to this issue. The existing studies for principal component analysis were based on ad-hoc methods (e.g., method with cumulative percent variance or average eigenvalue). We propose a novel method for selecting principal component based on L\(_{1}\)-type regularized regression modeling. In order to effectively perform for principal component regression, we consider adaptive L\(_{1}\)-type penalty based on singular values of components, and propose adaptive penalized principal component regression. The proposed method can perform feature selection incorporating explanation power of components for not only high-dimensional predictor variables but also response variable. In sparse regression modeling, choosing the regularization parameter is a crucial issue, since feature selection and estimation heavily depend on the selected regularization parameter. We derive a model selection criterion for choosing the regularization parameter of the proposed adaptive L\(_{1}\)-type regularization method in line with a generalized information criterion. Monte Carlo simulations and real data analysis demonstrate that the proposed modeling strategies outperform for principal component regression modeling.

Sparse HDLSS discrimination with constrained data piling

Regularization is a key component in high dimensional data analyses. In high dimensional discrimination with binary classes, the phenomenon of data piling occurs when the projection of data onto a discriminant vector is dichotomous, one for each class. Regularizing the degree of data piling yields a new class of discrimination rules for high dimension–low sample size data. A discrimination method that regularizes the degree of data piling while achieving sparsity is proposed and solved via a linear programming. Computational efficiency is further improved by a sign-preserving regularization that forces the signs of the estimator to be the same as the mean difference. The proposed classifier shows competitive performances for simulated and real data examples including speech recognition and gene expressions.

Subspace Learning via Local Probability Distribution for Hyperspectral Image Classification

The computational procedure of hyperspectral image (HSI) is extremely complex, not only due to the high dimensional information, but also due to the highly correlated data structure. The need of effective processing and analyzing of HSI has met many difficulties. It has been evidenced that dimensionality reduction has been found to be a powerful tool for high dimensional data analysis. Local Fisher’s liner discriminant analysis (LFDA) is an effective method to treat HSI processing. In this paper, a novel approach, called PD-LFDA, is proposed to overcome the weakness of LFDA. PD-LFDA emphasizes the probability distribution (PD) in LFDA, where the maximum distance is replaced with local variance for the construction of weight matrix and the class prior probability is applied to compute the affinity matrix. The proposed approach increases the discriminant ability of the transformed features in low dimensional space. Experimental results on Indian Pines 1992 data indicate that the proposed approach significantly outperforms the traditional alternatives.

A New Method for Tracking Configuration for Dirichlet Process Sampling

The method of fitting a hierarchical model with Dirichlet process mixing is a versatile tool for data analysts. It has been applied to density estimation, classification, clustering, and high dimensional data analysis. Many computing algorithms have been proposed to evaluate this mixture. Different labels in the algorithm that assign data points into clusters may actually yield the same partition configuration. This paper makes this notion rigorous by establishing an equivalence theorem. Thus, we would recommend adding the step of checking for equivalent configurations to the algorithms for evaluating hierarchical Dirichlet process mixing models for improved results, especially when cluster assignments are the major goals of the analysis. DOI: http://dx.doi.org/10.4038/sljastats.v5i4.7781

Investigation of the performance of a batch air spouting bed in conducting ion exchange reactions involving heavy metal removal

Spouted beds provide a means of good hydrodynamic conditions and circulation of solid particles. The spouting technique has been applied in many industrial processes, such as drying of granular materials, blending of polymer chips, coating of tablets, pyrolysis, gasification, and granulation of fertilizers. The aim of the present study is to examine the performance of a batch conical air spouting vessel for conducting ion exchange reactions involving heavy metal removal. The rate of ion exchange reaction was expressed in terms of volumetric mass transfer coefficient (K). The effect of various parameters such as type of heavy metal ion (Ni2+, Pb2+), superficial air velocity, and initial heavy metal concentration has been investigated. It has been found that volumetric mass transfer coefficient increases as air superficial velocity increases. Mass transfer correlations were obtained using dimensional analysis of the present mass transfer data. The importance of these mass transfer correlations in the design and operation of air spouting vessels used for conducting diffusion-controlled reactions was highlighted.

High Dimensional Correlation Matrices: CLT and Its Applications

Statistical inferences for sample correlation matrices are important in high dimensional data analysis. Motivated by this, this paper establishes a new central limit theorem (CLT) for a linear spectral statistic (LSS) of high dimensional sample correlation matrices for the case where the dimension p and the sample size n are comparable. This result is of independent interest in large dimensional random matrix theory. Meanwhile, we apply the linear spectral statistic to an independence test for p random variables, and then an equivalence test for p factor loadings and n factors in a factor model. The finite sample performance of the proposed test shows its applicability and effectiveness in practice. An empirical application to test the independence of household incomes from different cities in China is also conducted.

Some properties of generalized fused lasso and its applications to high dimensional data

Identifying homogeneous subgroups of variables can be challenging in high dimensional data analysis with highly correlated predictors. The generalized fused lasso has been proposed to simultaneously select correlated variables and identify them as predictive clusters (grouping property). In this article, we study properties of the generalized fused lasso. First, we present a geometric interpretation of the generalized fused lasso along with discussion of its persistency. Second, we analytically show its grouping property. Third, we give comprehensive simulation studies to compare our version of the generalized fused lasso with other existing methods and show that the proposed method outperforms other variable selection methods in terms of prediction error and parsimony. We describe two applications of our method in soil science and near infrared spectroscopy studies. These examples having vastly different data types demonstrate the flexibility of the methodology particularly for high-dimensional data.

Detecting Relevant Interactions in High Dimensional Data Analysis

In high dimensional data, relevant interactions can be difficult to identify due to the extremely large number of possible interactions among variables. Conventional methods use a screening stage to vastly reduce the dimension of the variable space before examining the interaction effect. However, this type of screening relies on the assumption that the relevant variables of interactions also have significantly strong marginal effects, which may not be the case in many applications. Hence we propose a multi-stage screening method that considers interaction effects to ensure that variables with weak marginal effect can still be identified if their interactions are strong. The proposed screening method can achieve the same reduction of dimensionality in linear computational time using a random sampling scheme. Experimental studies on three simulated scenarios, a financial news dataset and a biostatistics dataset show that the proposed approach can effectively identify variables that are of weak marginal effect but relevant interaction effect.

Similarity measure design for high dimensional data

Designing of similarity on high dimensional data was done. Similarity measure between high dimensional data was considered by analyzing neighbor information with respect to data sets. Obtained result could be applied to big data, because big data has multiple characteristics compared to simple data set. Definitely, analysis of high dimensional data could be the pre-study of big data. High dimensional data analysis was also compared with the conventional similarity. Traditional similarity measure on overlapped data was illustrated, and application to non-overlapped data was carried out. Its usefulness was proved by way of mathematical proof, and verified by calculation of similarity for artificial data example.

Bridge Estimation for Linear Regression Models with Mixing Properties

SummaryPenalized regression methods have for quite some time been a popular choice for addressing challenges in high dimensional data analysis. Despite their popularity, their application to time series data has been limited. This paper concerns bridge penalized methods in a linear regression time series model. We first prove consistency, sparsity and asymptotic normality of bridge estimators under a general mixing model. Next, as a special case of mixing errors, we consider bridge regression with autoregressive and moving average (ARMA) error models and develop a computational algorithm that can simultaneously select important predictors and the orders of ARMA models. Simulated and real data examples demonstrate the effective performance of the proposed algorithm and the improvement over ordinary bridge regression.