Graphical Methods Of Data Analysis Research Articles

Convergence rates of Metropolis–Hastings algorithms

AbstractGiven a target probability density known up to a normalizing constant, the Metropolis–Hastings algorithm simulates realizations from a Markov chain which are eventual realizations from the target probability density. A key element for ensuring a reliable Metropolis–Hastings simulation experiment is understanding how quickly the simulation will generate a representative sample from target density. This corresponds to understanding the convergence properties of the Metropolis–Hastings Markov chain. State‐of‐the‐art methods for convergence analysis of Metropolis–Hastings algorithms are considered and reviewed. Practically important topics are discussed for an interdisciplinary audience. This includes convergence properties in high dimensions, proper tuning, initialization, and limitations of current convergence analyses.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Markov Chain Monte Carlo Statistical and Graphical Methods of Data Analysis > Monte Carlo Methods Statistical and Graphical Methods of Data Analysis > Bayesian Methods and Theory

Robust clustering based on trimming

AbstractClustering is one of the most widely used unsupervised learning techniques. However, it is well‐known that outliers can have a significantly adverse impact on commonly applied clustering methods. On the other hand, clustered outliers can be particularly detrimental to (even robust) statistical procedures. Therefore, it makes sense to combine concepts from Robust Statistics and Cluster Analysis to deal with both clusters and outliers simultaneously through robust clustering approaches. Among the existing robust clustering techniques, we focus on those that rely on (impartial) trimming. Trimming offers the user an easy interpretation, as standard well‐known clustering methods are applied after a fraction of the potentially most outlying observations is removed. This trimming approach, when combined with appropriate constraints on the clusters' dispersion parameters, has shown a good performance and can be implemented efficiently thorough available algorithms.This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification Statistical and Graphical Methods of Data Analysis > Robust Methods

Highly accelerated life testing (HALT): A review from a statistical perspective

AbstractDespite its use in one form or another for at least four decades, HALT and related techniques [e.g., highly accelerated‐stress screening (HASS) and stress audits (HASA)] are not well understood within the statistical community and remain controversial. This largely reflects a conflict in motivation between engineers, testing under harsh conditions to discover and eliminate failure modes, and statisticians, taking a more cautious approach to develop quantitative estimates of parameters such as mean time between failures (MTBF). This review article will clarify HALT concepts and methods and explain where it fits within the universe of methods that involve the application of accelerating factors to compress the time required to evaluate or enhance product reliability. A major distinction is between methods such as HALT, a high‐stress test‐analyze‐fix‐test iterative process directed at improving reliability by discovering and fixing weak points in a design, and quantitative accelerated life testing (QALT), whose goal is the estimation of product life for a fixed design. We discuss methods such as physics of failure that offer some hope of bridging the gap between the qualitative nature of HALT, and purely quantitative statistical methods. We present a variety of engineering applications of HALT including metal fatigue, piping and pressure vessels, structural damage, radiation damage, and rotating machinery. We also discuss potential synergies between HALT and QALT, such as rapid identification, through HALT, of failure modes requiring quantitative analysis. For further study, extensive references to the applicable literature are provided as well as an appendix that describes related methods.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Reliability, Survivability, and Quality Control

The discrete empirical interpolation method in class identification and data summarization

AbstractThe discrete empirical interpolation method (DEIM) is well established as a means of performing model order reduction in approximating solutions to differential equations, but it has also more recently demonstrated potential in performing data class detection through subset selection. Leveraging the singular value decomposition (SVD) for dimension reduction, DEIM uses interpolatory projection to identify the representative rows and/or columns of a data matrix. This approach has been adapted to develop additional algorithms, including a CUR matrix factorization for performing dimension reduction while preserving the interpretability of the data. DEIM‐oversampling techniques have also been developed expressly for the purpose of index selection in identifying more DEIM representatives than would typically be allowed by the matrix rank. Even with these developments, there is still a relatively large gap in the literature regarding the use of DEIM in performing unsupervised learning tasks to analyze large datasets. Known examples of DEIM's demonstrated applicability include contexts such as physics‐based modeling/monitoring, electrocardiogram data summarization and classification, and document term subset selection. This overview presents a description of DEIM and some DEIM‐related algorithms, discusses existing results from the literature with an emphasis on more statistical‐learning‐based tasks, and identifies areas for further exploration moving forward.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Dimension Reduction Statistical Learning and Exploratory Methods of the Data Sciences > Exploratory Data Analysis Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification

Modeling multivariate extremes

AbstractThe Central Limit Theorem justifies using a normal distribution when looking at sums of many terms. In a parallel way, extreme value distributions arise in the study of maxima of many terms. The goal of this paper is to briefly review the univariate theory of extremes based on the Fisher‐Tippet‐Gnedenko Theorem. We then state the basics of the multivariate theory, which is significantly more complicated because it requires a measure to define the distribution. Some properties of these laws are explored, including a description of the support, an expression for the density when it exists, and some examples that illustrate possible joint dependence structures.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Multivariate Analysis Applications of Computational Statistics > Computational Finance

Recent advances on mechanisms of network generation: Community, exchangeability, and scale‐free properties

AbstractThe mechanisms of network generation have undergone extensive analysis and found broad applications in various real‐world scenarios. Among the fruitful literature on network models, numerous studies seek to explore and interpret fundamental graph structure properties, including the clustering effect, exchangeability, and scale‐free properties. In this paper, we present a comprehensive review of the statistical modeling methods for the mechanisms of network generation. We specifically focus on three representative classes of models, namely the stochastic block models, the exchangeable network models, and the preferential attachment models. For each model type, our approach begins by reviewing existing methods and model setups, followed by an exploration of the core modeling principles behind them. We also summarize relevant statistical inference techniques and provide a unified understanding of theoretical analyses. Furthermore, we emphasize several challenges and open problems that could shed light on future research. We conclude this review with the identification of some possible directions for future study.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Modeling Methods and Algorithms Algorithms and Computational Methods > Networks and Security

Biomarker data with measurement error in medical research: A literature review.

A biomarker is a measurable indicator of the severity or presence of a disease or medical condition in biomedical or epidemiological research. Biomarkers may help in early diagnosis and prevention of diseases. Several biomarkers have been identified for many diseases such as carbohydrate antigen 19-9 for pancreatic cancer. However, biomarkers may be measured with errors due to many reasons such as specimen collection or day-to-day within-subject variability of the biomarker, among others. Measurement error in the biomarker leads to bias in the regression parameter estimation for the association of the biomarker with disease in epidemiological studies. In addition, measurement error in the biomarkers may affect standard diagnostic measures to evaluate the performance of biomarkers such as the receiver operating characteristic (ROC) curve, area under the ROC curve, sensitivity, and specificity. Measurement error may also have an effect on how to combine multiple cancer biomarkers as a composite predictor for disease diagnosis. In follow-up studies, biomarkers are often collected intermittently at examination times, which may be sparse and typically biomarkers are not observed at the event times. Joint modeling of longitudinal and time-to-event data is a valid approach to account for measurement error in the analysis of repeatedly measured biomarkers and time-to-event outcomes. In this article, we provide a literature review on existing methods to correct for estimation in regression analysis, diagnostic measures, and joint modeling of longitudinal biomarkers and survival outcomes when the biomarkers are measured with errors. This article is categorized under: Statistical and Graphical Methods of Data Analysis > Robust MethodsStatistical and Graphical Methods of Data Analysis > EM AlgorithmStatistical Models > Survival Models.

From RNA sequencing measurements to the final results: A practical guide to navigating the choices and uncertainties of gene set analysis

AbstractGene set analysis (GSA), a popular approach for analyzing high‐throughput gene expression data, aims to identify sets of related genes that show significantly enriched or depleted expression patterns between different conditions. In the last years, a multitude of methods have been developed for this task. However, clear guidance is lacking: choosing the right method is the first hurdle a researcher is confronted with. No less challenging than overcoming this so‐called method uncertainty is the procedure of preprocessing, from knowing which steps are required to selecting a corresponding approach from the plethora of valid options to create the accepted input object (data preprocessing uncertainty), with clear guidance again being scarce. Here, we provide a practical guide through all steps required to conduct GSA, beginning with a concise overview of a selection of established methods, including Gene Set Enrichment Analysis and Database for Annotation, Visualization, and Integrated Discovery (DAVID). We thereby lay a special focus on reviewing and explaining the necessary preprocessing steps for each method under consideration (e.g., the necessity of a transformation of the RNA sequencing data)—an essential aspect that is typically paid only limited attention to in both existing reviews and applications. To raise awareness of the spectrum of uncertainties, our review is accompanied by an extensive overview of the literature on valid approaches for each step and illustrative R code demonstrating the complex analysis pipelines. It ends with a discussion and recommendations to both users and developers to ensure that the results of GSA are, despite the above‐mentioned uncertainties, replicable and transparent.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Analysis of High Dimensional Data Data: Types and Structure > Data Preparation and Processing Applications of Computational Statistics > Genomics/Proteomics/Genetics

Learning the sparse prior: Modern approaches

AbstractThe sparse prior has been widely adopted to establish data models for numerous applications. In this context, most of them are based on one of three foundational paradigms: the conventional sparse representation, the convolutional sparse representation, and the multi‐layer convolutional sparse representation. When the data morphology has been adequately addressed, a sparse representation can be obtained by solving the sparse coding problem specified by the data model. This article presents a comprehensive overview of these three models and their corresponding sparse coding problems and demonstrates that they can be solved using convex and non‐convex optimization approaches. When the data morphology is not known or cannot be analyzed, it must be learned from training data, thereby formulating dictionary learning problems. This article addresses two different dictionary learning paradigms. In an unsupervised scenario, dictionary learning involves the alternating or joint resolution of sparse coding and dictionary updating. Another option is to create a recurrent neural network by unrolling algorithms designed to solve sparse coding problems. These networks can then be used in a supervised learning setting to facilitate the training of dictionaries via forward‐backward optimization. This article lists numerous applications in various domains and outlines several directions for future research related to the sparse prior.This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods Statistical and Graphical Methods of Data Analysis > Modeling Methods and Algorithms Statistical Models > Nonlinear Models

A journey from univariate to multivariate functional time series: A comprehensive review

AbstractFunctional time series (FTS) analysis has emerged as a potent framework for modeling and forecasting time‐dependent data with functional attributes. In this comprehensive review, we navigate through the intricate landscape of FTS methodologies, meticulously surveying the core principles of univariate FTS and delving into the nuances of multivariate FTS. The journey commences with an exploration of the foundational aspects of univariate FTS analysis. We delve into representation, estimation, and modeling, spotlighting the effectiveness of various parametric and nonparametric models at our disposal. The stage then transitions to multivariate FTS analysis, where we confront the intricacies posed by high‐dimensional data. We explore strategies for dimensionality reduction, forecasting, and the integration of diverse parametric and nonparametric models within the multivariate realm. We also highlight commonly used R packages for modeling and forecasting FTS and multivariate FTS. Acknowledging the dynamic evolution of the field, we dissect challenges and chart future directions, paving a course for refinement and innovation. Through a fusion of multivariate statistics, functional analysis, and time series forecasting, this review underscores the interdisciplinary essence of FTS analysis. It not only reveals past accomplishments, but also illuminates the potential of FTS in unraveling insights and facilitating well‐informed decisions across diverse domains.This article is categorized under: Data: Types and Structure > Time Series, Stochastic Processes, and Functional Data Statistical and Graphical Methods of Data Analysis > Multivariate Analysis

Advancing microdata privacy protection: A review of synthetic data methods

AbstractSynthetic data generation is a powerful tool for privacy protection when considering public release of record‐level data files. Initially proposed about three decades ago, it has generated significant research and application interest. To meet the pressing demand of data privacy protection in a variety of contexts, the field needs more researchers and practitioners. This review provides a comprehensive introduction to synthetic data, including technical details of their generation and evaluation. Our review also addresses the challenges and limitations of synthetic data, discusses practical applications, and provides thoughts for future work.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Modeling Methods and Algorithms

A review of Monte Carlo and quasi‐Monte Carlo sampling techniques

AbstractThis article presents a comprehensive review and comparison of the Monte Carlo and quasi‐Monte Carlo sampling techniques, which are widely used in numerical integration, simulation, and optimization. Monte Carlo sampling involves the generation of pseudorandom numbers or vectors to estimate unknown quantities of interest. In contrast, quasi‐Monte Carlo sampling is specialized for situations where uniformity and reduced variance are important. It generates a deterministic low‐discrepancy sequence that spans the entire sampling space. This review aims to analyze the strengths and distinctions of these two sampling methodologies, offering valuable insights to researchers in search of sampling techniques aligned with their specific research objectives and needs. Furthermore, it seeks to equip practitioners with efficient algorithms for practical implementations.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Monte Carlo Methods Algorithms and Computational Methods > Numerical Methods Statistical and Graphical Methods of Data Analysis > Sampling

Unsupervised clustering using nonparametric finite mixture models

AbstractThis article presents basic ideas of finite mixture models in which the number of components is known and the distributions comprising the components are not assumed to come from any parametrically specified family.This article is categorized under: Algorithms and Computational Methods > Algorithms Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification Statistical and Graphical Methods of Data Analysis > Nonparametric Methods Statistical Models > Classification Models

A systematic review of exploratory factor analysis packages in R software

AbstractThe increasing prevalence of exploratory factor analysis (EFA) applications in scholarly literature reflects its popularity and the convenience of computer‐assisted analysis. With advancements in computer hardware and software, the complexity and variations of EFA analysis have also grown. Despite the availability of sophisticated computer programming, the appropriate utilization of EFA necessitates users to make informed judgments. Additionally, users are responsible for searching and identifying suitable statistical software to accommodate their data and analysis requirements. This review aims to enhance understanding of the EFA technique and summarize the analysis options available for EFA in R packages. A total of 50 packages were examined in this study. Specifically, the review focuses on (1) diagnostic functions, (2) factor extraction, (3) factor retention, (4) factor rotation, and (5) complex data and technique features provided by these packages. The review summarizes the available function options in R packages by outlining these five crucial steps in conducting an EFA analysis. This synthesis offers an overview of the similarities and distinctive features of each package, serving as a valuable resource for users in selecting a suitable EFA technique. It is important to note that there is no definitive approach to conducting an exploratory factor analysis. Users need to deliberately select and combine appropriate techniques to achieve optimal results.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Dimension Reduction Software for Computational Statistics > Software/Statistical Software

Prediction approaches for partly missing multi‐omics covariate data: A literature review and an empirical comparison study

AbstractAs the availability of omics data has increased in the last few years, more multi‐omics data have been generated, that is, high‐dimensional molecular data consisting of several types such as genomic, transcriptomic, or proteomic data, all obtained from the same patients. Such data lend themselves to being used as covariates in automatic outcome prediction because each omics type may contribute unique information, possibly improving predictions compared to using only one omics data type. Frequently, however, in the training data and the data to which automatic prediction rules should be applied, the test data, the different omics data types are not available for all patients. We refer to this type of data as block‐wise missing multi‐omics data. First, we provide a literature review on existing prediction methods applicable to such data. Subsequently, using a collection of 13 publicly available multi‐omics data sets, we compare the predictive performances of several of these approaches for different block‐wise missingness patterns. Finally, we discuss the results of this empirical comparison study and draw some tentative conclusions.This article is categorized under: Applications of Computational Statistics > Genomics/Proteomics/Genetics Applications of Computational Statistics > Health and Medical Data/Informatics Statistical and Graphical Methods of Data Analysis > Analysis of High Dimensional Data

Acceleration of the EM algorithm

AbstractThe expectation–maximization (EM) algorithm is a well‐known iterative algorithm for finding maximum likelihood estimates from incomplete data and is used in several statistical models with latent variables and missing data. The algorithm also exhibits a monotonic increase in a likelihood function and satisfies parameter constraints for its convergence. The popularity of the EM algorithm can be attributed to its stable convergence, simple implementation and flexibility in interpreting data incompleteness. Despite these computational advantages, the algorithm is linear convergent and suffers from very slow convergence when a statistical model has many parameters and a high proportion of missing data. Various algorithms have been proposed to accelerate the convergence of the EM algorithm. We introduce the acceleration of the EM algorithm using root‐finding and vector extrapolation algorithms. The root‐finding algorithms include Aitken's method and the Newton–Raphson, quasi‐Newton and conjugate gradient algorithms. These algorithms with faster convergence rates allow the EM algorithm to be sped up. The vector extrapolation algorithms transform the sequence of estimates from the EM algorithm into a fast convergent sequence and can accelerate the convergence without modifying the EM algorithm. We describe the derivation of these acceleration algorithms and attempt to apply them to two examples.This article is categorized under: Statistical and Graphical Methods of Data Analysis > EM Algorithm

On calibrated inverse probability weighting and generalized boosting propensity score models for mean estimation with incomplete survey data

AbstractIncomplete data, whether realized from nonresponse in survey data or counterfactual outcomes in observational studies, may lead to biased estimation of study variables. Nonresponse and selection bias may be mitigated with techniques that weight the incomplete data to match characteristics of the partially unobserved complete data. Inverse probability weighting is a widely used method in causal inference that relies on a propensity model to construct adjusted weights; whereas calibration is a common method used by survey statisticians to use constrained optimization to construct adjusted weights. This article reviews inverse probability weighting and entropy balancing calibration by distinguishing them in the statistical sense of variable balancing, extending propensity score construction to include generalized boosting models, and demonstrating the use of inverse probability weighting and calibration separately and together through a widely cited simulation study evaluation.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Sampling Data: Types and Structure > Categorical Data Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods

Neuroimaging statistical approaches for determining neural correlates of Alzheimer's disease via positron emission tomography imaging

AbstractAlzheimer's disease (AD) is a degenerative disorder involving significant memory loss and other cognitive deficits, manifesting as a progression from normal cognitive functioning to mild cognitive impairment to AD. The sooner an accurate diagnosis of probable AD is made, the easier it is to manage symptoms and plan for future therapy. Functional neuroimaging stands to be a useful tool in achieving early diagnosis. Among the many neuroimaging modalities, positron emission tomography (PET) provides direct regional assessment of, among others, brain metabolism, cerebral blood flow, amyloid deposition—all quantities of interest in the characterization of AD. However, there are analytic challenges in identifying early indicators of AD from these high‐dimensional imaging data sets, and it is unclear whether early indicators of AD are more likely to emerge in localized patterns of brain activity or in patterns of correlation between distinct brain regions. Early PET‐based analyses of AD focused on alterations in metabolic activity at the voxel‐level or in anatomically defined regions of interest. Other approaches, including seed‐voxel and multivariate techniques, seek to characterize metabolic connectivity by identifying other regions in the brain with similar patterns of activity across subjects. We briefly review various neuroimaging statistical approaches applied to determine changes in metabolic activity or metabolic connectivity associated with AD. We then present an approach that provides a unified statistical framework for addressing both metabolic activity and connectivity. Specifically, we apply a Bayesian spatial hierarchical framework to longitudinal metabolic PET scans from the Alzheimer's Disease Neuroimaging Initiative.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Analysis of High Dimensional Data Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods Statistical Models > Bayesian Models

Information criteria for model selection

AbstractThe rapid development of modeling techniques has brought many opportunities for data‐driven discovery and prediction. However, this also leads to the challenge of selecting the most appropriate model for any particular data task. Information criteria, such as the Akaike information criterion (AIC) and Bayesian information criterion (BIC), have been developed as a general class of model selection methods with profound connections with foundational thoughts in statistics and information theory. Many perspectives and theoretical justifications have been developed to understand when and how to use information criteria, which often depend on particular data circumstances. This review article will revisit information criteria by summarizing their key concepts, evaluation metrics, fundamental properties, interconnections, recent advancements, and common misconceptions to enrich the understanding of model selection in general.This article is categorized under: Data: Types and Structure > Traditional Statistical Data Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods Statistical and Graphical Methods of Data Analysis > Information Theoretic Methods Statistical Models > Model Selection

Sampling constrained continuous probability distributions: A review

AbstractThe problem of sampling constrained continuous distributions has frequently appeared in many machine/statistical learning models. Many Markov Chain Monte Carlo (MCMC) sampling methods have been adapted to handle different types of constraints on random variables. Among these methods, Hamilton Monte Carlo (HMC) and the related approaches have shown significant advantages in terms of computational efficiency compared with other counterparts. In this article, we first review HMC and some extended sampling methods, and then we concretely explain three constrained HMC‐based sampling methods, reflection, reformulation, and spherical HMC. For illustration, we apply these methods to solve three well‐known constrained sampling problems, truncated multivariate normal distributions, Bayesian regularized regression, and nonparametric density estimation. In this review, we also connect constrained sampling with another similar problem in the statistical design of experiments with constrained design space.This article is categorized under: Applications of Computational Statistics > Computational Mathematics Statistical and Graphical Methods of Data Analysis > Bayesian Methods and Theory Statistical and Graphical Methods of Data Analysis > Markov Chain Monte Carlo (MCMC) Statistical and Graphical Methods of Data Analysis > Sampling