Efficient Expectation-maximization Algorithm Research Articles

Latent event history models for quasi-reaction systems

Various processes, such as cell differentiation and disease spreading, can be modelled as quasi-reaction systems of particles using stochastic differential equations. The existing Local Linear Approximation (LLA) method infers the parameters driving these systems from measurements of particle abundances over time. While dense observations of the process in time should in theory improve parameter estimation, LLA fails in these situations due to numerical instability. Defining a latent event history model of the underlying quasi-reaction system resolves this problem. A computationally efficient Expectation-Maximization algorithm is proposed for parameter estimation, incorporating an extended Kalman filter for evaluating the latent reactions. A simulation study demonstrates the method's performance and highlights the settings where it is particularly advantageous compared to the existing LLA approaches. An illustration of the method applied to the diffusion of COVID-19 in Italy is presented.

Regression analysis of group-tested current status data

Summary Group testing is an effective way to reduce the time and cost associated with conducting large-scale screening for infectious diseases. Benefits are realized through testing pools formed by combining specimens, such as blood or urine, from different individuals. In some studies, individuals are assessed only once and a time-to-event endpoint is recorded, for example, the time until infection. Combining group testing with this type of endpoint results in group-tested current status data (Petito & Jewell, 2016). To analyse these complex data, we propose methods that estimate a proportional hazard regression model based on test outcomes from measuring the pools. A sieve maximum likelihood estimation approach is developed that approximates the cumulative baseline hazard function with a piecewise constant function. To identify the sieve estimator, a computationally efficient expectation-maximization algorithm is derived by using data augmentation. Asymptotic properties of both the parametric and nonparametric components of the sieve estimator are then established by applying modern empirical process theory. Numerical results from simulation studies show that our proposed method performs nominally and has advantages over the corresponding estimation method based on individual testing results. We illustrate our work by analysing a chlamydia dataset collected by the State Hygienic Laboratory at the University of Iowa.

Genetic Underpinnings of Brain Structural Connectome for Young Adults

With distinct advantages in power over behavioral phenotypes, brain imaging traits have become emerging endophenotypes to dissect molecular contributions to behaviors and neuropsychiatric illnesses. Among different imaging features, brain structural connectivity (i.e., structural connectome) which summarizes the anatomical connections between different brain regions is one of the most cutting edge while under-investigated traits; and the genetic influence on the structural connectome variation remains highly elusive. Relying on a landmark imaging genetics study for young adults, we develop a biologically plausible brain network response shrinkage model to comprehensively characterize the relationship between high dimensional genetic variants and the structural connectome phenotype. Under a unified Bayesian framework, we accommodate the topology of brain network and biological architecture within the genome; and eventually establish a mechanistic mapping between genetic biomarkers and the associated brain sub-network units. An efficient expectation-maximization algorithm is developed to estimate the model and ensure computing feasibility. In the application to the Human Connectome Project Young Adult (HCP-YA) data, we establish the genetic underpinnings which are highly interpretable under functional annotation and brain tissue eQTL analysis, for the brain white matter tracts connecting the hippocampus and two cerebral hemispheres. We also show the superiority of our method in extensive simulations. Supplementary materials for this article are available online.

Dynamic Probabilistic Predictable Feature Analysis for Multivariate Temporal Process Monitoring

Dynamic statistical process monitoring methods have been widely studied and applied in modern industrial processes. These methods aim to extract the most predictable temporal information and develop the corresponding dynamic monitoring schemes. However, measurement noise is widespread in real-world industrial processes, and ignoring its effect will lead to suboptimal modeling and monitoring performance. In this article, a probabilistic predictable feature analysis (PPFA) is proposed for multivariate time series modeling, and a multistep dynamic predictive monitoring scheme is developed. The model parameters are estimated with an efficient expectation–maximization algorithm, where the genetic algorithm and the Kalman filter are designed and incorporated. Furthermore, a novel dynamic statistical monitoring index, the dynamic index, is proposed as an important supplement of T2 and SPE to detect dynamic anomalies. The effectiveness of the proposed algorithm is demonstrated via its application on the three-phase flow facility and a medium-speed coal mill.

Template Independent Component Analysis with Spatial Priors for Accurate Subject-Level Brain Network Estimation and Inference

Independent component analysis is commonly applied to functional magnetic resonance imaging (fMRI) data to extract independent components (ICs) representing functional brain networks. While ICA produces reliable group-level estimates, single-subject ICA often produces noisy results. Template ICA is a hierarchical ICA model using empirical population priors to produce more reliable subject-level estimates. However, this and other hierarchical ICA models assume unrealistically that subject effects are spatially independent. Here, we propose spatial template ICA (stICA), which incorporates spatial priors into the template ICA framework for greater estimation efficiency. Additionally, the joint posterior distribution can be used to identify brain regions engaged in each network using an excursions set approach. By leveraging spatial dependencies and avoiding massive multiple comparisons, stICA has high power to detect true effects. We derive an efficient expectation-maximization algorithm to obtain maximum likelihood estimates of the model parameters and posterior moments of the latent fields. Based on analysis of simulated data and fMRI data from the Human Connectome Project, we find that stICA produces estimates that are more accurate and reliable than benchmark approaches, and identifies larger and more reliable areas of engagement. The algorithm is computationally tractable, achieving convergence within 12 hr for whole-cortex fMRI analysis. Supplementary materials for this article are available online.

Quantile hidden semi-Markov models for multivariate time series.

This paper develops a quantile hidden semi-Markov regression to jointly estimate multiple quantiles for the analysis of multivariate time series. The approach is based upon the Multivariate Asymmetric Laplace (MAL) distribution, which allows to model the quantiles of all univariate conditional distributions of a multivariate response simultaneously, incorporating the correlation structure among the outcomes. Unobserved serial heterogeneity across observations is modeled by introducing regime-dependent parameters that evolve according to a latent finite-state semi-Markov chain. Exploiting the hierarchical representation of the MAL, inference is carried out using an efficient Expectation-Maximization algorithm based on closed form updates for all model parameters, without parametric assumptions about the states’ sojourn distributions. The validity of the proposed methodology is analyzed both by a simulation study and through the empirical analysis of air pollutant concentrations in a small Italian city.Supplementary InformationThe online version contains supplementary material available at 10.1007/s11222-022-10130-1.

A destructive shifted Poisson cure model for interval censored data and an efficient estimation algorithm

In this paper, we consider a competitive risk scenario in which the initial number of competing risks, assumed to follow a shifted Poisson distribution, is subject to a destructive element. In such settings, both the number of initial risks and risks remaining active after destruction represent missing data. Assuming the population of interest to have a cure component and the form of the data as interval-censored, we extend the destructive shifted Poisson cure model with Weibull lifetimes to accommodate interval-censored data and develop an efficient expectation maximization algorithm for this model. By using the conditional distributions of the missing data, the conditional expected complete log-likelihood function is decomposed into simpler functions which are then maximized independently. An extensive simulation study is carried out to demonstrate the performance of the proposed estimation method through calculated bias, root mean square error, and coverage probability of the asymptotic confidence interval. Also considered is the efficacy of the proposed EM algorithm as compared to direct maximization of the observed log-likelihood function. Finally, data from a crime recidivism study is analyzed for illustrative purpose.

Joint dimension reduction and clustering analysis of single-cell RNA-seq and spatial transcriptomics data.

Dimension reduction and (spatial) clustering is usually performed sequentially; however, the low-dimensional embeddings estimated in the dimension-reduction step may not be relevant to the class labels inferred in the clustering step. We therefore developed a computation method, Dimension-Reduction Spatial-Clustering (DR-SC), that can simultaneously perform dimension reduction and (spatial) clustering within a unified framework. Joint analysis by DR-SC produces accurate (spatial) clustering results and ensures the effective extraction of biologically informative low-dimensional features. DR-SC is applicable to spatial clustering in spatial transcriptomics that characterizes the spatial organization of the tissue by segregating it into multiple tissue structures. Here, DR-SC relies on a latent hidden Markov random field model to encourage the spatial smoothness of the detected spatial cluster boundaries. Underlying DR-SC is an efficient expectation-maximization algorithm based on an iterative conditional mode. As such, DR-SC is scalable to large sample sizes and can optimize the spatial smoothness parameter in a data-driven manner. With comprehensive simulations and real data applications, we show that DR-SC outperforms existing clustering and spatial clustering methods: it extracts more biologically relevant features than conventional dimension reduction methods, improves clustering performance, and offers improved trajectory inference and visualization for downstream trajectory inference analyses.

Reliability analysis for multi-component systems considering stochastic dependency based on factor analysis

Reliability analysis for engineering systems with multiple components has gained increasing interest in recent. Most existing works assume that components follow identical degradation models and preset joint distributions or link functions to characterize the degradation interactions. However, distinct degradation characteristics of components are commonly observed in practice. Besides, the degradation interactions are usually complex and diverse, making the preset dependency structures not applicable. Confronted with the diverse degradation characteristics and complex degradation interactions, this paper offers a flexible reliability analysis framework for multi-component systems. First, a stochastic process-based general degradation model combining the Wiener process, Gamma process, and inverse Gaussian process is adopted to describe component degradation processes, and the factor analysis is employed to characterize the degradation interactions by seeking the latent common factors that dominate their interdependency. Thus the assumption of the identical degradation process and preset dependency structure can be relaxed, which enhances the robustness of the method. On this basis, we derive the explicit form of the system reliability function. An efficient Expectation-Maximization algorithm is then utilized for statistical inference to enable a fast computation. Finally, the superiority of the proposed method is demonstrated by two real case studies on lithium-ion battery packs.

Cutting the Unnecessary Long Tail: Cost-Effective Big Data Clustering in the Cloud

Clustering big data often requires tremendous computational resources where cloud computing is undoubtedly one of the promising solutions. However, the computation cost in the cloud can be unexpectedly high if it cannot be managed properly. The long tail phenomenon has been observed widely in the big data clustering area, which indicates that the majority of time is often consumed in the middle to late stages in the clustering process. In this research, we try to cut the unnecessary long tail in the clustering process to achieve a sufficiently satisfactory accuracy at the lowest possible computation cost. A novel approach is proposed to achieve cost-effective big data clustering in the cloud. By training the regression model with the sampling data, we can make widely used k-means and EM (Expectation-Maximization) algorithms stop automatically at an early point when the desired accuracy is obtained. Experiments are conducted on four popular data sets and the results demonstrate that both k-means and EM algorithms can achieve high cost-effectiveness in the cloud with our proposed approach. For example, in the case studies with the much more efficient k-means algorithm, we find that achieving a 99% accuracy needs only 47.71%-71.14% of the computation cost required for achieving a 100% accuracy while the less efficient EM algorithm needs 16.69%-32.04% of the computation cost. To put that into perspective, in the United States land use classification example, our approach can save up to $94,687.49 for the government in each use.

Inference for One-Shot Devices with Dependent k-Out-of-M Structured Components under Gamma Frailty

A device that performs its intended function only once is referred to as a one-shot device. Actual lifetimes of such kind of devices under test cannot be observed, and they are either left-censored or right-censored. In addition, one-shot devices often consist of multiple components that could cause the failure of the device. The components are coupled together in the manufacturing process or assembly, resulting in the failure modes possessing latent heterogeneity and dependence. In this paper, we develop an efficient expectation–maximization algorithm for determining the maximum likelihood estimates of model parameters, on the basis of one-shot device test data with multiple failure modes under a constant-stress accelerated life-test, with the dependent components having exponential lifetime distributions under gamma frailty that facilitates an easily understandable interpretation. The maximum likelihood estimate and confidence intervals for the mean lifetime of k-out-of-M structured one-shot device under normal operating conditions are also discussed. The performance of the proposed inferential methods is finally evaluated through Monte Carlo simulations. Three examples including Class-H failure modes data, mice data from ED01 experiment, and simulated data with four failure modes are used to illustrate the proposed inferential methods.

SC-MEB: spatial clustering with hidden Markov random field using empirical Bayes.

Spatial transcriptomics has been emerging as a powerful technique for resolving gene expression profiles while retaining tissue spatial information. These spatially resolved transcriptomics make it feasible to examine the complex multicellular systems of different microenvironments. To answer scientific questions with spatial transcriptomics and expand our understanding of how cell types and states are regulated by microenvironment, the first step is to identify cell clusters by integrating the available spatial information. Here, we introduce SC-MEB, an empirical Bayes approach for spatial clustering analysis using a hidden Markov random field. We have also derived an efficient expectation-maximization algorithm based on an iterative conditional mode for SC-MEB. In contrast to BayesSpace, a recently developed method, SC-MEB is not only computationally efficient and scalable to large sample sizes but is also capable of choosing the smoothness parameter and the number of clusters. We performed comprehensive simulation studies to demonstrate the superiority of SC-MEB over some existing methods. We applied SC-MEB to analyze the spatial transcriptome of human dorsolateral prefrontal cortex tissues and mouse hypothalamic preoptic region. Our analysis results showed that SC-MEB can achieve a similar or better clustering performance to BayesSpace, which uses the true number of clusters and a fixed smoothness parameter. Moreover, SC-MEB is scalable to large ‘sample sizes’. We then employed SC-MEB to analyze a colon dataset from a patient with colorectal cancer (CRC) and COVID-19, and further performed differential expression analysis to identify signature genes related to the clustering results. The heatmap of identified signature genes showed that the clusters identified using SC-MEB were more separable than those obtained with BayesSpace. Using pathway analysis, we identified three immune-related clusters, and in a further comparison, found the mean expression of COVID-19 signature genes was greater in immune than non-immune regions of colon tissue. SC-MEB provides a valuable computational tool for investigating the structural organizations of tissues from spatial transcriptomic data.

Generative model for reciprocity and community detection in networks

We present a probabilistic generative model and efficient algorithm to model reciprocity in directed networks. Unlike other methods that address this problem such as exponential random graphs, it assigns latent variables as community memberships to nodes and a reciprocity parameter to the whole network rather than fitting order statistics. It formalizes the assumption that a directed interaction is more likely to occur if an individual has already observed an interaction towards her. It provides a natural framework for relaxing the common assumption in network generative models of conditional independence between edges, and it can be used to perform inference tasks such as predicting the existence of an edge given the observation of an edge in the reverse direction. Inference is performed using an efficient expectation-maximization algorithm that exploits the sparsity of the network, leading to an efficient and scalable implementation. We illustrate these findings by analyzing synthetic and real data, including social networks, academic citations and the Erasmus student exchange program. Our method outperforms others in both predicting edges and generating networks that reflect the reciprocity values observed in real data, while at the same time inferring an underlying community structure. We provide an open-source implementation of the code online.

ScAMACE: model-based approach to the joint analysis of single-cell data on chromatin accessibility, gene expression and methylation.

The advancement in technologies and the growth of available single-cell datasets motivate integrative analysis of multiple single-cell genomic datasets. Integrative analysis of multimodal single-cell datasets combines complementary information offered by single-omic datasets and can offer deeper insights on complex biological process. Clustering methods that identify the unknown cell types are among the first few steps in the analysis of single-cell datasets, and they are important for downstream analysis built upon the identified cell types. We propose scAMACE for the integrative analysis and clustering of single-cell data on chromatin accessibility, gene expression and methylation. We demonstrate that cell types are better identified and characterized through analyzing the three data types jointly. We develop an efficient Expectation-Maximization algorithm to perform statistical inference, and evaluate our methods on both simulation study and real data applications. We also provide the GPU implementation of scAMACE, making it scalable to large datasets. The software and datasets are available at https://github.com/cuhklinlab/scAMACE_py (python implementation) and https://github.com/cuhklinlab/scAMACE (R implementation). Supplementary data are available at Bioinformatics online.

Integrative learning for population of dynamic networks with covariates

Although there is a rapidly growing literature on dynamic connectivity methods, the primary focus has been on separate network estimation for each individual, which fails to leverage common patterns of information. We propose novel graph-theoretic approaches for estimating a population of dynamic networks that are able to borrow information across multiple heterogeneous samples in an unsupervised manner and guided by covariate information. Specifically, we develop a Bayesian product mixture model that imposes independent mixture priors at each time scan and uses covariates to model the mixture weights, which results in time-varying clusters of samples designed to pool information. The computation is carried out using an efficient Expectation-Maximization algorithm. Extensive simulation studies illustrate sharp gains in recovering the true dynamic network over existing dynamic connectivity methods. An analysis of fMRI block task data with behavioral interventions reveal sub-groups of individuals having similar dynamic connectivity, and identifies intervention-related dynamic network changes that are concentrated in biologically interpretable brain regions. In contrast, existing dynamic connectivity approaches are able to detect minimal or no changes in connectivity over time, which seems biologically unrealistic and highlights the challenges resulting from the inability to systematically borrow information across samples.

Zero-Inflated gaussian mixed models for analyzing longitudinal microbiome data

MotivationThe human microbiome is variable and dynamic in nature. Longitudinal studies could explain the mechanisms in maintaining the microbiome in health or causing dysbiosis in disease. However, it remains challenging to properly analyze the longitudinal microbiome data from either 16S rRNA or metagenome shotgun sequencing studies, output as proportions or counts. Most microbiome data are sparse, requiring statistical models to handle zero-inflation. Moreover, longitudinal design induces correlation among the samples and thus further complicates the analysis and interpretation of the microbiome data.ResultsIn this article, we propose zero-inflated Gaussian mixed models (ZIGMMs) to analyze longitudinal microbiome data. ZIGMMs is a robust and flexible method which can be applicable for longitudinal microbiome proportion data or count data generated with either 16S rRNA or shotgun sequencing technologies. It can include various types of fixed effects and random effects and account for various within-subject correlation structures, and can effectively handle zero-inflation. We developed an efficient Expectation-Maximization (EM) algorithm to fit the ZIGMMs by taking advantage of the standard procedure for fitting linear mixed models. We demonstrate the computational efficiency of our EM algorithm by comparing with two other zero-inflated methods. We show that ZIGMMs outperform the previously used linear mixed models (LMMs), negative binomial mixed models (NBMMs) and zero-inflated Beta regression mixed model (ZIBR) in detecting associated effects in longitudinal microbiome data through extensive simulations. We also apply our method to two public longitudinal microbiome datasets and compare with LMMs and NBMMs in detecting dynamic effects of associated taxa.

Dimension reduction in binary response regression: A joint modeling approach

Categorical responses cause no conceptual complications for dimension reduction in regression, but the performance of some methods may suffer in this context and hence supervised dimension reduction in practice must recognize the nature of the response. Using a continuous latent variable to represent an unobserved response underlying the binary response, a joint model is proposed for dimension reduction in binary regression. The minimal sufficient linear reduction is obtained, and an efficient expectation maximization algorithm is developed for carrying out maximum likelihood estimation. Simulated examples and an application to a dataset concerning the identification of handwritten digits are presented to compare the performance of the proposed method with that of existing methods.

The conditional censored graphical lasso estimator

In many applied fields, such as genomics, different types of data are collected on the same system, and it is not uncommon that some of these datasets are subject to censoring as a result of the measurement technologies used, such as data generated by polymerase chain reactions and flow cytometer. When the overall objective is that of network inference, at possibly different levels of a system, information coming from different sources and/or different steps of the analysis can be integrated into one model with the use of conditional graphical models. In this paper, we develop a doubly penalized inferential procedure for a conditional Gaussian graphical model when data can be subject to censoring. The computational challenges of handling censored data in high dimensionality are met with the development of an efficient Expectation-Maximization algorithm, based on approximate calculations of the moments of truncated Gaussian distributions and on a suitably derived two-step procedure alternating graphical lasso with a novel block-coordinate multivariate lasso approach. We evaluate the performance of this approach on an extensive simulation study and on gene expression data generated by RT-qPCR technologies, where we are able to integrate network inference, differential expression detection and data normalization into one model.

Estimation of treatment effects and model diagnostics with two-way time-varying treatment switching: an application to a head and neck study.

Treatment switching frequently occurs in clinical trials due to ethical reasons. Intent-to-treat analysis without adjusting for switching yields biased and inefficient estimates of the treatment effects. In this paper, we propose a class of semiparametric semi-competing risks transition survival models to accommodate two-way time-varying switching. Theoretical properties of the proposed method are examined. An efficient expectation-maximization algorithm is derived to obtain maximum likelihood estimates and model diagnostic tools. Existing software is used to implement the algorithm. Simulation studies are conducted to demonstrate the validity of the model. The proposed method is further applied to data from a clinical trial with patients having recurrent or metastatic squamous-cell carcinoma of head and neck.

Identification of Structural Vector Autoregressions by Stochastic Volatility

We propose to exploit stochastic volatility for statistical identification of structural vector autoregressive models (SV-SVAR). We discuss full and partial identification of the model and develop efficient Expectation Maximization algorithms for Maximum Likelihood inference. Simulation evidence suggests that, compared to alternative models, the SV-SVAR works well in identifying structural parameters also under misspecification of the variance process. We apply the model to study the importance of oil supply shocks for driving oil prices. Since shocks identified by heteroskedasticity may not be economically meaningful, we exploit the framework to test instrumental variable restrictions which are overidentifying in the heteroskedastic model. Our findings suggest that conventional supply shocks are negligible drivers of oil prices, while news shocks about future supply account for almost all the variation.