Additional Latent Variables Research Articles

Interpretable machine learning model using Extreme Gradient Boosting with Nonnegative Matrix Factorization improves the accuracy of arrhythmic risk prediction in Brugada Syndrome

Abstract Background Both depolarization and repolarization abnormalities contribute to ventricular arrhythmogenesis in Brugada syndrome (BrS). In this study, we tested the hypothesis that incorporating latent features extracted by various nonnegative matrix factorization (NML) techniques into an interpretable machine learning (IML) prediction model can outperform IML models without latent variables and logistic regression model. Methods This study was based on a published anonymised dataset of BrS patients from the Hong Kong, China. XGBoost was selected as the IML model, with and without incorporating latent features using 11 different NMF techniques: Bayesian nonnegative matrix factorization (BNMF), Iterated Conditional Modes nonnegative matrix factorization (ICM), Fisher Nonnegative Matrix Factorization for learning Local features (LFNMF), Alternating Nonnegative Least Squares Matrix Factorization Using Projected Gradient (bound constrained optimization) method for each subproblem (LSNMF), Non-smooth Nonnegative Matrix Factorization (NSNMF), Probabilistic Nonnegative Matrix Factorization (PMF), Probabilistic Sparse Matrix Factorization (PSMF), Sparse Nonnegative Matrix Factorization (SNMF) based on alternating nonnegativity constrained least squares, Sparse Network-Regularized Multiple Nonnegative Matrix Factorization (SNMNMF), Penalized Matrix Factorization for Constrained Clustering (PMFCC) and Separable Nonnegative Matrix Factorization (SepNMF). Results A total of 548 patients were included (7.3% females, age at diagnosis: 51.0 [38.0-61.0] years old. Of these, 66 suffered from spontaneous ventricular tachyarrhythmias over 84±55 months follow-up. The baseline model using multivariable logistic regression achieved an area under the curve (AUC) of 0.78 [0.72-0.85], which was improved to 0.88 [0.83-0.93] for the IML model without NMF. The AUC was further increased by incorporating additional latent variables extracted using NSNMF (0.95 [0.92-0.98]) and PMFCC (0.94 [0.88-1.00]). Conclusion Incorporation of latent variables by different NMF techniques into an IML prediction model significantly improved the accuracy of risk prediction in BrS.

Estimation of nonlinear mixed-effects continuous-time models using the continuous-discrete extended Kalman filter.

Many intensive longitudinal measurements are collected at irregularly spaced time intervals, and involve complex, possibly nonlinear and heterogeneous patterns of change. Effective modelling of such change processes requires continuous-time differential equation models that may be nonlinear and include mixed effects in the parameters. One approach of fitting such models is to define random effect variables as additional latent variables in a stochastic differential equation (SDE) model of choice, and use estimation algorithms designed for fitting SDE models, such as the continuous-discrete extended Kalman filter (CDEKF) approach implemented in the dynr R package, to estimate the random effect variables as latent variables. However, this approach's efficacy and identification constraints in handling mixed-effects SDE models have not been investigated. In the current study, we analytically inspect the identification constraints of using the CDEKF approach to fit nonlinear mixed-effects SDE models; extend a published model of emotions to a nonlinear mixed-effects SDE model as an example, and fit it to a set of irregularly spaced ecological momentary assessment data; and evaluate the feasibility of the proposed approach to fit the model through a Monte Carlo simulation study. Results show that the proposed approach produces reasonable parameter and standard error estimates when some identification constraint is met. We address the effects of sample size, process noise variance, and data spacing conditions on estimation results.

Learning additional latent representations using variational autoencoders

A noticeable trend of machine learning is to deal with data of various modalities. Besides multimodal motivation, learning more from general information without forgetting the prior, or incremental learning, could also benefit unimodal machine learning. Human understanding often starts from a simplistic, generic view of the whole problem and then fills in the details and nuances. Incremental learning could enable imitation of such a learning process to learn more and potentially faster. This paper examines the possibility of learning additional latent variables from known latent variables for variational autoencoders. A method is proposed to facilitate learning additional information based on a modified β-TCVAE loss function that incorporates known general mutual information. A qualitative comparison is conducted on the dSprites dataset to evaluate the effect of this modification and the change of network structures on the learned latent, which hints at the structural tendency for β-TCVAE to learn new information from explicit known latent.

Holographic Single-Particle Imaging for Weakly Scattering, Heterogeneous Nanoscale Objects

Single-particle imaging (SPI) at x-ray free-electron lasers is a technique to determine the three-dimensional structure of nanoscale objects like biomolecules from a large number of diffraction patterns of copies of these objects in random orientations. The technique has been limited to relatively low resolution due to background noise and heterogeneity of the target particles. A recently introduced reference-enhanced holographic SPI methodology uses strongly scattering holographic references to improve background tolerance, and, thus, the achievable resolution, at the cost of additional latent variables beyond orientation. Here, we describe an improved reconstruction algorithm based on maximum likelihood estimation, which scales better, enabling fine sampling of latent parameters to reach high resolutions, and much better performance in the low signal limit. Furthermore, we show that structural variations within the target particle are averaged in real space, significantly improving robustness to conformational heterogeneity in comparison to conventional SPI. With these computational improvements, we believe reference-enhanced SPI is capable of reaching sub-nanometer resolution biomolecule imaging.

The effects of CNS atrophy and ICVD on tests of executive function and functional status are mediated by intelligence

BackgroundImpairments in executive function (EF) are often attributed to ischemic cerebrovascular disease (ICVD) and frontal circuit pathology. However, EF can be distinguished from general intelligence and the latter is likely to manifest in “executive” measures. We aimed to distinguish the effects of imaging biomarkers on these constructs. MethodsWe tested neuroimaging biomarkers as independent predictors of observed 12 month-prospective cognitive performance by a Multiple Indicators Multiple Causes (MIMIC) model in the Alzheimer's Disease Neuroimaging Initiative (ADNI) (N ≅ 1750). ResultsICVD was associated with ‘’Organization” (ORG) and “Planning” (PLAN) domain scores from the test of Every Day Cognition. Left anterior cingulate (LAC) atrophy was independently associated with Trail-Making part B and Animal Naming. The MIMIC model had excellent fit and tests additional latent variables i.e., EF and dEF (a latent δ homolog derived from Spearman's general intelligence factor, g). Only dEF was associated with instrumental activities of daily living (IADL). ICVD and LAC were both associated with observed executive measures through dEF. ICVD was independently associated with those same measures through EF. ConclusionsObserved EF is independently determined by multiple factors. The effects of EF-associated MRI biomarkers can be related to disability and dementia only via their effects on g. Because g /δ are unlikely to be located within the frontal lobes, the dementia-specific variance in executive measures may have little to do with either frontal structure or function. Conversely, domain-specific variance in EF may have little to do with either IADL-impairment or dementia.

A latent slice sampling algorithm

Motivated by a sampling algorithm for discrete spaces, a variation of the slice sampler for continuous spaces is introduced. It utilizes latent variables and is related to Neal's slice sampler. The key difference is that the additional latent variables allow the sequential stepping out or doubling procedures, which makes the basic slice sampler difficult to use in high dimensional problems, to be avoided. On the other hand, the latent slice sampling algorithm is applicable on high dimensional problems where the variables can all be treated in a single block.

Endogeneity and Measurement Bias of the Indicator Variables in Hybrid Choice Models: A Monte Carlo Investigation

We investigate the problem of endogeneity and measurement bias arising from incorporating indicator variables (e.g., measures of attitudes) into discrete choice models. We demonstrate that although a hybrid choice framework can resolve both endogeneity and measurement problems, the former requires explicit accounting for in the model, which has not typically been done in applied studies to date. By conducting a Monte Carlo experiment, we demonstrate the extent of the bias resulting from measurement and endogeneity problems. We propose two novel solutions to address the endogeneity problem: explicitly accounting for correlation between structural and discrete choice component error terms (or with random parameters in a utility function), or introducing additional latent variables. Using simulated data, we demonstrate that these approaches work as expected, i.e. they successfully recover the true values of all model parameters.

Label-Noise Robust Deep Generative Model for Semi-Supervised Learning

Deep generative models have demonstrated an excellent ability to generate data by learning their distribution. Despite their unsupervised nature, these models can be implemented in semi-supervised learning scenarios by treating the class labels as additional latent variables. In this article, we propose a deep generative model for semi-supervised learning that offsets label noise, which is a ubiquitous feature in large-scale datasets owing to the high cost of annotation. We assume that noisy labels are generated from true labels and employ a noise transition matrix to describe the transition. We estimate this matrix by adjusting its entries to minimize its difference with the true transition matrix and use the estimated matrix to formulate the objective function for inference, which consists of an evidence lower bound and a classification risk. However, because directly minimizing the latter with noisy labels may result in an inaccurate classifier, we propose a statistically consistent estimator for computing the classification risk solely with noisy data. Empirical results on benchmark datasets demonstrate that the proposed model improves the classification performance over that of the baseline algorithms. We also present a case study on semiconductor manufacturing. Additionally, we empirically show that the proposed model, as a generative model, is capable of reconstructing data even with noisy labels.

A Comparison of Different Approaches for Estimating Cross-Lagged Effects from a Causal Inference Perspective

This article compares different approaches for estimating cross-lagged effects with a cross-lagged panel design under a causal inference perspective. We distinguish between models that rely on no unmeasured confounding (i.e., observed covariates are sufficient to remove confounding) and latent variable-type models (e.g., random intercept cross-lagged panel model) that use parametric assumptions to adjust for unmeasured time-invariant confounding by including additional latent variables. Simulation studies confirm that the cross-lagged panel model provides biased estimates of the cross-lagged effect in the presence of unmeasured confounding. However, the simulations also show that the latent variable-type approaches strongly depend on the specific parametric assumptions, and produce biased estimates under different data-generating scenarios. Finally, we discuss the role of the longitudinal design and the limitations of assessing model fit for estimating cross-lagged effects.

Inversion of low- to medium-frequency velocities and densities from AVO data using invertible neural networks

Amplitude-variation-with-offset (AVO) inversion and neural networks (NN) are widely used to invert elastic parameters. With more constraints from well-log data, neural network-based inversion may estimate elastic parameters with greater precision and resolution than traditional AVO inversion; however, neural network approaches necessitate a massive number of reliable training samples. Furthermore, because the lack of low-frequency information in seismic gathers leads to multiple solutions of the inverse problem, both inversions rely heavily on proper low-frequency initial models. To mitigate the dependence of inversions on accurate training samples and initial models, we have adopted solving inverse problems with the recently developed invertible neural networks (INNs). Unlike conventional neural networks, which address the ambiguous inverse issues directly, INNs learn definite forward modeling and use additional latent variables to increase the uniqueness of solutions. Motivated by the newly developed neural networks, we adopt an INN-based AVO inversion method, which can reliably invert low- to medium-frequency velocities and densities with randomly generated easy-to-access data sets rather than trustworthy training samples or well-prepared initial models. Tests on synthetic and field data indicate that our method is feasible, antinoise capable, and practicable.

Comparison of the young aged and elderly female users’ adoption of mHealth services

The authors’ aim in this study was to investigate the factors influencing the adoption of mHealth services between the young aged and the elderly female in Bangladesh. Using the extended Technology Acceptance Model (TAM) with three additional latent variables, i.e., perceived reliability, price value and technology anxiety, the proposed model was empirically examined based on the required data from a total survey of 542 respondents. Researchers revealed that perceived usefulness had significant impact on behavioral intention for both the young aged and the elderly female but perceived ease of use was insignificant for both groups. Perceived reliability and price value were found to have strong impact on actual usage. But the effect of technology anxiety was significant for elderly female but insignificant for young aged female due to age characteristics. Therefore, this study is the first initiative to compare the adoption factors between young aged and elderly female in Bangladesh.

Determination of factors affecting the response efficacy of Filipinos under Typhoon Conson 2021 (Jolina): An extended protection motivation theory approach

The response efficacy measures can be utilized to assess a person's beliefs as to whether the recommended action step will actually minimize the impact of a natural disaster such as a typhoon. This study examines the response efficacy of Filipinos under Typhoon Conson 2021 (Jolina) using the Extended Protection Motivation Theory (PMT) approach. To accurately measure the factors and their relationships to response efficacy, an online questionnaire was developed and distributed using a convenience sampling method to 388 Filipinos a few days before the typhoon hit the Philippines. Several latent variables in PMT such as understanding of typhoon, perceived severity, response cost, self-efficacy, and response efficacy together with some additional latent variables such as typhoon experience, geographical perspective, and perceived susceptibility were analyzed simultaneously. Structural Equation Modeling (SEM) showed that perceived severity of the typhoon and self-efficacy were the key factors affecting the response efficacy of Filipinos in preparing for typhoon Jolina. Moreover, it was also found that understanding typhoons, self-efficacy, perceived susceptibility, and past typhoon experience indirectly affected response efficacy. The results of this study could be utilized by future researchers and planners of natural disasters to find ways of enhancing the response efficacy in preparing for typhoons. Finally, the findings of this study can also be utilized as a theoretical framework for government worldwide in designing and implementing strategies and policies for natural disaster risk protection.

Modulating scalable Gaussian processes for expressive statistical learning

For a learning task, Gaussian process (GP) is interested in learning the statistical relationship between inputs and outputs, since it offers not only the prediction mean but also the associated variability. The vanilla GP however is hard to learn complicated distribution with the property of, e.g., heteroscedastic noise, multi-modality and non-stationarity, from massive data due to the Gaussian marginal and the cubic complexity. To this end, this article studies new scalable GP paradigms including the non-stationary heteroscedastic GP, the mixture of GPs and the latent GP, which introduce additional latent variables to modulate the outputs or inputs in order to learn richer, non-Gaussian statistical representation. Particularly, we resort to different variational inference strategies to arrive at analytical or tighter evidence lower bounds (ELBOs) of the marginal likelihood for efficient and effective model training. Extensive numerical experiments against state-of-the-art GP and neural network (NN) counterparts on various tasks verify the superiority of these scalable modulated GPs, especially the scalable latent GP, for learning diverse data distributions.

Latent Variable Modelling with Non-Ignorable Item Non-Response: Multigroup Response Propensity Models for Cross-National Analysis

SummaryWhen missing data are produced by a non-ignorable non-response mechanism, analysis of the observed data should include a model for the probabilities of responding. We propose such models for non-response in survey questions which are treated as measures of latent constructs and analysed by using latent variable models. The non-response models that we describe include additional latent variables (latent response propensities) which determine the response probabilities. We argue that this model should be specified as flexibly as possible, and we propose models where the response propensity is a categorical variable (a latent response class). This can be combined with any latent variable model for the survey items, and an association between the latent variables measured by the items and the latent response propensities then implies a model with non-ignorable non-response. We consider in particular such models for the analysis of data from cross-national surveys, where the non-response model may also vary across the countries. The models are applied to data on welfare attitudes in 29 countries in the European Social Survey.

Inferring catalysis in biological systems.

In systems biology, one is often interested in the communication patterns between several species, such as genes, enzymes or proteins. These patterns become more recognisable when temporal experiments are performed. This temporal communication can be structured by reaction networks such as gene regulatory networks or signalling pathways. Mathematical modelling of data arising from such networks can reveal important details, thus helping to understand the studied system. In many cases, however, corresponding models still deviate from the observed data. This may be due to unknown but present catalytic reactions. From a modelling perspective, the question of whether a certain reaction is catalysed leads to a large increase of model candidates. For large networks the calibration of all possible models becomes computationally infeasible. We propose a method which determines a substantially reduced set of appropriate model candidates and identifies the catalyst of each reaction at the same time. This is incorporated in a multiple-step procedure which first extends the network by additional latent variables and subsequently identifies catalyst candidates using similarity analysis methods. Results from synthetic data examples suggest a good performance even for non-informative data with few observations. Applied on CD95 apoptotic pathway our method provides new insights into apoptosis regulation.

Latent Variable Modelling with Non-Ignorable Item Nonresponse: A General Framework and Multigroup Models for Cross-National Analysis

When missing data are produced by a non-ignorable nonresponse mechanism, analysis of the observed data should include a model for the probabilities of responding. In this paper we propose such models for nonresponse in survey questions which are treated as multiple-item measures of latent constructs and analysed using latent variable models. The nonresponse models that we describe include additional latent variables (latent response propensities) which determine the response probabilities. We argue that this model should be specified as flexibly as possible, and propose models where the response propensity is a categorical variable (a latent response class). This can be combined with any latent variable model for the survey items themselves, and an association between the latent variables measured by the items and the latent response propensities implies a model with non-ignorable nonresponse. We consider in particular the analysis of data from cross-national surveys, where the nonresponse model may also vary across the countries. The models are applied to analyse data on welfare attitudes in 29 countries in the European Social Survey.

The Expectation–Maximization approach for Bayesian quantile regression

This paper deals with Bayesian linear quantile regression models based on a recently developed Expectation–Maximization Variable Selection (EMVS) method. By using additional latent variables, the proposed approach enjoys enormous computational savings compared to commonly used Markov Chain Monte Carlo (MCMC) algorithm. Using location-scale mixture representation of asymmetric Laplace distribution (ALD), we develop a rapid and efficient Expectation–Maximization (EM) algorithm, which is illustrated with several carefully designed simulation examples. We further apply the proposed method to construct financial index tracking portfolios.

Model selection for partial least squares calibration and implications for analysis of atmospheric organic aerosol samples with mid‐infrared spectroscopy

In developing partial least squares calibration models, selecting the number of latent variables used for their construction to minimize both model bias and model variance remains a challenge. Several metrics exist for incorporating these trade‐offs, but the cost of model parsimony and the potential for underfitting on achievable prediction errors are difficult to anticipate. We propose a metric that penalizes growing model variance against decreasing bias as additional latent variables are added. The magnitude of the penalty is scaled by a user‐defined parameter that is formulated to provide a constraint on the fractional increase in root mean square error of cross‐validation (RMSECV) when selecting a parsimonious model over the conventional minimum RMSECV solution. We evaluate this approach for quantification of four organic functional groups using 238 laboratory standards and 750 complex atmospheric organic aerosol mixtures with mid‐infrared spectroscopy. Parametric variation of this penalty demonstrates that increase in prediction errors due to underfitting is bounded by the magnitude of the penalty for samples similar to laboratory standards used for model training and validation. Imposing an ensemble of penalties corresponding to a 0–30% allowable increase in RMSECV through sum of ranking differences leads to the selection of a model that increases the actual RMSECV up to 20% for laboratory standards but achieves an 85% reduction in the mean error in predicted concentrations for environmental mixtures. Partial least squares models developed with laboratory mixtures can provide useful predictions in complex environmental samples, but may benefit from protection against overfitting. © 2015 The Authors. Journal of Chemometrics published by John Wiley & Sons Ltd.

A fixed-links modeling approach to assess individual differences in the attentional blink: Analysis of behavioral and psychophysiological data

The attentional blink (AB) is a fundamental limitation of the ability to select relevant information from irrelevant information. It can be observed with the detection rate in an AB task as well as with the corresponding P300 amplitude of the event-related potential. In previous research, however, correlations between these two levels of observation were weak and rather inconsistent. A possible explanation of this finding might be that multiple processes underlie the AB and, thus, obscure a possible relationship between AB-related detection rate and the corresponding P300 amplitude. The present study investigated this assumption by applying a fixed-links modeling approach to represent behavioral individual differences in the AB as a latent variable. Concurrently, this approach enabled us to control for additional sources of variance in AB performance by deriving two additional latent variables. The correlation between the latent variable reflecting behavioral individual differences in AB magnitude and a corresponding latent variable derived from the P300 amplitude was high (r=.70). Furthermore, this correlation was considerably stronger than the correlations of other behavioral measures of the AB magnitude with their psychophysiological counterparts (all rs<.40). Our findings clearly indicate that the systematic disentangling of various sources of variance by utilizing the fixed-links modeling approach is a promising tool to investigate behavioral individual differences in the AB and possible psychophysiological correlates of these individual differences.

Alternating Minimization Algorithm with Automatic Relevance Determination for Transmission Tomography under Poisson Noise

We propose a globally convergent alternating minimization (AM) algorithm for image reconstruction in transmission tomography, which extends automatic relevance determination (ARD) to Poisson noise models with Beer's law. The algorithm promotes solutions that are sparse in the pixel/voxel--difference domain by introducing additional latent variables, one for each pixel/voxel, and then learning these variables from the data using a hierarchical Bayesian model. Importantly, the proposed AM algorithm is free of any tuning parameters with image quality comparable to standard penalized likelihood methods. Our algorithm exploits optimization transfer principles which reduce the problem into parallel one-dimensional optimization tasks (one for each pixel/voxel), making the algorithm feasible for large-scale problems. This approach considerably reduces the computational bottleneck of ARD associated with the posterior variances. Positivity constraints inherent in transmission tomography problems are also enforced. ...