Continuous Latent Variables Research Articles

Concurrent topology optimization of sandwich structures with multi-configuration and variable-diameter lattice infill

The superior stiffness-to-weight and strength-to-weight mechanical advantages of sandwich structures can be fully exploited through concurrent design of entire topology, infill configuration and density, where the high-performance yet complicated structure can be fabricated through additive manufacturing. However, the emerging design challenges are concurrent design updating related to sandwich topology, infill configuration and density, which is a design problem with continuous and discrete variables mathematically. In this paper, a concurrent topology optimization is proposed for sandwich structures with multi-configuration and variable-diameter lattice infill. Three design variable fields are employed to describe the fundamental topology considering sandwich structural topology, infill configuration and density simultaneously. Corresponding material interpolation model is developed by combining DSP-based shell-infill description and multi-response latent-variable surrogate model based effective material property calculation. Two-stage design model is formulated as a rough design model considering all design variables followed by a refined design model with only infill density variables, which is developed to strictly satisfy the material allowance constraint due to the mapping between discrete infill configuration variables and continuous latent configuration variables. Corresponding sensitivities of compliance and constraint with respect to the structural topology, infill configuration and density variables are derived, and the method of moving asymptotes (MMA) is employed to solve the design model efficiently. Several numerical examples are presented to systematically demonstrate the effectiveness of the proposed approach.

Measurement and Uncertainty Preserving Parametric Modeling for Continuous Latent Variables With Discrete Indicators and External Variables

Research in education and behavioral sciences often involves the use of latent variable models that are related to indicators, as well as related to covariates or outcomes. Such models are subject to interpretational confounding, which occurs when fitting the model with covariates or outcomes alters the results for the measurement model. This has received attention in models for continuous observable variables but to date has not been examined in the context of discrete variables. This work demonstrates that interpretational confounding can occur in models for discrete variables, and develops a multistage Bayesian estimation approach to deal with this problem. The key features of this approach are that it is (a) measurement preserving, in that it precludes the possibility of interpretational confounding, and (b) uncertainty preserving, in that the uncertainty from the earlier stage of estimating the measurement model is propagated to the second stage of estimating the relations between the latent variable(s) and any covariates or outcomes. Previous work on these methods had only considered models for continuous observed variables, and software was limited to models with a single latent variable and either covariates or outcomes. This work extends the approach and software to a more general class of solutions, including discrete variables, illustrating the procedures with analyses of real data. Functions for conducting the analyses in widely available software are provided.

Bayesian clustering of mixed-type data with relevant variable identification

This paper presents a Bayesian nonparametric model for clustering datasets with continuous, ordinal, and nominal variables. The ordinal and nominal variables are treated using the latent variables framework based on the multivariate probit and the multinomial probit models. Combining the continuous variables with the latent continuous variables allows us to jointly model a set of mixed-type variables via the Dirichlet process Gaussian mixture model. The use of hierarchical shrinkage prior on the component means leads to improved clustering performances and provides an intuitive way to identify relevant clustering variables. The numerical results on simulated and real data illustrate the applicability of the proposed model.

FLPM: A property modification scheme for data protection in federated learning

Federated learning (FL) is a critical technology for implementing time-critical computing systems in the Internet of Things (IoT). It allows for continuous updates to machine learning (ML) models across IoT devices. However, the vulnerability of ML models and the complexity of IoT pose significant threats to device data security and privacy, affecting the robustness of time-critical computing systems constructed through FL. Recent research on FL data protection has made progress, but challenges remain in balancing privacy protection with model availability. For example, cryptography-based defense schemes increase time overhead in time-critical computing systems, while differential privacy negatively impacts system performance. This paper proposes the FL properties modification scheme (FLPM) for data preprocessing to resist property inference attacks and data poisoning attacks. FLPM modifies training data properties using algorithms for property separation, selection, and control based on continuous latent variables. While this sacrifices a small amount of classification accuracy, it significantly improves data protection capabilities. Detailed experimental results demonstrate that FLPM successfully separates and controls image property vectors. In the FL classification task, the property modification data achieve a precision of 94.44%. This scheme effectively prevents property inference attacks and data poisoning attacks. FLPM can reduce the AUC score for property inference attacks from 0.94 to 0.56 and reduce the success rate of data poisoning attacks to 5.13%, 7.07%, and 4.60%.

Bayesian meta-learning for personalized thermal comfort modeling

Data-driven models coupled with machine learning techniques have been developed to predict thermal comfort of individuals. However, collecting (quantitatively and qualitatively) sufficient data to develop and train the models is often challenging. This paper presents a Bayesian meta-learning approach for developing reliable, data-driven personalized thermal comfort models using limited data from individuals. The learning process considers general thermal comfort impact factors (environmental variables, clothing level and metabolic rate) as well as personal thermal characteristics. The personal thermal characteristics are expressed as a vector of continuous latent variables, estimated using limited data from each person. A high-dimensional neural network was developed to map model inputs (e.g., air temperature, relative humidity) and the vector of the continuous latent variables with personal thermal sensation (model output). The model parameters in the neural network are trained with data from various people using a subset of the ASHRAE RP-884 database. The neural network is transferrable without any update or modification (i.e., the same trained network can be used to predict the thermal preference of new individuals given their personal thermal characteristics), making the learning approach data-efficient. The results show that the developed Bayesian meta-learning approach to infer personal thermal comfort performs better than existing methods, especially when using limited data. This is important considering the practical limitations in collecting sufficient thermal response data from individuals in real buildings.

Beyond Continuous versus Categorical Dichotomy: Uncovering Latent Structure of Non-electoral Political Participation Using Zero-Inflated Models

AbstractWhile modeling political participation as a latent variable, researchers usually choose whether to conceptualize and model participation as a latent continuous or latent categorical variable. When participation is modeled as a continuous variable, factor analytic and item-response theory models are used. When modeled as a categorical variable, latent class analysis is employed. However, both conceptualizations and modeling approaches rest upon very strong assumptions. In the continuous case, all subjects are assumed to come from the same homogenous population; in the categorical case, we assume that no quantitative heterogeneity exists within the latent classes. In this work, I argue that these assumptions are implausible and propose to model participation using zero-inflated measurement and regression models that assume the existence of two latent classes-politically disengaged and politically active-with the latter class being quantitatively heterogenous (people in that class are thought to participate to a varying degree). The results show that the models accounting for the latent class of politically disengaged have much better out-of-sample predictive accuracy. Moreover, modeling the zero-inflation changes estimates of measurement and regression models, and offers new research opportunities because with zero-inflated models we can explicitly tackle the question of what impacts the probability of ending up in the latent class of politically disengaged.

A computationally efficient mixture innovation model for time-varying parameter regressions

The mixture innovation (MI) model places a spike-and-slab mixture distribution for the innovations of time-varying regression coefficients and permits flexible time variation patterns while allowing for dynamic shrinkage. Despite its appeal, the standard Bayesian algorithm to block sample the vector of 0/1 mixture indicators at each time t needs to evaluate the model likelihood over all its 2K scenarios for a regression model with K regressors and becomes impractical when K grows. As an alternative, a new specification of the MI model is proposed in which the 0/1 mixture indicators in the original MI model are approximated by a logistic function of latent continuous variables. As such the model likelihood only needs to be evaluated twice in an Metropolis-Hastings step to block update the latent variables and hence the approximated mixture indicators at each time t, offering large improvement in computational efficiency while keeping the benefits of the MI model. An efficient MCMC algorithm is developed to estimate the new model. A simulation study shows that the new model can achieve the same level of estimation accuracy as the original MI model but at a much smaller computation cost. The new model is further tested in two empirical applications where block sampling the mixture indicators at each time t in the original MI model is practically infeasible.

A comprehensive model framework for between-individual differences in longitudinal data.

Across different fields of research, the similarities and differences between various longitudinal models are not always eminently clear due to differences in data structure, application area, and terminology. Here we propose a comprehensive model framework that will allow simple comparisons between longitudinal models, to ease their empirical application and interpretation. At the within-individual level, our model framework accounts for various attributes of longitudinal data, such as growth and decline, cyclical trends, and the dynamic interplay between variables over time. At the between-individual level, our framework contains continuous and categorical latent variables to account for between-individual differences. This framework encompasses several well-known longitudinal models, including multilevel regression models, growth curve models, growth mixture models, vector-autoregressive models, and multilevel vector-autoregressive models. The general model framework is specified and its key characteristics are illustrated using famous longitudinal models as concrete examples. Various longitudinal models are reviewed and it is shown that all these models can be united into our comprehensive model framework. Extensions to the model framework are discussed. Recommendations for selecting and specifying longitudinal models are made for empirical researchers who aim to account for between-individual differences. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Association between prenatal socioeconomic disadvantage, adverse birth outcomes, and inflammatory response at birth

Prenatal socioeconomic disadvantage is associated with inflammation in mid- to late-life, yet whether a pro-inflammatory phenotype is present at birth and the role of adverse birth outcomes in this pathway remains unclear. We utilized data on prenatal socioeconomic disadvantage at the individual- (i.e., mother’s and father’s education level, insurance type, marital status, and Women, Infants, and Children benefit receipt) and census-tract level as well as preterm (< 37 weeks gestation) and small-for-gestational-age (SGA) (i.e., < 10th percentile of sex-specific birth weight for gestational age) birth status, and assessed inflammatory markers (i.e., C-reactive protein, serum amyloid p, haptoglobin, and α-2 macroglobulin) in archived neonatal bloodspots from a Michigan population-based cohort of 1000 neonates. Continuous latent variables measuring individual- and combined individual- and neighborhood-level prenatal socioeconomic disadvantage were constructed and latent profile analysis was used to create a categorical inflammatory response variable (high versus low) based on continuous inflammatory marker levels. Structural equation models were used to estimate the total and direct effect of prenatal socioeconomic disadvantage on the inflammatory response at birth as well as indirect effect via preterm or SGA birth (among term neonates only), adjusting for mother’s age, race/ethnicity, body mass index, smoking status, comorbidities, and antibiotic use/infection as well as grandmother’s education level. There was a statistically significant total effect of both individual- and combined individual- and neighborhood-level prenatal socioeconomic disadvantage on high inflammatory response among all neonates as well as among term neonates only, and a positive but not statistically significant direct effect in both groups. The indirect effects via preterm and SGA birth were both negative, but not statistically significant. Our findings suggest prenatal socioeconomic disadvantage contributes to elevated neonatal inflammatory response, but via pathways outside of these adverse birth outcomes.

MixER: linear interpolation of latent space for entity resolution

Entity resolution, accurately identifying various representations of the same real-world entities, is a crucial part of data integration systems. While existing learning-based models can achieve good performance, the models are extremely dependent on the quantity and quality of training data. In this paper, the MixER model is proposed to alleviate these problems. The MixER utilizes our newly designed data augmentation method called EMix. The EMix can map discrete entity records to continuous latent space variables (e.g., probability distributions) and then linearly interpolate entity records in latent space to generate many augmented training samples. The matching model is further optimized based on the augmented data to strengthen its generalization capability. The MixER model achieves significant strengths in the data sensitivity experiments when training data is below 50. In robustness experiments, the MixER model presents an absolute performance advantage when the label noise exceeds 20%. In addition, ablation experiments demonstrate that the newly developed EMix can effectively improve the generalization ability of the matching model. The overall experimental results prove that the MixER model exhibited excellent data sensitivity and robustness over the current state-of-the-art methods.

An Energy-Based Prior for Generative Saliency.

We propose a novel generative saliency prediction framework that adopts an informative energy-based model as a prior distribution. The energy-based prior model is defined on the latent space of a saliency generator network that generates the saliency map based on a continuous latent variables and an observed image. Both the parameters of saliency generator and the energy-based prior are jointly trained via Markov chain Monte Carlo-based maximum likelihood estimation, in which the sampling from the intractable posterior and prior distributions of the latent variables are performed by Langevin dynamics. With the generative saliency model, we can obtain a pixel-wise uncertainty map from an image, indicating model confidence in the saliency prediction. Different from existing generative models, which define the prior distribution of the latent variables as a simple isotropic Gaussian distribution, our model uses an energy-based informative prior which can be more expressive in capturing the latent space of the data. With the informative energy-based prior, we extend the Gaussian distribution assumption of generative models to achieve a more representative distribution of the latent space, leading to more reliable uncertainty estimation. We apply the proposed frameworks to both RGB and RGB-D salient object detection tasks with both transformer and convolutional neural network backbones. We further propose an adversarial learning algorithm and a variational inference algorithm as alternatives to train the proposed generative framework. Experimental results show that our generative saliency model with an energy-based prior can achieve not only accurate saliency predictions but also reliable uncertainty maps that are consistent with human perception.

Coding of latent variables in sensory, parietal, and frontal cortices during closed-loop virtual navigation.

We do not understand how neural nodes operate and coordinate within the recurrent action-perception loops that characterize naturalistic self-environment interactions. Here, we record single-unit spiking activity and local field potentials (LFPs) simultaneously from the dorsomedial superior temporal area (MSTd), parietal area 7a, and dorsolateral prefrontal cortex (dlPFC) as monkeys navigate in virtual reality to 'catch fireflies'. This task requires animals to actively sample from a closed-loop virtual environment while concurrently computing continuous latent variables: (i) the distance and angle travelled (i.e., path integration) and (ii) the distance and angle to a memorized firefly location (i.e., a hidden spatial goal). We observed a patterned mixed selectivity, with the prefrontal cortex most prominently coding for latent variables, parietal cortex coding for sensorimotor variables, and MSTd most often coding for eye movements. However, even the traditionally considered sensory area (i.e., MSTd) tracked latent variables, demonstrating path integration and vector coding of hidden spatial goals. Further, global encoding profiles and unit-to-unit coupling (i.e., noise correlations) suggested a functional subnetwork composed by MSTd and dlPFC, and not between these and 7a, as anatomy would suggest. We show that the greater the unit-to-unit coupling between MSTd and dlPFC, the more the animals' gaze position was indicative of the ongoing location of the hidden spatial goal. We suggest this MSTd-dlPFC subnetwork reflects the monkeys' natural and adaptive task strategy wherein they continuously gaze toward the location of the (invisible) target. Together, these results highlight the distributed nature of neural coding during closed action-perception loops and suggest that fine-grain functional subnetworks may be dynamically established to subserve (embodied) task strategies.

A new goodness-of-fit measure for probit models: Surrogate R2.

Probit models are used extensively for inferential purposes in the social sciences as discrete data are prevalent in a vast body of social studies. Among many accompanying model inference problems, a critical question remains unsettled: how to develop a goodness-of-fit measure that resembles the ordinary least square (OLS) R2 used for linear models. Such a measure has long been sought to achieve 'comparability' of different empirical models across multiple samples addressing similar social questions. To this end, we propose a novel R2 measure for probit models using the notion of surrogacy - simulating a continuous variable as a surrogate of the original discrete response (Liu & Zhang, Journal of the American Statistical Association, 113, 845 and 2018). The proposed R2 is the proportion of the variance of the surrogate response explained by explanatory variables through a linear model, and we call it a surrogate R2 . This paper shows both theoretically and numerically that the surrogate R2 approximates the OLS R2 based on the latent continuous variable, preserves the interpretation of explained variation, and maintains monotonicity between nested models. As no other pseudo R2 , McKelvey and Zavoina's and McFadden's included, can meet all the three criteria simultaneously, our measure fills this crucial void in probit model inference.

Disentangling the correlated continuous and discrete generative factors of data

Real-world data typically include discrete generative factors, such as category labels and the existence of objects, as well as continuous generative factors. Continuous generative factors may be dependent on or independent of discrete generative factors. For instance, an intra-class variation of a category is dependent on the discrete generative factor, whereas a common variation of all categories is not. Most previous attempts to integrate discrete generative factors into disentanglement assumed statistical independence between the continuous and discrete variables. In this paper, we propose a Variational Autoencoder(VAE) model capable of disentangling both continuous generative factors. To represent these generative factors, we introduce two sets of continuous latent variables: a private variable and a public variable. The private and public variables represent the intra-class variations and common variations in categories, respectively. Our proposed framework models the private variable as a Gaussian mixture and the public variable as a Gaussian. Each mode of the private variable is responsible for a class of discrete variables. Our proposed model, called Discond-VAE, DISentangles the class-dependent CONtinuous factors from the Discrete factors by introducing private variables. The experiments showed that Discond-VAE could discover private and public factors from the data. Moreover, even under the dataset with only public factors, Discond-VAE does not fail and adapts private variables to represent public factors.

A comparison of mixed-variables Bayesian optimization approaches

Most real optimization problems are defined over a mixed search space where the variables are both discrete and continuous. In engineering applications, the objective function is typically calculated with a numerically costly black-box simulation. General mixed and costly optimization problems are therefore of a great practical interest, yet their resolution remains in a large part an open scientific question. In this article, costly mixed problems are approached through Gaussian processes where the discrete variables are relaxed into continuous latent variables. The continuous space is more easily harvested by classical Bayesian optimization techniques than a mixed space would. Discrete variables are recovered either subsequently to the continuous optimization, or simultaneously with an additional continuous-discrete compatibility constraint that is handled with augmented Lagrangians. Several possible implementations of such Bayesian mixed optimizers are compared. In particular, the reformulation of the problem with continuous latent variables is put in competition with searches working directly in the mixed space. Among the algorithms involving latent variables and an augmented Lagrangian, a particular attention is devoted to the Lagrange multipliers for which a local and a global estimation techniques are studied. The comparisons are based on the repeated optimization of three analytical functions and a beam design problem.

Preferential sampling for bivariate spatial data

Preferential sampling provides a formal modeling specification to capture the effect of bias in a set of sampling locations on inference when a geostatistical model is used to explain observed responses at the sampled locations. In particular, it enables modification of spatial prediction adjusted for the bias. Its original presentation in the literature addressed assessment of the presence of such sampling bias while follow on work focused on regression specification to improve spatial interpolation under such bias. All of the work in the literature to date considers the case of a univariate response variable at each location, either continuous or modeled through a latent continuous variable. The contribution here is to extend the notion of preferential sampling to the case of bivariate response at each location. This exposes sampling scenarios where both responses are observed at a given location as well as scenarios where, for some locations, only one of the responses is recorded. That is, there may be different sampling bias for one response than for the other. It leads to assessing the impact of such bias on co-kriging. It also exposes the possibility that preferential sampling can bias inference regarding dependence between responses at a location. We develop the idea of bivariate preferential sampling through various model specifications and illustrate the effect of these specifications on prediction and dependence behavior. We do this both through simulation examples as well as with a forestry dataset that provides mean diameter at breast height (MDBH) and trees per hectare (TPH) as the point-referenced bivariate responses.

Explaining the association between social and lifestyle factors and cognitive functions: a pathway analysis in the Memento cohort

BackgroundThis work aimed to investigate the potential pathways involved in the association between social and lifestyle factors, biomarkers of Alzheimer’s disease and related dementia (ADRD), and cognition.MethodsThe authors studied 2323 participants from the Memento study, a French nationwide clinical cohort. Social and lifestyle factors were education level, current household incomes, physical activity, leisure activities, and social network from which two continuous latent variables were computed: an early to midlife (EML) and a latelife (LL) indicator. Brain magnetic resonance imaging (MRI), lumbar puncture, and amyloid-positron emission tomography (PET) were used to define three latent variables: neurodegeneration, small vessel disease (SVD), and AD pathology. Cognitive function was defined as the underlying factor of a latent variable with four cognitive tests. Structural equation models were used to evaluate cross-sectional pathways between social and lifestyle factors and cognition.ResultsParticipants’ mean age was 70.9 years old, 62% were women, 28% were apolipoprotein-ε4 carriers, and 59% had a Clinical Dementia Rating (CDR) score of 0.5. Higher early to midlife social indicator was only directly associated with better cognitive function (direct β = 0.364 (0.322; 0.405), with no indirect pathway through ADRD biomarkers (total β = 0.392 (0.351; 0.429)). In addition to a direct effect on cognition (direct β = 0.076 (0.033; 0.118)), the association between latelife lifestyle indicator and cognition was also mostly mediated by an indirect effect through lower neurodegeneration (indirect β = 0.066 (0.042; 0.090) and direct β = − 0.116 (− 0.153; − 0.079)), but not through AD pathology nor SVD.ConclusionsEarly to midlife social factors are directly associated with higher cognitive functions. Latelife lifestyle factors may help preserve cognitive functions through lower neurodegeneration.

Discovering diverse solutions in deep reinforcement learning by maximizing state–action-based mutual information

Reinforcement learning algorithms are typically limited to learning a single solution for a specified task, even though diverse solutions often exist. Recent studies showed that learning a set of diverse solutions is beneficial because diversity enables robust few-shot adaptation. Although existing methods learn diverse solutions by using the mutual information as unsupervised rewards, such an approach often suffers from the bias of the gradient estimator induced by value function approximation. In this study, we propose a novel method that can learn diverse solutions without suffering the bias problem. In our method, a policy conditioned on a continuous or discrete latent variable is trained by directly maximizing the variational lower bound of the mutual information, instead of using the mutual information as unsupervised rewards as in previous studies. Through extensive experiments on robot locomotion tasks, we demonstrate that the proposed method successfully learns an infinite set of diverse solutions by learning continuous latent variables, which is more challenging than learning a finite number of solutions. Subsequently, we show that our method enables more effective few-shot adaptation compared with existing methods.

Inference of Multiplicative Factors Underlying Neural Variability in Calcium Imaging Data.

Understanding brain function requires disentangling the high-dimensional activity of populations of neurons. Calcium imaging is an increasingly popular technique for monitoring such neural activity, but computational tools for interpreting extracted calcium signals are lacking. While there has been a substantial development of factor analysis-type methods for neural spike train analysis, similar methods targeted at calcium imaging data are only beginning to emerge. Here we develop a flexible modeling framework that identifies low-dimensional latent factors in calcium imaging data with distinct additive and multiplicative modulatory effects. Our model includes spike-and-slab sparse priors that regularize additive factor activity and gaussian process priors that constrain multiplicative effects to vary only gradually, allowing for the identification of smooth and interpretable changes in multiplicative gain. These factors are estimated from the data using a variational expectation-maximization algorithm that requires a differentiable reparameterization of both continuous and discrete latent variables. After demonstrating our method on simulated data, we apply it to experimental data from the zebrafish optic tectum, uncovering low-dimensional fluctuations in multiplicative excitability that govern trial-to-trial variation in evoked responses.

Adaptive Bayesian Estimation of Discrete‐Continuous Distributions Under Smoothness and Sparsity

We consider nonparametric estimation of a mixed discrete‐continuous distribution under anisotropic smoothness conditions and a possibly increasing number of support points for the discrete part of the distribution. For these settings, we derive lower bounds on the estimation rates. Next, we consider a nonparametric mixture of normals model that uses continuous latent variables for the discrete part of the observations. We show that the posterior in this model contracts at rates that are equal to the derived lower bounds up to a log factor. Thus, Bayesian mixture of normals models can be used for (up to a log factor) optimal adaptive estimation of mixed discrete‐continuous distributions. The proposed model demonstrates excellent performance in simulations mimicking the first stage in the estimation of structural discrete choice models.