Latent States Research Articles

A deep clustering-based state-space model for improved disease risk prediction in personalized healthcare

AbstractDecision support systems are being developed to assist clinicians in complex decision-making processes by leveraging information from clinical knowledge and electronic health records (EHRs). One typical application is disease risk prediction, which can be challenging due to the complexity of modelling longitudinal EHR data, including unstructured medical notes. To address this challenge, we propose a deep state-space model (DSSM) that simulates the patient’s state transition process and formally integrates latent states with risk observations. A typical DSSM consists of three parts: a prior module that generates the distribution of the current latent state based on previous states; a posterior module that approximates the latent states using up-to-date medical notes; and a likelihood module that predicts disease risks using latent states. To efficiently and effectively encode raw medical notes, our posterior module uses an attentive encoder to better extract information from unstructured high-dimensional medical notes. Additionally, we couple a predictive clustering algorithm into our DSSM to learn clinically useful representations of patients’ latent states. The latent states are clustered into multiple groups, and the weighted average of the cluster centres is used for prediction. We demonstrate the effectiveness of our deep clustering-based state-space model using two real-world EHR datasets, showing that it not only generates better risk prediction results than other baseline methods but also clusters similar patient health states into groups.

Comparing revised latent state-trait models including autoregressive effects.

Understanding the longitudinal dynamics of behavior, their stability and change over time, are of great interest in the social and behavioral sciences. Researchers investigate the degree to which an observed measure reflects stable components of the construct, situational fluctuations, method effects, or just random measurement error. An important question in such models is whether autoregressive effects occur between the residuals, as in the trait-state occasion model (TSO model), or between the state variables, as in the latent state-trait model with autoregression (LST-AR model). In this article, we compare the two approaches by applying revised latent state-trait theory (LST-R theory). Similarly to Eid et al. (2017) regarding the TSO model, we show how to formulate the LST-AR model using definitions from LST-R theory, and we discuss the practical implications. We demonstrate that the two models are equivalent when the trait loadings are allowed to vary over time. This is also true for bivariate model versions. The different but same approaches to modeling latent states and traits with autoregressive effects are illustrated with a longitudinal study of cancer-related fatigue in Hodgkin lymphoma patients. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Modeling multivariate positive‐valued time series using R‐INLA

AbstractIn this article, we describe fast Bayesian statistical analysis of vector positive‐valued time series, with application to interesting financial data streams. We discuss a flexible level correlated model (LCM) framework for building hierarchical models for vector positive‐valued time series. The LCM allows us to combine marginal gamma distributions for the positive‐valued component responses, while accounting for association among the components at a latent level. We introduce vector autoregression evolution of the latent states, deriving its precision matrix and enabling its estimation using integrated nested Laplace approximation (INLA) for fast approximate Bayesian modeling via the R‐INLA package, building custom functions to handle this setup. We use the proposed method to model interdependencies between intraday volatility measures from several stock indexes.

Variable Selection for Hidden Markov Models with Continuous Variables and Missing Data

AbstractWe propose a variable selection method for multivariate hidden Markov models with continuous responses that are partially or completely missing at a given time occasion. Through this procedure, we achieve a dimensionality reduction by selecting the subset of the most informative responses for clustering individuals and simultaneously choosing the optimal number of these clusters corresponding to latent states. The approach is based on comparing different model specifications in terms of the subset of responses assumed to be dependent on the latent states, and it relies on a greedy search algorithm based on the Bayesian information criterion seen as an approximation of the Bayes factor. A suitable expectation-maximization algorithm is employed to obtain maximum likelihood estimates of the model parameters under the missing-at-random assumption. The proposal is illustrated via Monte Carlo simulation and an application where development indicators collected over eighteen years are selected, and countries are clustered into groups to evaluate their growth over time.

High-resolution tracking of unconfined zebrafish behavior reveals stimulatory and anxiolytic effects of psilocybin

Serotonergic psychedelics are emerging therapeutics for psychiatric disorders, yet their underlying mechanisms of action in the brain remain largely elusive. Here, we developed a wide-field behavioral tracking system for larval zebrafish and investigated the effects of psilocybin, a psychedelic serotonin receptor agonist. Machine learning analyses of precise body kinematics identified latent behavioral states reflecting spontaneous exploration, visually-driven rapid swimming, and irregular swim patterns following stress exposure. Using this method, we found that acute psilocybin treatment has two behavioral effects: [i] facilitation of spontaneous exploration (“stimulatory”) and [ii] prevention of irregular swim patterns following stress exposure (“anxiolytic”). These effects differed from the effect of acute SSRI treatment and were rather similar to the effect of ketamine treatment. Neural activity imaging in the dorsal raphe nucleus suggested that psilocybin inhibits serotonergic neurons by activating local GABAergic neurons, consistent with psychedelic-induced suppression of serotonergic neurons in mammals. These findings pave the way for using larval zebrafish to elucidate neural mechanisms underlying the behavioral effects of serotonergic psychedelics.

MASER: Enhancing EEG Spatial Resolution With State Space Modeling.

Consumer-grade Electroencephalography (EEG) devices equipped with few electrodes often suffer from low spatial resolution, hindering the accurate capture of intricate brain activity patterns. To address this issue, we propose MASER, a novel super-resolution approach for EEG recording. In MASER, we design the eMamba block for extracting EEG features based on the principles of state space models (SSMs). We further stack eMamba blocks to form a low-resolution feature extractor and a high-resolution signal predictor, which enhances the feature representation. During the training of MASER, we fully consider the characteristics of multidimensional biological series signals, incorporating a smoothness constraint loss to achieve more consistent high-resolution reconstructions. MASER pioneers EEG-oriented state space modeling, effectively capturing the temporal dynamics and latent states, thereby revealing complex neural interactions over time. Extensive experiments show that the proposed MASER outperforms the state-of-the-art methods in super-resolution quality on two public EEG datasets, with normalized mean square error reduced by 16.25% and Pearson correlation improved by 1.13%. Moreover, a case study of motor imagery recognition highlights the advantages conferred by high-resolution EEG signals. With a 4x increase in spatial resolution by MASER, the recognition accuracy improves by 5.74%, implying a significant performance elevation in brain-computer interface (BCI) command mapping. By enhancing the spatial resolution of EEG signals, MASER makes EEG-based applications more accessible, reducing cost and setup time while maintaining high performance across various domains such as gaming, education, and healthcare.

Fuzzy State-Driven Cross-Time Spatial Dependence Learning for Multivariate Time-Series Anomaly Detection.

Cross-time spatial dependence (i.e., the interaction between different variables at different time points) is indispensable for detecting anomalies in multivariate time series, as certain anomalies may have time delays in their propagation from one variable to another. However, accurately capturing cross-time spatial dependence remains a challenge. Specifically, real-world time series usually exhibits complex and incomprehensible evolutions that may be compounded by multiple temporal states (i.e., temporal patterns, such as rising, fluctuating, and peak). These temporal states mix and overlap with each other and exhibit dynamic and heterogeneous evolution laws in different time series, making the cross-time spatial dependence extremely intricate and mutable. Therefore, a cross-time spatial graph network with fuzzy embedding is proposed to disentangle latent and mixing temporal states and exploit it to meticulously learn cross-time spatial dependence. First, considering that temporal states are diversiform and their mixing modes are unknown, we introduce a fuzzy state set to uniformly characterize potential temporal states and adaptively generate corresponding membership degrees to depict how these states mix. Further, we propose a cross-time spatial graph, quantifying similarities among fuzzy states and sensing their dynamic evolutions, to flexibly learn mutable cross-time spatial dependence. Finally, we design state diversity and temporal proximity constraints to ensure the differences among fuzzy states and the evolution continuity of fuzzy states. Experiments on real-world datasets show that the proposed model outperforms the state-of-the-art models.

Targeted Stochastic Gradient MCMC for HMMs with Rare Latent States

Targeted Stochastic Gradient MCMC for HMMs with Rare Latent States

Public Support for Emigration Post-EU Accession: An LM Model Analysis

Aim: Since 2004 European countries, such as Germany and the United Kingdom, have noted an unusual upward trend of migrants from Central and Eastern Europe. The author aimed to examine the popular perception of emigration from Poland in the years following EU accession. Moreover, the study presents the effect of the observed socio-economic features (i.e. age, education, marital status, socio-professional status) affecting the conditional probabilities of the response variable, considering also the unobserved heterogeneity between the respondents representing Polish society. Methodology: The study was based on the Latent Markov (LM) models which allowed finding homogenous groups of respondents on the basis of their definite responses measured at different points in time. The author compared the respondents behaviour in the groups arriving just after EU accession, in relation to the other surveyed surges in migration using the Latent Markov models with different types of constraints. Then, to show the effect of observable socio-economic characteristics, taking into account the unobserved heterogeneity between the subjects, the study employed the version including the covariates in the measurement part of the model. Results: Based on the Polish longitudinal social survey, i.e. Social Diagnosis, the study shows that the analysed data can be explained by two latent states of Poles sharing the same propensity towards emigration. It was observed that Poles were more stable in their feelings concerning their approach to the emigration in the period 2011–2015 (following the first post-accession phase). Based on the available (time-constant and time-varying) covariates, the author demonstrated that certain types of people are more prone to migrate than others. Implications and recommendations: The presented approach could be applied to data from other economies – both within the EU and outside it – which have also seen massive outflows of people, to show the changing behaviour of society. This methodology might also be helpful with policy challenges and the effective management of migration, as well as for future data concerning the attitudes towards refugees from Ukraine coming to Poland at present. Originality/value: Unlike previous studies, the author assumed that the attitudes toward emigration can vary on the response variable, i.e. because of the unobserved covariate such as ‘patriotic behaviour’ (unobserved heterogeneity), which may change over time. The study applied the appropriate LM techniques extended to include the covariates in the measurement part of the LM model and the longitudinal weights of the survey. Note that the applied model also allowed achieving a better measure-of-fit compared to the standard random-effect model, and the parameters of the LM model additionally enabled to evaluate the dynamic of the latent trait.

Nonlinearities in macroeconomic tail risk through the lens of big data quantile regressions

SummaryModeling and predicting extreme movements in GDP is notoriously difficult, and the selection of appropriate covariates and/or possible forms of nonlinearities are key in obtaining precise forecasts. In this paper, our focus is on using large datasets in quantile regression models to forecast the conditional distribution of US GDP growth. To capture possible nonlinearities, we include several nonlinear specifications. The resulting models will be huge dimensional, and we thus rely on a set of shrinkage priors. Since Markov chain Monte Carlo estimation becomes slow in these dimensions, we rely on fast variational Bayes approximations to the posterior distribution of the coefficients and the latent states. We find that our proposed set of models produces precise forecasts. These gains are especially pronounced in the tails. Using Gaussian processes to approximate the nonlinear component of the model further improves the good performance, in particular in the right tail.

Understanding step selection analysis through numerical integration

Abstract Step selection functions (SSFs) are flexible statistical models used to jointly describe animals' movement and habitat preferences. The popularity of SSFs has grown rapidly, and various extensions have been developed to increase their utility, including the ability to use multiple statistical distributions to describe movement constraints, interactions to allow movements to depend on local environmental features, and random effects and latent states to account for within‐ and among‐individual variability. Although the SSF is a relatively simple statistical model, its presentation has not been consistent in the literature, leading to confusion about model flexibility and interpretation. We believe that part of the confusion has arisen from the conflation of the SSF model with the methods used for statistical inference, and in particular, parameter estimation. Notably, conditional logistic regression (CLR) can be used to fit SSFs in exponential form, and this model fitting approach is often presented interchangeably with the actual model (the SSF itself). However, reliance on CLR reduces model flexibility, and suggests a misleading interpretation of step selection analysis as being equivalent to a case–control study. In this review, we explicitly distinguish between model formulation and inference technique, presenting a coherent framework to fit SSFs based on numerical integration and maximum likelihood estimation. We provide an overview of common numerical integration techniques (including Monte Carlo integration, importance sampling and quadrature), and explain how they relate to popular methods used in step selection analyses. This general framework unifies different model fitting techniques for SSFs, and opens the way for improved inferential methods. In this approach, it is straightforward to model movement with distributions outside the exponential family, and to apply different SSF model formulations to the same data set and compare them with AIC. By separating the model formulation from the inference technique, we hope to clarify many important concepts in step selection analysis.

How leader's type shapes the effect of nuclear latency on dispute involvement

What is the effect of nuclear latency on states’ conflict behaviors? Defined as the ability to produce fissile material without the actual possession of nuclear weapons, nuclear latency is much more common in the international system than nuclear weapons are. Yet present studies might have underestimated its impact on conflict. We argue that, given its ambiguous nature as a dual-use technology, nuclear latency has the potential to significantly alter the effect of leader type on conflict. In particular, while conventional wisdom about hawks and doves suggests that the former should be more prone to conflict than the latter, nuclear latency blurs the distinction between leader's types. Bridging together studies of nuclear latency and leader type, we study how nuclear latency shapes the impact of leader type on the propensity of initiating conflict or being targeted. We find that in the absence of latency, hawkish leaders are more likely to initiate disputes than dovish leaders. However, latency muddies the waters. Thus, leader characteristics does not affect dispute initiation for latent states.

Unsupervised learning of stationary and switching dynamical system models from Poisson observations

Objective. Investigating neural population dynamics underlying behavior requires learning accurate models of the recorded spiking activity, which can be modeled with a Poisson observation distribution. Switching dynamical system models can offer both explanatory power and interpretability by piecing together successive regimes of simpler dynamics to capture more complex ones. However, in many cases, reliable regime labels are not available, thus demanding accurate unsupervised learning methods for Poisson observations. Existing learning methods, however, rely on inference of latent states in neural activity using the Laplace approximation, which may not capture the broader properties of densities and may lead to inaccurate learning. Thus, there is a need for new inference methods that can enable accurate model learning. Approach. To achieve accurate model learning, we derive a novel inference method based on deterministic sampling for Poisson observations called the Poisson Cubature Filter (PCF) and embed it in an unsupervised learning framework. This method takes a minimum mean squared error approach to estimation. Terms that are difficult to find analytically for Poisson observations are approximated in a novel way with deterministic sampling based on numerical integration and cubature rules. Main results. PCF enabled accurate unsupervised learning in both stationary and switching dynamical systems and largely outperformed prior Laplace approximation-based learning methods in both simulations and motor cortical spiking data recorded during a reaching task. These improvements were larger for smaller data sizes, showing that PCF-based learning was more data efficient and enabled more reliable regime identification. In experimental data and unsupervised with respect to behavior, PCF-based learning uncovered interpretable behavior-relevant regimes unlike prior learning methods. Significance. The developed unsupervised learning methods for switching dynamical systems can accurately uncover latent regimes and states in population spiking activity, with important applications in both basic neuroscience and neurotechnology.

A Hidden Markov Model for Condition Monitoring of Time Series Data in Complex Network Systems

Time series data are ubiquitous in complex network systems, where each component of the system is treated as a vertex and has sequentially collected autocorrelated observations for condition monitoring. In many cases, components transit between different states, such as “normal,” “degrading,” “failure,” etc., and under different states their distributions vary, resulting in the mixture marginal distribution of each vertex's time series data. Moreover, states of different components influence each other through the system's network topology, i.e., the state of a component itself and its neighbors jointly affect its data distribution. For efficient condition monitoring, it is important to capture the state evolution of all vertices as a whole over time. However, the state-switching may be unobservable, which complicates the modeling. Hence, this article proposes a hidden Markov model for networked time series data with mixture marginal distributions. The Markov transition rule for latent states captures the state-switching behavior and the AR model reveals the temporal dependence of vertices. The states of different vertices further influence each other through the network topology structure. The Baum–Welch algorithm is used to estimate the parameters of the proposed model, allowing for state inference and data fitting. Extensive numerical studies as well as real case studies demonstrate the effectiveness and applicability of the proposed model.

Decoding the internesting movements of marine turtles using a fine-scale behavioral state approach

IntroductionAn understanding of animal behavior is critical to determine their ecological role and to inform conservation efforts. However, observing hidden behaviors can be challenging, especially for animals that spend most of their time underwater. Animal-borne devices are valuable tools to estimate hidden behavioral states.MethodsWe investigated the fine-scale behavior of internesting hawksbill turtles using the mixed-membership method for movement (M4) which integrated dive variables with spatial components and estimated latent behavioral states.ResultsFive latent behavioral states were identified: 1) pre-nesting, 2) transit, 3) quiescence, and 4) area restricted search within and 5) near the residence of turtles. The last three states associated with a residency period, showed lower activity levels. Notably, when compared to other behaviors the pre-nesting exhibited shallower and remarkably long dives of up to 292 minutes. We noted high fidelity to residence core areas and nesting beaches, within and between nesting seasons, with residence areas decreasing within a season.DiscussionThe latent behaviors identified provide the most detailed breakdown of turtle movement behaviors during the internesting period to date, providing valuable insights into their ecology and behavior. This information can inform marine turtle conservation and management efforts since utilization distributions of individual behavioral states can be used to determine spatially-explicit susceptibility of turtles to various threats based on their behavior. The analyses of utilization distribution revealed a minimal overlap with existing marine protected areas (0.4%), and we show how a new proposal would expand protection to 30%. In short, this study provides valuable guidance for conservation and management of internesting marine turtles at a fine spatiotemporal resolution and can be used to enhance national action plans for endangered species, including the expansion of existing Marine Protected Areas. By flexibly incorporating biologically informative parameters, this approach can be used to study behavior outside of the hawksbill breeding season or even beyond this species.

Learning to Predict 3D Rotational Dynamics from Images of a Rigid Body with Unknown Mass Distribution

In many real-world settings, image observations of freely rotating 3D rigid bodies may be available when low-dimensional measurements are not. However, the high-dimensionality of image data precludes the use of classical estimation techniques to learn the dynamics. The usefulness of standard deep learning methods is also limited, because an image of a rigid body reveals nothing about the distribution of mass inside the body, which, together with initial angular velocity, is what determines how the body will rotate. We present a physics-based neural network model to estimate and predict 3D rotational dynamics from image sequences. We achieve this using a multi-stage prediction pipeline that maps individual images to a latent representation homeomorphic to SO(3), computes angular velocities from latent pairs, and predicts future latent states using the Hamiltonian equations of motion. We demonstrate the efficacy of our approach on new rotating rigid-body datasets of sequences of synthetic images of rotating objects, including cubes, prisms and satellites, with unknown uniform and non-uniform mass distributions. Our model outperforms competing baselines on our datasets, producing better qualitative predictions and reducing the error observed for the state-of-the-art Hamiltonian Generative Network by a factor of 2.

Followership styles scrutinized: temporal consistency and relationships with job attitudes and self-efficacy.

While followership has been repeatedly acknowledged as an important part of leadership, key questions are still awaiting empirical testing. In our two studies, we test Kelley's prominent concept of followership styles for the first time in a longitudinal design. Specifically, we use a latent-state trait approach to examine the degree to which followership behaviors (i.e., active engagement [AE] and independent, critical thinking [ICT]) reflect rather stable or rather dynamic behaviors. Furthermore, we examine the relationships of followership behaviors with job attitudes (i.e., job satisfaction and organizational commitment) and self-efficacy in latent states cross-lagged models. We first test our hypotheses in a sample of N = 184 employees from eleven German service organizations, which were surveyed twice with a time lag of nine to 12 months. To replicate and extend our findings from Study 1, we conducted Study 2 with a sample of N = 570 participants from a German open-access panel, which were surveyed twice with a time lag of four months. In Study 2, we additionally test leader humility and perceived organizational support (POS) as potential moderators of the relationships between followership and job attitudes. While our findings support Kelley's conceptualization of followership styles as rather consistent behavior patterns, mixed results were found for the relationships with the other variables. We discuss the theoretical and practical implications of our findings as well as the relevance of time in followership research.

Evidence for Reduced Sensory Precision and Increased Reliance on Priors in Hallucination-Prone Individuals in a General Population Sample.

There is increasing evidence that people with hallucinations overweight perceptual beliefs relative to incoming sensory evidence. Past work demonstrating prior overweighting has used simple, nonlinguistic stimuli. However, auditory hallucinations in psychosis are often complex and linguistic. There may be an interaction between the type of auditory information being processed and its perceived quality in engendering hallucinations. We administered a linguistic version of the conditioned hallucinations (CH) task to an online sample of 88 general population participants. Metrics related to hallucination-proneness, hallucination severity, stimulus thresholds, and stimulus detection rates were collected. Data were used to fit parameters of a Hierarchical Gaussian Filter (HGF) model of perceptual inference to determine how latent perceptual states influenced task behavior. Replicating past results, higher CH rates were observed both in those with recent hallucinatory experiences as well as participants with high hallucination-proneness; CH rates were positively correlated with increased prior weighting; and increased prior weighting was related to hallucination severity. Unlike past results, participants with recent hallucinatory experiences as well as those with higher hallucination-proneness had higher stimulus thresholds, lower sensitivity to stimuli presented at the highest threshold, and had lower response confidence, consistent with lower precision of sensory evidence. We replicate the finding that increased CH rates and recent hallucinations correlate with increased prior weighting using a linguistic version of the CH task. Results support a role for reduced sensory precision in the interplay between prior weighting and hallucination-proneness.

Multi-view dreaming: multi-view world model with contrastive learning

In this paper, we propose Multi-View Dreaming, a novel reinforcement learning agent for integrated recognition and control from multi-view observations by extending Dreaming. Most current reinforcement learning method assumes a single-view observation space, and this imposes limitations on the observed data, such as lack of spatial information and occlusions. This makes obtaining ideal observational information from the environment difficult and is a bottleneck for real-world robotics applications. In this paper, we use contrastive learning to train a shared latent space between different viewpoints and show how the Products of Experts approach can be used to integrate and control the probability distributions of latent states for multiple viewpoints. We also propose Multi-View DreamingV2, a variant of Multi-View Dreaming that uses a categorical distribution to model the latent state instead of the Gaussian distribution. Experiments show that the proposed method outperforms simple extensions of existing methods in a realistic robot control task.

An evaluation of self-reported material well-being using latent Markov models with covariates.

A household's financial satisfaction is one of the most significant factors driving subjective well-being. However, Poland ranks close to the lowest position, 22nd out of the 28 EU members, in self-reported financial status. The paper investigates the problem of determining patterns of Polish households' behaviour and shows the evolution of the subjective assessment of financial situation based on the eight waves of the Polish Household panel data. The analysis is carried out on the basis of latent Markov (LM) models, which allow for socio-economic features affecting the parameters of the latent process. We compare different types of LM models considering: (1) different numbers of latent structures; (2) different types of the latent process constraints; (3) socio-economic background characteristics; and (4) survey weights (being excluded in most of the empirical analyses). The final model identifies three latent states, specifies common initial and transition probabilities over a 15-year period and, as a result, enables us to better characterise the families likely to change their position, especially families reporting worsening in their financial situation. To show the main direction of self-reporting financial condition, we present the predicted path for respondents characterised by the selected socio-economic features, relying on algorithm maximising posterior probabilities of the selected LM model.