Belief State Research Articles

A New Post-Processing Method using Latent Structure Influence Models for Channel Fusion in Automatic Sleep Staging.

Human sleep stage dynamics can be adequately represented using Markov chain models, and the accuracy of sleep stage classification can be improved by considering these dynamics. The present study proposes a new post-processing method based on channel fusion using Latent Structure Influence Models (LSIMs). LSIMs can simultaneously model sequences of different channels to have a nonlinear dynamical fusion based on sleep stage dynamics. The proposed method develops and examines two channel-fusion algorithms: the standard LSIM fusion and the integrated LSIM fusion, in which the latter is more efficient and performs better. The proposed LSIM-based method simultaneously incorporates the nonlinear interactions between channels and the sleep stage dynamics. In the first step, existing sleep staging systems process every data channel independently and produce stage score sequences for each channel. These single-channel scores are then projected into belief space using the marginal one-slice parameter of all channels by LSIM fusion algorithms. The logarithms of marginal one-slice parameters are concatenated to obtain log-scale belief state space (LBSS) features in the standard LSIM fusion. In the integrated LSIM fusion, integrated LBSS (ILBSS) features are formed by combining the LBSS features of several LSIMs. Finally, a KNN classifier maps the LBSS and ILBSS features onto the sleep stages. By utilizing four recently developed sleep staging systems, the proposed method is applied to the publicly available SleepEDF-20 database that contains five AASM sleep stages (N1, N2, N3, REM, and W). Compared to single-channel (Fpz-Cz, Pz-Oz, and EOG) results, integrated LSIM fusion results have a statistically significant improvement of 1.5% in 2-channel fusion (Fpz-Cz and Pz-Oz) and 2.5% in 3-channel fusion (Fpz-Cz, Pz-Oz, and EOG). With an overall accuracy of 87.3% for 3-channel post-processing, the integrated LSIM fusion system offers one of the highest overall accuracy rates among existing studies.

Radar Anti-Jamming Countermeasures Intelligent Decision-Making: A Partially Observable Markov Decision Process Approach

Current electronic warfare jammers and radar countermeasures are characterized by dynamism and uncertainty. This paper focuses on a decision-making framework of radar anti-jamming countermeasures. The characteristics and implementation process of radar intelligent anti-jamming systems are analyzed, and a scheduling method for radar anti-jamming action based on the Partially Observable Markov Process (POMDP) is proposed. The sample-based belief distribution is used to reflect the radar’s cognition of the environment and describes the uncertainty of the recognition of jamming patterns in the belief state space. The belief state of jamming patterns is updated with Bayesian rules. The reward function is used as the evaluation criterion to select the best anti-jamming strategy, so that the radar is in a low threat state as often as possible. Numerical simulation combines the behavioral prior knowledge base of radars and jammers and obtains the behavioral confrontation benefit matrix from the past experience of experts. The radar controls the output according to the POMDP policy, and dynamically performs the best anti-jamming action according to the change of jamming state. The results show that the POMDP anti-jamming policy is better than the conventional policy. The POMDP approach improves the adaptive anti-jamming capability of the radar and can quickly realize the anti-jamming decision to jammers. This work provides some design ideas for the subsequent development of an intelligent radar.

Religious Values in the Rowa Traditional Ceremony in Manggarai Communities in Flores

This study aims to investigate religious values in the traditional Rowa ceremony or funeral ceremony for the Manggarai community in Flores. The Rowa traditional ceremony is a series of ceremonies for the death of a person starting from the ceremony of bathing the body, fixing the coffin, releasing and burial of the body. The utterances at the Rowa traditional ceremony are a means of presenting themselves to the Manggarai people before each other, their ancestors and God. The traditional speech in the Rowa traditional ceremony is in the form of a poem called torok tae. As a poem, torok tae has a variety of traditional languages, connotative word choices and has a special semantic meaning. This study used qualitative research methods. Data was collected using the listening and speaking method and perfected by the observing and speaking technique involving speaking, taking notes and recording the entire series of Rowa traditional ceremonies. The results of the study prove that the utterances in the Rowa traditional ceremony have religious values which include the values of trust, reconciliation and safety. The value of belief states belief in the Highest Being (God) in cultural terms addressed Mori, Mori Dewa, Dewa Mori Ngaran Jari, Jari Dewa Mori Ngaran, Mori Ngaran and anthropomorphic greeting Ende Wa. Belief in the existence of wakar souls, ase kae which revive humans and the existence of life after death. The value of reconciliation states the restoration of the vertical relationship between God the Creator and humans as His creations. The value of salvation is a form of recognition by the Manggarai people for God's holiness, God's protection and new life after death.

Cognitive aging and combined interventions: self-efficiency as a device to improve brain function

Cerebral neurodegeneration may be associated with functional changes, being able to generate cognitive decline, a decrease in the skills that make up brain function. The main objective of this study is to evaluate how self-efficacy belief, personal development and mood states are related to neurocognitive aging. The present study is based on the quasi-experimental and longitudinal development model. The study sample consisted of 33 elderly people with a mean age of 71 years (SD = ± 6.32), spaced from 61 to 89 years of age. Six instruments were applied for data collection, namely: Sociodemographic Data Sheet, Addenbrooke Cognitive Examination, Personal Development Scale, General Self-Efficacy Scale - Revised Version, Geriatrics Depression Scale and Beck Anxiety Scale. Statistical normality tests were used at a significance level of 5% (p ≤ 0.05). Regarding the results, the sample is statistically significant in relation to four variables, namely: memory (p = 0.00), verbal fluency (p = 0.00), cognition (p = 0.00) and self-efficacy (p = 0.00); moreover, from the difference between groups related to cognitive interventions (Stimullus + MEMO), based on the self-efficacy construct, for example, it is possible to notice that the self-efficacy variable (p = 0.03) manifests statistical significance. It is concluded that the sense of perceived efficacy can be used as a device to mediate neurocognitive gains, being able to improve brain function, supported by combined interventions.

Relational vs representational social cognitive processing: a coordinate-based meta-analysis of neuroimaging data.

The neurocognitive bases of social cognition have been framed in terms of representing others' actions through the mirror system and their mental states via the mentalizing network. Alongside representing another person's actions or mental states, however, social cognitive processing is also shaped by their (mis)match with one's own corresponding states. Here, we addressed the distinction between representing others' states through the action observation or mentalizing networks (i.e.representational processing) and detecting the extent to which such states align with one's own ones (i.e.relational processing, mediated by social conflict). We took a meta-analytic approach to unveil the neural bases of both relational and representational processing by focusing on previously reported brain activations from functional magnetic resonance imaging studies using false-belief and action observation tasks. Our findings suggest that relational processing for belief and action states involves, respectively, the left and right temporo-parietal junction, likely contributing to self-other differentiation. Moreover, distinct sectors of the posterior fronto-medial cortex support social conflict processing for belief and action, possibly through the inhibition of conflictual representations. These data might pave the way for further studies addressing social conflict as an important component of normal and pathological processing, and inform the design of rehabilitative treatments for social deficits.

Seeing the Error in My "Bayes": A Quantified Degree of Belief Change Correlates with Children's Pupillary Surprise Responses Following Explicit Predictions.

Bayesian models allow us to investigate children's belief revision alongside physiological states, such as "surprise". Recent work finds that pupil dilation (or the "pupillary surprise response") following expectancy violations is predictive of belief revision. How can probabilistic models inform the interpretations of "surprise"? Shannon Information considers the likelihood of an observed event, given prior beliefs, and suggests stronger surprise occurs following unlikely events. In contrast, Kullback-Leibler divergence considers the dissimilarity between prior beliefs and updated beliefs following observations-with greater surprise indicating more change between belief states to accommodate information. To assess these accounts under different learning contexts, we use Bayesian models that compare these computational measures of "surprise" to contexts where children are asked to either predict or evaluate the same evidence during a water displacement task. We find correlations between the computed Kullback-Leibler divergence and the children's pupillometric responses only when the children actively make predictions, and no correlation between Shannon Information and pupillometry. This suggests that when children attend to their beliefs and make predictions, pupillary responses may signal the degree of divergence between a child's current beliefs and the updated, more accommodating beliefs.

The Logic of Framing Effects

Framing effects concern the having of different attitudes towards logically or necessarily equivalent contents. Framing is of crucial importance for cognitive science, behavioral economics, decision theory, and the social sciences at large. We model a typical kind of framing, grounded in (i) the structural distinction between beliefs activated in working memory and beliefs left inactive in long term memory, and (ii) the topic- or subject matter-sensitivity of belief: a feature of propositional attitudes which is attracting growing research attention. We introduce a class of models featuring (i) and (ii) to represent, and reason about, agents whose belief states can be subject to framing effects. We axiomatize a logic which we prove to be sound and complete with respect to the class.

Small steps for mankind: Modeling the emergence of cumulative culture from joint active inference communication.

Although the increase in the use of dynamical modeling in the literature on cultural evolution makes current models more mathematically sophisticated, these models have yet to be tested or validated. This paper provides a testable deep active inference formulation of social behavior and accompanying simulations of cumulative culture in two steps: First, we cast cultural transmission as a bi-directional process of communication that induces a generalized synchrony (operationalized as a particular convergence) between the belief states of interlocutors. Second, we cast social or cultural exchange as a process of active inference by equipping agents with the choice of who to engage in communication with. This induces trade-offs between confirmation of current beliefs and exploration of the social environment. We find that cumulative culture emerges from belief updating (i.e., active inference and learning) in the form of a joint minimization of uncertainty. The emergent cultural equilibria are characterized by a segregation into groups, whose belief systems are actively sustained by selective, uncertainty minimizing, dyadic exchanges. The nature of these equilibria depends sensitively on the precision afforded by various probabilistic mappings in each individual's generative model of their encultured niche.

Personalized and Energy-Efficient Health Monitoring: A Reinforcement Learning Approach

We consider a network of controlled sensors that monitor the unknown health state of a patient. We assume that the health state process is a Markov chain with a transition matrix that is unknown to the controller. At each timestep, the controller chooses a subset of sensors to activate, which incurs an energy (i.e., battery) cost. Activating more sensors improves the estimation of the unknown state, which introduces an energy-accuracy tradeoff. Our goal is to minimize the combined energy and state misclassification costs over time. Activating sensors now also provides measurements that can be used to learn the model, improving future decisions. Therefore, the learning aspect is intertwined with the energy-accuracy tradeoff. While Reinforcement Learning (RL) is often used when the model is unknown, it cannot be directly applied in health monitoring since the controller does not know the (health) state. Therefore, the monitoring problem is a partially observable Markov decision process (POMDP) where the cost feedback is also only partially available since the misclassification cost is unknown. To overcome this difficulty, we propose a monitoring algorithm that combines RL for POMDPs and online estimation of the expected misclassification cost based on a Hidden Markov Model (HMM). We show empirically that our algorithm achieves comparable performance with a monitoring system that assumes a known transition matrix and quantizes the belief state. It also outperforms the model-based approach where the estimated transition matrix is used for value iteration. Thus, our algorithm can be useful in designing energy-efficient and personalized health monitoring systems.

Synthesis of Non-blocking Controllers for Linear Temporal Logic Tasks under Partial Observations⋆

In this paper, we investigate the formal synthesis of discrete controllers for linear temporal logic tasks under partial information. Existing works on this topic mainly focus on find sure winning or almost sure winning control strategies under some assumptions regarding the system's atomic propositions. In this work, we consider non-blockingness as the metric as the achievement of the task. Specifically, we require that at each instant, the controller maintains the possibility to achieve the LTL task under some environment's behavior. We first present an offline algorithm for the computation of the winning region over the belief state space. Then we present an online control algorithm that effective solves the control synthesis problem. The proposed control algorithm is also illustrated by a case study of robot task planning.

Integrating individual and social learning: accuracy and evolutionary viability

Much of what we know, we know thanks to our interactions with others. There is a variety of ways in which we learn from others. We sometimes simply adopt the viewpoints of those we regard as experts, but we also sometimes change our viewpoints in more subtle ways based on the viewpoints of people we regard as our peers. Both forms of social learning have been receiving increasing attention. However, studies investigating how best to combine them, and how to combine the two with individual forms of learning, are still few and far between. This paper looks at ways to integrate various forms of social learning with learning at an individual level within a broadly Bayesian framework. Using agent-based models, we compare the different ways in terms of accuracy of belief states as well as in terms of evolutionary viability. The outcomes of our simulations suggest that agents are best off spending most of their time engaging in social learning, reserving only a limited amount of time for individual learning.

Nursing students' attitudes and beliefs regarding sexual healthcare in Ethiopia: An online cross-sectional study.

Unresolved sexual issues can have an impact on a person's wellbeing, social interactions, or even medication compliance. Given the enormous global demand to enhance and preserve sexual health, faculty members have a specific goal of developing nursing workforce who are truly prepared to work with clients who have sexual health issues. Hence, the study's purpose has been to evaluate the current state of attitude and belief of nursing students toward sexual healthcare and the factors that influence it in Southwest Ethiopia. An online cross-sectional survey was conducted in Mizan-Tepi University involving 134 undergraduate nursing students from February 1 to March 10, 2022. The Sexual Attitude and Beliefs Scale (SABS) were used to assess the level of attitudes and beliefs regarding sexual care. Statistical Package for Social Science (SPSS) Version 26 was employed for data analysis. Multivariable linear regression analyses was conducted to identify predictors of attitudes and beliefs regarding sexual healthcare. The significance level was set at p<0.05. The mean age of the nursing students participating in this study was 28.47±5.2. In our study, mean SABS score of the nursing students was found as 42.3 ± 2.1. The mean score of the SABS items ranged from 1.68±0.93 to 4.37±1.48. Having receiving sexual health education (p<0.001) was significantly associated with attitude and beliefs regarding sexual healthcare. Ethiopian nursing students have been found to have a negative attitude and beliefs about sexual health care. Because comprehensive sex education is sorely lacking in nursing schools, this scrutiny prevents student nurses from receiving an adequate education. As there is a lack of appropriate sexual health subject matter, it is critical to standardize sexual health education in nursing programs so that nursing students can learn to provide comprehensive care to clients.

A reactive power optimization partially observable Markov decision process with data uncertainty using multi-agent actor-attention-critic algorithm

We present an innovative partially observable Markov decision process (POMDP) modelling method for the reactive power optimization process of the active distribution network (ADN) under the high permeability of the distributed generation. This model is tolerant of the uncertainty resulting from data uncertainty. We believe that the belief state space in the POMDP model corresponds to the state space in the Markov decision process (MDP) model, and we apply the multi-agent actor-attention-critic (MAAC) reinforcement learning (RL) algorithm to the proposed model. This technique extracts the most effective information with the highest quality from the huge historical measurement database, hence enhancing the learning effectiveness of agents and the stability of the optimization strategy. We simulate reactive power optimization in a modified IEEE-33 nodes ADN and a modified IEEE-123nodes ADN. The simulation demonstrates the stability and economic superiority of the proposed approach under varying degrees of data uncertainty relative to previous RL algorithms based on the MDP model. The simulation demonstrates that the proposed POMDP model is more appropriate for the real operation of the partially observable distribution network than the MDP models. And the optimal strategy obtained by the proposed MAAC algorithm is reliable with deteriorating data quality.

Judgment aggregation, discursive dilemma and reflective equilibrium: Neural language models as self-improving doxastic agents.

Neural language models (NLMs) are susceptible to producing inconsistent output. This paper proposes a new diagnosis as well as a novel remedy for NLMs' incoherence. We train NLMs on synthetic text corpora that are created by simulating text production in a society. For diagnostic purposes, we explicitly model the individual belief systems of artificial agents (authors) who produce corpus texts. NLMs, trained on those texts, can be shown to aggregate the judgments of individual authors during pre-training according to sentence-wise vote ratios (roughly, reporting frequencies), which inevitably leads to so-called discursive dilemmas: aggregate judgments are inconsistent even though all individual belief states are consistent. As a remedy for such inconsistencies, we develop a self-training procedure-inspired by the concept of reflective equilibrium-that effectively reduces the extent of logical incoherence in a model's belief system, corrects global mis-confidence, and eventually allows the model to settle on a new, epistemically superior belief state. Thus, social choice theory helps to understand why NLMs are prone to produce inconsistencies; epistemology suggests how to get rid of them.

Partial observability and management of ecological systems.

The actual state of ecological systems is rarely known with certainty, but management actions must often be taken regardless of imperfect measurement (partial observability). Because of the difficulties in accounting for partial observability, it is usually treated in an ad hoc fashion, or simply ignored altogether. Yet incorporating partial observability into decision processes lends a realism that has the potential to improve ecological outcomes significantly. We review frameworks for dealing with partial observability, focusing specifically on dynamic ecological systems with Markovian transitions, i.e., transitions among system states that are influenced by the current system state and management action over time. Fully observable states are represented in an observable Markov decision process (MDP), whereas obscure or hidden states are represented in a partially observable process (POMDP). POMDPs can be seen as a natural extension of observable MDPs. Management under partial observability generalizes the situation for complete observability, by recognizing uncertainty about the system's state and incorporating sequential observations associated with, but not the same as, the states themselves. Decisions that otherwise would depend on the actual state must be based instead on state probability distributions (“belief states”). Partial observability requires adaptation of the entire decision process, including the use of belief states and Bayesian updates, valuation that includes expectations over observations, and optimal strategy that identifies actions for belief states over a continuous belief space. We compare MDPs and POMDPs and highlight POMDP applications to some common ecological problems. We clarify the structure and operations, approaches for finding solutions, and analytic challenges of POMDPs for practicing ecologists. Both observable and partially observable MDPs can use an inductive approach to identify optimal strategies and values, with a considerable increase in mathematical complexity with POMDPs. Better understanding of POMDPs can help decision makers manage imperfectly measured ecological systems more effectively.

Uncertainty-of-Information Scheduling: A Restless Multiarmed Bandit Framework

This paper proposes using the uncertainty of information (UoI), measured by Shannon’s entropy, as a metric for information freshness. We consider a system in which a central monitor observes M binary Markov processes through m communication channels (m<M). The UoI of a Markov process corresponds to the monitor’s uncertainty about its state. At each time step, only m Markov processes can be selected to update its state to the monitor; hence there is a tradeoff among the UoIs of the processes that depend on the scheduling policy used to select the processes to be updated. The age of information (AoI) of a process corresponds to the time since its last update. In general, the associated UoI can be a non-increasing function, or even an oscillating function, of its AoI, making the scheduling problem particularly challenging. This paper investigates scheduling policies that aim to minimize the average sum-UoI of the processes over the infinite time horizon. We formulate the problem as a restless multi-armed bandit (RMAB) problem, and develop a Whittle index policy that is near-optimal for the RMAB after proving its indexability. We further provide an iterative algorithm to compute the Whittle index for the practical deployment of the policy. Although this paper focuses on UoI scheduling, our results apply to a general class of RMABs for which the UoI scheduling problem is a special case. Specifically, this paper’s Whittle index policy is valid for any RMAB in which the bandits are binary Markov processes and the penalty is a concave function of the belief state of the Markov process. Numerical results demonstrate the excellent performance of the Whittle index policy for this class of RMABs.

Théorie multidimensionnelle et computationnelle de l’humeur

What is mood? Despite its crucial place in psychiatric nosography and cognitive science, it is still difficult to delimit its conceptual ground. The distinction between emotion and mood is ambiguous: mood is often presented as an affective state that is more prolonged and less intense than emotion, or as an affective polarity distinguishing high and low mood swinging around a baseline. However, these definitions do not match the clinical reality of mood disorders such as unipolar depression and bipolar disorder, and do not allow us to understand the effect of mood on behaviour, perception and cognition. In this paper, we propose a multidimensional and computational theory of mood inspired by contemporary hypotheses in theoretical neuroscience and philosophy of emotion. After suggesting an operational distinction between emotion and mood, we show how a succession of emotions can cumulatively generate congruent mood over time, making mood an emerging state from emotion. We then present how mood determines mental and behavioral states when interacting with the environment, constituting a dispositional state of emotion, perception, belief, and action. Using this theoretical framework, we propose a computational representation of the emerging and dispositional dimensions of mood by formalizing mood as a layer of third-order Bayesian beliefs encoding the precision of emotion, and regulated by prediction errors associated with interoceptive predictions. Finally, we show how this theoretical framework sheds light on the processes involved in mood disorders, the emergence of mood congruent beliefs, or the mechanisms of antidepressant treatments in clinical psychiatry.

The problem of closure and questioning attitudes

The problem of closure for the traditional unstructured possible worlds model of attitudinal content is that it treats belief and other cognitive states as closed under entailment, despite apparent counterexamples showing that this is not a necessary property of such states. One solution to this problem, which has been proposed recently by several authors (Schaffer in: Hawthorne and Gendler (eds) Oxford studies in epistemology, Oxford University Press, Oxford, pp 235–271, 2005; Yalcin in Philos Phenomenol Res 97(1):23–47, 2018; Hoek in: Kindermann (ed) Unstructured content, Oxford University Press, Oxford, forthcoming), is to restrict closure in an unstructured setting by treating propositional attitudes as question-sensitive. Here I argue that this line of response is unsatisfying as it stands because the problem of closure is more general than is typically discussed. A version of the problem recurs for attitudes like wondering, entertaining, considering, and so on, which are directed at questions rather than propositions. For such questioning attitudes, the appeal to question-sensitivity is much less convincing as a solution to the problem of closure.

Post-storm repair crew dispatch for distribution grid restoration using stochastic Monte Carlo tree search and deep neural networks

Natural disasters such as storms usually bring significant damages to distribution grids. This paper investigates the optimal routing of utility vehicles to restore outages in the distribution grid as fast as possible after a storm. First, the post-storm repair crew dispatch task with multiple utility vehicles is formulated as a sequential stochastic optimization problem. In the formulated optimization model, the belief state of the power grid is updated according to the phone calls from customers and the information collected by utility vehicles. Second, an AlphaZero based utility vehicle routing (AlphaZero-UVR) approach is developed to achieve the real-time dispatching of the repair crews. The proposed AlphaZero-UVR approach combines stochastic Monte-Carlo tree search (MCTS) with deep neural networks to give a lookahead search decisions, which can learn to navigate repair crews without human guidance. Simulation results show that the proposed approach can efficiently navigate crews to repair all outages.

Markov Decision Processes with Incomplete Information and Semiuniform Feller Transition Probabilities

This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies the class of Markov Decision Processes with Incomplete information and with semi-uniform Feller transition probabilities. The important feature of this class of models is that the classic reduction of such a model with incomplete observation to the completely observable Markov Decision Process with belief states preserves semi-uniform Feller continuity of transition probabilities. Under mild assumptions on cost functions, optimal policies exist, optimality equations hold, and value iterations converge to optimal values for this class of models. In particular, for Partially Observable Markov Decision Processes the results of this paper imply new and generalize several known sufficient conditions on transition and observation probabilities for the existence of optimal policies, validity of optimality equations, and convergence of value iterations.