State Inference Research Articles

ZPA: A Smart Home Privacy Analysis System Based on ZigBee Encrypted Traffic

Currently, the ZigBee protocol is widely used in smart homes and provides convenience to people. However, smart home devices often carry a large amount of real physical world information, which may result in information leakage problems. In this paper, to reveal the privacy security issues existing in ZigBee-based smart home networks, we design a smart home privacy analysis system based on ZigBee-encrypted traffic, called ZPA. ZPA can extract ZigBee data features based on the device’s operating mode and time window and use state-of-the-art machine-learning models to identify the type and status of smart home devices that could leak users’ private information. Through the analysis of 20 different devices from 5 manufacturers, the results show that even if the ZigBee traffic is protected by encryption, the accuracy of the proposed method in device type identification and state inference can reach approximately 93% and 98%, respectively. The types and statuses of devices in smart homes will reveal the user’s activity information to a certain extent. The privacy security of ZigBee-based smart devices still needs to be further strengthened.

Reading the mind in the eyes of Black and White people: Interracial contact and perceived race affects brain activity when inferring mental states

Inferring others’ mental states, or mentalizing, is a critical social cognitive ability that underlies humans’ remarkable capacity for complex social interactions. Recent work suggests that interracial contact shapes the recruitment of brain regions involved in mentalizing during impression formation. However, it remains unclear how a target's perceived racial group and a perceiver's previous contact with that racial group shapes mental state inferences. In this study, we examined brain activity in regions of interest associated with mentalizing and race perception among self-identified White perceivers who varied in lifetime contact while they inferred secondary emotions from perceived White eyes and perceived Black eyes (i.e., the Reading the Mind in the Eyes test). The interaction between lifetime contact and perceived target race predicted activity in the superior temporal sulcus (STS), a region consistently implicated in mental state inferences from perceptual cues, tracking eye gaze, and biological motion. Low and average contact White perceivers showed more left STS activity when inferring mental states from perceived White eyes than perceived Black eyes, whereas high contact White perceivers showed similar left STS activity regardless of perceived target race. These results indicate that interracial contact decreases racial biases in the recruitment of regions involved in mentalizing when inferring mental states from perceptual cues.

The Catch-22 of Forgetfulness: Responsibility for Mental Mistakes

ABSTRACT Attribution theorists assume that character information informs judgments of blame. But there is disagreement over why. One camp holds that character information is a fundamental determinant of blame. Another camp holds that character information merely provides evidence about the mental states and processes that determine responsibility. We argue for a two-channel view, where character simultaneously has fundamental and evidential effects on blame. In two large factorial studies (n = 495), participants rate whether someone is blameworthy when he makes a mistake (burns a cake or misses a bus stop). Although mental state inferences predict blame judgments, character information does not. Using mediation analyses, we find that character information influences responsibility via two channels (Studies 3–4; n = 447), which are sensitive to different kinds of information (Study 5; n = 149). On the one hand, forgetfulness increases judgments of responsibility, because mental lapses manifest an objectionable character flaw. On the other hand, forgetfulness decreases judgments of state control, which in turn decreases responsibility judgments. These two channels cancel out, which is why we find no aggregate effect of forgetfulness on responsibility. Our results challenge several fundamental assumptions about the role of character information in moral judgment, including that good character typically mitigates blame.

A Primer on Topological Data Analysis to Support Image Analysis Tasks in Environmental Science

Abstract Topological data analysis (TDA) is a tool from data science and mathematics that is beginning to make waves in environmental science. In this work, we seek to provide an intuitive and understandable introduction to a tool from TDA that is particularly useful for the analysis of imagery, namely, persistent homology. We briefly discuss the theoretical background but focus primarily on understanding the output of this tool and discussing what information it can glean. To this end, we frame our discussion around a guiding example of classifying satellite images from the sugar, fish, flower, and gravel dataset produced for the study of mesoscale organization of clouds by Rasp et al. We demonstrate how persistent homology and its vectorization, persistence landscapes, can be used in a workflow with a simple machine learning algorithm to obtain good results, and we explore in detail how we can explain this behavior in terms of image-level features. One of the core strengths of persistent homology is how interpretable it can be, so throughout this paper we discuss not just the patterns we find but why those results are to be expected given what we know about the theory of persistent homology. Our goal is that readers of this paper will leave with a better understanding of TDA and persistent homology, will be able to identify problems and datasets of their own for which persistent homology could be helpful, and will gain an understanding of the results they obtain from applying the included GitHub example code. Significance Statement Information such as the geometric structure and texture of image data can greatly support the inference of the physical state of an observed Earth system, for example, in remote sensing to determine whether wildfires are active or to identify local climate zones. Persistent homology is a branch of topological data analysis that allows one to extract such information in an interpretable way—unlike black-box methods like deep neural networks. The purpose of this paper is to explain in an intuitive manner what persistent homology is and how researchers in environmental science can use it to create interpretable models. We demonstrate the approach to identify certain cloud patterns from satellite imagery and find that the resulting model is indeed interpretable.

Supplemental Material for Judging a Book by Its Cover: Cultural Differences in Inference of the Inner State Based on the Outward Appearance

Supplemental Material for Judging a Book by Its Cover: Cultural Differences in Inference of the Inner State Based on the Outward Appearance

Bright-field to fluorescence microscopy image translation for cell nuclei health quantification.

Microscopy is a widely used method in biological research to observe the morphology and structure of cells. Amongst the plethora of microscopy techniques, fluorescent labeling with dyes or antibodies is the most popular method for revealing specific cellular organelles. However, fluorescent labeling also introduces new challenges to cellular observation, as it increases the workload, and the process may result in nonspecific labeling. Recent advances in deep visual learning have shown that there are systematic relationships between fluorescent and bright-field images, thus facilitating image translation between the two. In this article, we propose the cross-attention conditional generative adversarial network (XAcGAN) model. It employs state-of-the-art GANs (GANs) to solve the image translation task. The model uses supervised learning and combines attention-based networks to explore spatial information during translation. In addition, we demonstrate the successful application of XAcGAN to infer the health state of translated nuclei from bright-field microscopy images. The results show that our approach achieves excellent performance both in terms of image translation and nuclei state inference.

Novel theory of mind task demonstrates representation of minds in mental state inference

Theory of mind (ToM), the ability to represent the mental states of oneself and others, is argued to be central to human social experience, and impairments in this ability are thought to underlie several psychiatric and developmental conditions. To examine the accuracy of mental state inferences, a novel ToM task was developed, requiring inferences to be made about the mental states of ‘Targets’, prior participants who took part in a videoed mock interview. Participants also made estimates of the Targets’ personality traits. These inferences were compared to ground-truth data, provided by the Targets, of their true traits and mental states. Results from 55 adult participants demonstrated that trait inferences were used to derive mental state inferences, and that the accuracy of trait estimates predicted the accuracy of mental state inferences. Moreover, the size and direction of the association between trait accuracy and mental state accuracy varied according to the trait—mental state combination. The accuracy of trait inferences was predicted by the accuracy of participants’ understanding of trait covariation at the population level. Findings are in accordance with the Mind-space theory, that representation of the Target mind is used in the inference of their mental states.

Aquarius—Enable Fast, Scalable, Data-Driven Service Management in the Cloud

In order to dynamically manage and update networking policies in cloud data centers, Virtual Network Functions (VNFs) use, and therefore actively collect, networking state information - and in the process, incur additional control signaling and management overhead, especially in larger data centers. In the meantime, VNFs in production prefer distributed and straightforward heuristics over advanced learning algorithms to avoid intractable additional processing latency under high-performance and low-latency networking constraints. This paper identifies the challenges of deploying learning algorithms in the context of cloud data centers, and proposes Aquarius to bridge the application of machine learning (ML) techniques on distributed systems and service management. Aquarius passively yet efficiently gathers reliable observations, and enables the use of ML techniques to collect, infer, and supply accurate networking state information—without incurring additional signaling and management overhead. It offers fine-grained and programmable visibility to distributed VNFs, and enables both open- and close-loop control over networking systems. This paper illustrates the use of Aquarius with a traffic classifier, an auto-scaling system, and a load balancer—and demonstrates the use of three different ML paradigms—unsupervised, supervised, and reinforcement learning, within Aquarius, for network state inference and service management. Testbed evaluations show that Aquarius suitably improves network state visibility and brings notable performance gains for various scenarios with low overhead.

Finite-size correlation behavior near a critical point: A simple metric for monitoring the state of a neural network.

In this article, a correlation metric κ_{c} is proposed for the inference of the dynamical state of neuronal networks. κ_{C} is computed from the scaling of the correlation length with the size of the observation region, which shows qualitatively different behavior near and away from the critical point of a continuous phase transition. The implementation is first studied on a neuronal network model, where the results of this new metric coincide with those obtained from neuronal avalanche analysis, thus well characterizing the critical state of the network. The approach is further tested with brain optogenetic recordings in behaving mice from a publicly available database. Potential applications and limitations for its use with currently available optical imaging techniques are discussed.

Partially Hidden Markov Chain Multivariate Linear Autoregressive model: inference and forecasting—application to machine health prognostics

Time series subject to regime shifts have attracted much interest in domains such as econometry, finance or meteorology. For discrete-valued regimes, models such as the popular Hidden Markov Chain (HMC) describe time series whose state process is unknown at all time-steps. Sometimes, time series are annotated. Thus, another category of models handles the case with regimes observed at all time-steps. We present a novel model which addresses the intermediate case: (i) state processes associated to such time series are modelled by Partially Hidden Markov Chains (PHMCs); (ii) a multivariate linear autoregressive (MLAR) model drives the dynamics of the time series, within each regime. We describe a variant of the expectation maximization (EM) algorithm devoted to PHMC-MLAR model learning. We propose a hidden state inference procedure and a forecasting function adapted to the semi-supervised framework. We first assess inference and prediction performances, and analyze EM convergence times for PHMC-MLAR, using simulated data. We show the benefits of using partially observed states as well as a fully labelled scheme with unreliable labels, to decrease EM convergence times. We highlight the robustness of PHMC-MLAR to labelling errors in inference and prediction tasks. Finally, using turbofan engine data from a NASA repository, we show that PHMC-MLAR outperforms or largely outperforms other models: PHMC and MSAR (Markov Switching AutoRegressive model) for the feature prediction task, PHMC and five out of six recent state-of-the-art methods for the prediction of machine useful remaining life.

A Markov-switching hidden heterogeneous network autoregressive model for multivariate time series data with multimodality

Multivariate networked time series data are ubiquitous in many applications, where multiple variables of interest are sequentially collected over time for each vertex as multivariate time series. Data of different vertices may have heterogeneous influence on each other through the network topology. These time series may usually exhibit multimodal marginal distributions, due to complex system variations. In this article, we propose a novel approach for such data modeling. In particular, we assume that each vertex has multiple latent states and exhibits state-switching behaviors according to a Markov process. The multivariate time series of each vertex depend on a defined latent effect variable influenced by both its own latent state and the latent effects of its neighbors through a heterogeneous network autoregressive model according to the network topology. Furthermore, the influence of some exogenous covariates on the time series can also be incorporated in the model. Some model properties are discussed, and a variational EM algorithm is proposed for model parameter estimation and state inference. Extensive synthetic experiments and a real-world case study demonstrate the effectiveness and applicability of the proposed model.

Deep reinforcement learning achieves multifunctional morphing airfoil control

Smooth camber morphing aircraft offer increased control authority and improved aerodynamic efficiency. Smart material actuators have become a popular driving force for shape changes, capable of adhering to weight and size constraints and allowing for simplicity in mechanical design. As a step towards creating uncrewed aerial vehicles (UAVs) capable of autonomously responding to flow conditions, this work examines a multifunctional morphing airfoil’s ability to follow commands in various flows. We integrated an airfoil with a morphing trailing edge consisting of an antagonistic pair of macro fiber composites (MFCs), serving as both skin and actuator, and internal piezoelectric flex sensors to form a closed loop composite system. Closed loop feedback control is necessary to accurately follow deflection commands due to the hysteretic behavior of MFCs. Here we used a deep reinforcement learning algorithm, Proximal Policy Optimization, to control the morphing airfoil. Two neural controllers were trained in a simulation developed through time series modeling on long short-term memory recurrent neural networks. The learned controllers were then tested on the composite wing using two state inference methods in still air and in a wind tunnel at various flow speeds. We compared the performance of our neural controllers to one using traditional position-derivative feedback control methods. Our experimental results validate that the autonomous neural controllers were faster and more accurate than traditional methods. This research shows that deep learning methods can overcome common obstacles for achieving sufficient modeling and control when implementing smart composite actuators in an autonomous aerospace environment.

Hidden Mixture Vehicle Discharge State Inference at Signalized Intersection Using Vehicle Travel Time and Discharge Headway Data

Accurate and reliable traffic state identification is crucial to developing responsive and proactive traffic management applications. In this study, the problem of vehicle discharge state identification at signalized intersections is investigated, which focuses on the vehicle discharge process during the green interval. Instead of using detailed vehicle trajectory data and treating the observations of vehicles independently, this study formulates the vehicle discharge process in a Hidden Markov Model (HMM) framework using sequential observations of vehicle travel time and discharge headway as inputs. Three vehicle discharge states (i.e., overflow, single stop, and free arrival) are encoded as latent states, and a restricted left-to-right state transition matrix is imposed to respect the nature of the vehicle discharge process in the real world. The standard HMM is further extended to incorporate two informative covariates to parameterize the probabilities of the initial states and state transitions. The proposed models have been validated on the Next Generation Simulation (NGSIM) dataset. Compared to a benchmark model, the proposed models show their strength in correctly inferring the vehicle discharge state and are more reliable to use in presence of random missing observations. The effectiveness of covariate incorporation is also investigated, and several extended applications of the proposed models are discussed.

A novel system identification algorithm for nonlinear Markov jump system

System identification of nonlinear Markov jump systems (NMJSs) is crucial in modeling complex systems that contain unknown continuous and discrete states. In this system, the dynamics of continuous states is governed by the discrete states, which means the system can operate among different modes. The existing studies primarily identify the mode transition probability and noise model, whereas the parameters concerning the nonlinear continuous state transition function and observation function are commonly not considered. To identify all the parameters, a novel NMJSs identification algorithm based on a new particle system is proposed. The hidden state inference is implemented via an extended smoother that embodies the new particle system and Rao-Blackwellized particle Gibbs ancestor sampling kernel. The smoother not only estimates hidden state but also enables parameter update of the nonlinear functions. After inferencing the hidden states’ posterior density, a log-likelihood-based loss function is utilized to reduce the uncertainty of identifying the parameters related to the continuous state transition function. Then expectation–maximization and gradient descent are adopted to update the parameters. Finally, different experiments are conducted to verify the proposed algorithm. The results show that the proposed algorithm can approximate the parameter that well describes the data, and outperform other related approaches.

Empathy and correct mental state inferences both promote prosociality

In a world with rapidly increasing population that competes for the earth’s limited resources, cooperation is crucial. While research showed that empathizing with another individual in need enhances prosociality, it remains unclear whether correctly inferring the other’s inner, mental states on a more cognitive level (i.e., mentalizing) elicits helping behavior as well. We applied a video-based laboratory task probing empathy and a performance measure of mentalizing in adult volunteers (N = 94) and assessed to which extent they were willing to help the narrators in the videos. We replicate findings that an empathy induction leads to more prosocial decisions. Crucially, we also found that correct mentalizing increases the willingness to help. This evidence helps clarify an inconsistent picture of the relation between mentalizing and prosociality.

A Temporal–Spatial Deep Learning Approach for Driver Distraction Detection Based on EEG Signals

Distracted driving has been recognized as a major challenge to traffic safety improvement. This article presents a novel driving distraction detection method that is based on a new deep network. Unlike traditional methods, the proposed method uses both temporal information and spatial information of electroencephalography (EEG) signals as model inputs. Convolutional techniques and gated recurrent units were adopted to map the relationship between drivers’ distraction status and EEG signals in the time domain. A driving simulation experiment was conducted to examine the effectiveness of the proposed method. Twenty-four healthy volunteers participated and three types of secondary tasks (i.e., cellphone operation task, clock task, and 2-back task) were used to induce distraction during driving. Drivers’ EEG responses were measured using a 32-channel electrode cap, and the EEG signals were preprocessed to remove artifacts and then split into short EEG sequences. The proposed deep-network-based distraction detection method was trained and tested on the collected EEG data. To evaluate its effectiveness, it was also compared with the networks using temporal or spatial information alone. The results showed that our proposed distraction detection method achieved an overall binary (distraction versus nondistraction) classification accuracy of 0.92. In terms of task-specific distraction detection, its accuracy was 0.88. Further analysis on the individual difference in detection performance showed that drivers’ EEG performance differed across individuals, which suggests that adaptive learning for each individual driver would be needed when developing in-vehicle distraction detection applications. Note to Practitioners—Driver distraction detection is crucial for safety enhancement to avoid crashes caused by nondriving-related activities, such as calling and texting while driving. Related previous studies mainly focus on detection by monitoring head and eye movement using computer vision technologies or by extracting indicators from driving performance measures for driver state inference. However, complex traffic environments (e.g., dynamically changing light distribution on driver’s face and nighttime driving with low illumination) strongly limit the effectiveness of computer vision technologies, and the driving performance characteristics may also be caused by factors other than distraction (e.g., fatigue). To solve these problems, this article seeks to develop a deep learning-based approach to map the unique relationship between driver distraction and the bioelectric electroencephalography (EEG) signals that are not affected by traffic environments. The proposed method can be integrated into the driver assistance systems and autonomous vehicles to deal with emergency situations that need drivers to handle. The timely detection of distraction by our method will significantly facilitate its practical applications in collision avoidance or danger mitigation in the handover process.

Species delimitation of the liquorice tribe (Leguminosae: Glycyrrhizeae) based on phylogenomic and machine learning analyses

AbstractThe liquorice tribe Glycyrrhizeae is a leguminous herbaceous group of plants comprised of the genera Glycyrrhiza and Glycyrrhizopsis. Some Glycyrrhiza taxa contain glycyrrhizin, a pharmacologically significant sweet substance that also has applications in crafting industrial materials. Here, we utilized an expanded taxon sampling of Glycyrrhizeae to reconstruct the phylogenetic relationships in the tribe based on genome skimming data, including whole chloroplast genomes, nuclear ribosomal DNA, and low‐copy nuclear DNA. We also launched machine learning analysis (MLA) for one species pair with controversial taxonomic boundary. The integrated results indicated Glycyrrhizopsis should be split from Glycyrrhiza, while the former genus Meristotropis should be treated as part of Glycyrrhiza. Glycyrrhizopsis includes two species, Glycyrrhizopsis asymmetrica and Glycyrrhizopsis flavescens, and we recognize 13 species in Glycyrrhiza: Glycyrrhiza acanthocarpa, Glycyrrhiza astragalina, Glycyrrhiza bucharica, Glycyrrhiza echinata, Glycyrrhiza foetida, Glycyrrhiza glabra, Glycyrrhiza gontscharovii, Glycyrrhiza lepidota, Glycyrrhiza macedonica, Glycyrrhiza pallidiflora, Glycyrrhiza squamulosa, Glycyrrhiza triphylla, and Glycyrrhiza yunnanensis. We propose a broader G. glabra that includes former Glycyrrhiza aspera, G. glabra s.s., Glycyrrhiza inflata, and Glycyrrhiza uralensis, and represents the glycyrrhizin‐contained medicinal group. Our ancestral state inferences show the ancestor of Glycyrrhiza lacked glycyrrhizin, and the presence of glycyrrhizin evolved twice within Glycyrrhiza during the last one million years. Our integrative phylogenomics‐MLA study not only provides new insights into long‐standing taxonomic controversies of Glycyrrhizeae, but also represents a useful approach for future taxonomic studies on other plant taxa.

Optimal control methods for nonlinear parameter estimation in biophysical neuron models.

Functional forms of biophysically-realistic neuron models are constrained by neurobiological and anatomical considerations, such as cell morphologies and the presence of known ion channels. Despite these constraints, neuron models still contain unknown static parameters which must be inferred from experiment. This inference task is most readily cast into the framework of state-space models, which systematically takes into account partial observability and measurement noise. Inferring only dynamical state variables such as membrane voltages is a well-studied problem, and has been approached with a wide range of techniques beginning with the well-known Kalman filter. Inferring both states and fixed parameters, on the other hand, is less straightforward. Here, we develop a method for joint parameter and state inference that combines traditional state space modeling with chaotic synchronization and optimal control. Our methods are tailored particularly to situations with considerable measurement noise, sparse observability, very nonlinear or chaotic dynamics, and highly uninformed priors. We illustrate our approach both in a canonical chaotic model and in a phenomenological neuron model, showing that many unknown parameters can be uncovered reliably and accurately from short and noisy observed time traces. Our method holds promise for estimation in larger-scale systems, given ongoing improvements in calcium reporters and genetically-encoded voltage indicators.

Glucose consumption rate-dependent transcriptome profiling of Escherichia coli provides insight on performance as microbial factories

BackgroundThe modification of glucose import capacity is an engineering strategy that has been shown to improve the characteristics of Escherichia coli as a microbial factory. A reduction in glucose import capacity can have a positive effect on production strain performance, however, this is not always the case. In this study, E. coli W3110 and a group of four isogenic derivative strains, harboring single or multiple deletions of genes encoding phosphoenolpyruvate:sugar phosphotransferase system (PTS)-dependent transporters as well as non-PTS transporters were characterized by determining their transcriptomic response to reduced glucose import capacity.ResultsThese strains were grown in bioreactors with M9 mineral salts medium containing 20 g/L of glucose, where they displayed specific growth rates ranging from 0.67 to 0.27 h−1, and specific glucose consumption rates (qs) ranging from 1.78 to 0.37 g/g h. RNA-seq analysis revealed a transcriptional response consistent with carbon source limitation among all the mutant strains, involving functions related to transport and metabolism of alternate carbon sources and characterized by a decrease in genes encoding glycolytic enzymes and an increase in gluconeogenic functions. A total of 107 and 185 genes displayed positive and negative correlations with qs, respectively. Functions displaying positive correlation included energy generation, amino acid biosynthesis, and sugar import.ConclusionChanges in gene expression of E. coli strains with impaired glucose import capacity could be correlated with qs values and this allowed an inference of the physiological state of each mutant. In strains with lower qs values, a gene expression pattern is consistent with energy limitation and entry into the stationary phase. This physiological state could explain why these strains display a lower capacity to produce recombinant protein, even when they show very low rates of acetate production. The comparison of the transcriptomes of the engineered strains employed as microbial factories is an effective approach for identifying favorable phenotypes with the potential to improve the synthesis of biotechnological products.

Does the TPJ fit it all? Representational similarity analysis of different forms of mentalizing

ABSTRACT Mentalizing is the key socio-cognitive ability. Its heterogeneous structure may result from a variety of forms of mental state inference, which may be based on lower-level processing of cues encoded in the observable behavior of others, or rather involve higher-level computations aimed at understanding another person’s perspective. Here we aimed to investigate the representational content of the brain regions engaged in mentalizing. To this end, 61 healthy adults took part in an fMRI study. We explored ROI activity patterns associated with five well-recognized ToM tasks that induce either decoding of mental states from motion kinematics or belief-reasoning. By using multivariate representational similarity analysis, we examined whether these examples of lower- and higher-level forms of social inference induced common or distinct patterns of brain activity. Distinct patterns of brain activity related to decoding of mental states from motion kinematics and belief-reasoning were found in lTPJp and the left IFG, but not the rTPJp. This may indicate that rTPJp supports a general mechanism for the representation of mental states. The divergent patterns of activation in lTPJp and frontal areas likely reflect differences in the degree of involvement of cognitive functions which support the basic mentalizing processes engaged by the two task groups.