Online Inference Research Articles

An Online Causal Inference Framework for Modeling and Designing Systems Involving User Preferences: A State-Space Approach

We provide a causal inference framework to model the effects of machine learning algorithms on user preferences. We then use this mathematical model to prove that the overall system can be tuned to alter those preferences in a desired manner. A user can be an online shopper or a social media user, exposed to digital interventions produced by machine learning algorithms. A user preference can be anything from inclination towards a product to a political party affiliation. Our framework uses a state-space model to represent user preferences as latent system parameters which can only be observed indirectly via online user actions such as a purchase activity or social media status updates, shares, blogs, or tweets. Based on these observations, machine learning algorithms produce digital interventions such as targeted advertisements or tweets. We model the effects of these interventions through a causal feedback loop, which alters the corresponding preferences of the user. We then introduce algorithms in order to estimate and later tune the user preferences to a particular desired form. We demonstrate the effectiveness of our algorithms through experiments in different scenarios.

Online Object Tracking, Learning and Parsing with And-Or Graphs.

This paper presents a method, called AOGTracker, for simultaneously tracking, learning and parsing (TLP) of unknown objects in video sequences with a hierarchical and compositional And-Or graph (AOG) representation. The TLP method is formulated in the Bayesian framework with a spatial and a temporal dynamic programming (DP) algorithms inferring object bounding boxes on-the-fly. During online learning, the AOG is discriminatively learned using latent SVM[1] to account for appearance (e.g., lighting and partial occlusion) and structural (e.g., different poses and viewpoints) variations of a tracked object, as well as distractors (e.g., similar objects) in background. Three key issues in online inference and learning are addressed: (i) maintaining purity of positive and negative examples collected online, (ii) controling model complexity in latent structure learning, and (iii) identifying critical moments to re-learn the structure of AOG based on its intrackability. The intrackability measures uncertainty of an AOG based on its score maps in a frame. In experiments, our AOGTracker is tested on two popular tracking benchmarks with the same parameter setting: the TB-100/50/CVPR2013 benchmarks , [3] , and the VOT benchmarks[4] -VOT 2013, 2014, 2015 and TIR2015 (thermal imagery tracking). In the former, our AOGTracker outperforms state-of-the-art tracking algorithms including two trackers based on deep convolutional network [5] , [6] . In the latter, our AOGTracker outperforms all other trackers in VOT2013 and is comparable to the state-of-the-art methods in VOT2014, 2015 and TIR2015.

Influence of different open circuit voltage tests on state of charge online estimation for lithium-ion batteries

Battery state of charge (SOC) estimation is a crucial function of battery management systems (BMSs), since accurate estimated SOC is critical to ensure the safety and reliability of electric vehicles. A widely used technique for SOC estimation is based on online inference of battery open circuit voltage (OCV). Low-current OCV and incremental OCV tests are two common methods to observe the OCV-SOC relationship, which is an important element of the SOC estimation technique. In this paper, two OCV tests are run at three different temperatures and based on which, two SOC estimators are compared and evaluated in terms of tracking accuracy, convergence time, and robustness for online estimating battery SOC. The temperature dependency of the OCV-SOC relationship is investigated and its influence on SOC estimation results is discussed. In addition, four dynamic tests are presented, one for estimator parameter identification and the other three for estimator performance evaluation. The comparison results show that estimator 2 (based on the incremental OCV test) has higher tracking accuracy and is more robust against varied loading conditions and different initial values of SOC than estimator 1 (based on the low-current OCV test) with regard to ambient temperature. Therefore, the incremental OCV test is recommended for predetermining the OCV-SOCs for battery SOC online estimation in BMSs.

A fault diagnosis method based on signed directed graph and matrix for nuclear power plants

In order to solve SDG online fault diagnosis and inference, matrix diagnosis and inference methods are proposed for fault detection and diagnosis (FDD). Firstly, “rules matrix” based on SDG model is used for FDD. Secondly, “status matrix” is proposed to achieve SDG online inference. According to different diagnosis results, “status matrix” is applied for the depth-first search and the breadth-first search respectively to find the propagation paths of each fault. Finally, the SDG model of the secondary-loop system in pressurized water reactor (PWR) is built to verify the effectiveness of the proposed method. The simulation experiment results indicate that the “status matrix” used for online inference can be used to find the fault propagation paths and to explain the causes for fault. Therefore, it can be concluded that the proposed method is one of the fault diagnosis for nuclear power plants (NPPs), which can be used to facilitate the development of fault diagnostic system.

Bayesian Particle Tracking of Traffic Flows

We develop a Bayesian particle filter for tracking traffic flows that is capable of capturing non-linearities and discontinuities present in flow dynamics. Our model includes a hidden state variable that captures sudden regime shifts between traffic free flow, breakdown and recovery. We develop an efficient particle learning algorithm for real time on-line inference of states and parameters. This requires a two step approach, first, resampling the current particles, with a mixture predictive distribution and second, propagation of states using the conditional posterior distribution. Particle learning of parameters follows from updating recursions for conditional sufficient statistics. To illustrate our methodology, we analyze measurements of daily traffic flow from the Illinois interstate I-55 highway system. We demonstrate how our filter can be used to inference the change of traffic flow regime on a highway road segment based on a measurement from freeway single-loop detectors. Finally, we conclude with directions for future research.

Modelling Odor Decoding in the Antennal Lobe by Combining Sequential Firing Rate Models with Bayesian Inference.

The olfactory information that is received by the insect brain is encoded in the form of spatiotemporal patterns in the projection neurons of the antennal lobe. These dense and overlapping patterns are transformed into a sparse code in Kenyon cells in the mushroom body. Although it is clear that this sparse code is the basis for rapid categorization of odors, it is yet unclear how the sparse code in Kenyon cells is computed and what information it represents. Here we show that this computation can be modeled by sequential firing rate patterns using Lotka-Volterra equations and Bayesian online inference. This new model can be understood as an ‘intelligent coincidence detector’, which robustly and dynamically encodes the presence of specific odor features. We found that the model is able to qualitatively reproduce experimentally observed activity in both the projection neurons and the Kenyon cells. In particular, the model explains mechanistically how sparse activity in the Kenyon cells arises from the dense code in the projection neurons. The odor classification performance of the model proved to be robust against noise and time jitter in the observed input sequences. As in recent experimental results, we found that recognition of an odor happened very early during stimulus presentation in the model. Critically, by using the model, we found surprising but simple computational explanations for several experimental phenomena.

The relative roles of visuospatial and linguistic working memory systems in generating inferences during visual narrative comprehension.

This study investigated the relative roles of visuospatial versus linguistic working memory (WM) systems in the online generation of bridging inferences while viewers comprehend visual narratives. We contrasted these relative roles in the visuospatial primacy hypothesis versus the shared (visuospatial & linguistic) systems hypothesis, and tested them in 3 experiments. Participants viewed picture stories containing multiple target episodes consisting of a beginning state, a bridging event, and an end state, respectively, and the presence of the bridging event was manipulated. When absent, viewers had to infer the bridging-event action to comprehend the end-state image. A pilot study showed that after viewing the end-state image, participants' think-aloud protocols contained more inferred actions when the bridging event was absent than when it was present. Likewise, Experiment 1 found longer viewing times for the end-state image when the bridging-event image was absent, consistent with viewing times revealing online inference generation processes. Experiment 2 showed that both linguistic and visuospatial WM loads attenuated the inference viewing time effect, consistent with the shared systems hypothesis. Importantly, however, Experiment 3 found that articulatory suppression did not attenuate the inference viewing time effect, indicating that (sub)vocalization did not support online inference generation during visual narrative comprehension. Thus, the results support a shared-systems hypothesis in which both visuospatial and linguistic WM systems support inference generation in visual narratives, with the linguistic WM system operating at a deeper level than (sub)vocalization.

Evolutionary Sequential Monte Carlo Samplers for Change-Point Models

Sequential Monte Carlo (SMC) methods are widely used for non-linear filtering purposes. Nevertheless the SMC scope encompasses wider applications such as estimating static model parameters so much that it is becoming a serious alternative to Markov-Chain Monte-Carlo (MCMC) methods. Not only SMC algorithms draw posterior distributions of static or dynamic parameters but additionally provide an estimate of the marginal likelihood. The tempered and time (TNT) algorithm, developed in the paper, combines (off-line) tempered SMC inference with on-line SMC inference for drawing realizations from many sequential posterior distributions without experiencing a particle degeneracy problem. Furthermore, it introduces a new MCMC rejuvenation step that is generic, automated and well-suited for multi-modal distributions. As this update relies on the wide heuristic optimization literature, numerous extensions are already available. The algorithm is notably appropriate for estimating Change-point models. As an example, we compare Change-point GARCH models through their marginal likelihoods over time.

Surrogate Modeling Approach to Support Real-Time Structural Assessment and Decision Making

This paper proposes a data-driven strategy to assist online rapid decision making for an unmanned aerial vehicle that uses sensed data to estimate its structural state, uses this estimate to update its corresponding flight capabilities, and then dynamically replans its mission accordingly. The approach comprises offline and online computational phases constructed to address the sense–plan–act information flow while avoiding a costly online inference step. During the offline phase, high-fidelity finite element simulations are used to construct reduced-order models and classification criteria: proper orthogonal decomposition approximations and self-organizing maps are combined to realize a fast mapping from measured quantities to system capabilities. During the online phase, the surrogate mapping is employed to directly estimate the vehicle’s evolving structural capability from sensor data. The approach is demonstrated for a test problem of a composite wing panel on an unmanned aerial vehicle that undergoes degradation in structural properties.

Divide et impera: subgoaling reduces the complexity of probabilistic inference and problem solving.

It has long been recognized that humans (and possibly other animals) usually break problems down into smaller and more manageable problems using subgoals. Despite a general consensus that subgoaling helps problem solving, it is still unclear what the mechanisms guiding online subgoal selection are during the solution of novel problems for which predefined solutions are not available. Under which conditions does subgoaling lead to optimal behaviour? When is subgoaling better than solving a problem from start to finish? Which is the best number and sequence of subgoals to solve a given problem? How are these subgoals selected during online inference? Here, we present a computational account of subgoaling in problem solving. Following Occam's razor, we propose that good subgoals are those that permit planning solutions and controlling behaviour using less information resources, thus yielding parsimony in inference and control. We implement this principle using approximate probabilistic inference: subgoals are selected using a sampling method that considers the descriptive complexity of the resulting sub-problems. We validate the proposed method using a standard reinforcement learning benchmark (four-rooms scenario) and show that the proposed method requires less inferential steps and permits selecting more compact control programs compared to an equivalent procedure without subgoaling. Furthermore, we show that the proposed method offers a mechanistic explanation of the neuronal dynamics found in the prefrontal cortex of monkeys that solve planning problems. Our computational framework provides a novel integrative perspective on subgoaling and its adaptive advantages for planning, control and learning, such as for example lowering cognitive effort and working memory load.

Efficient particle-based online smoothing in general hidden Markov models: The PaRIS algorithm

This thesis consists of two papers studying online inference in general hidden Markov models using sequential Monte Carlo methods.The first paper present an novel algorithm, the particle-based, rapid incremental smoother (PaRIS), aimed at efficiently perform online approximation of smoothed expectations of additive state functionals in general hidden Markov models. The algorithm has, under weak assumptions, linear computational complexity and very limited memory requirements. The algorithm is also furnished with a number of convergence results, including a central limit theorem.The second paper focuses on the problem of online estimation of parameters in a general hidden Markov model. The algorithm is based on a forward implementation of the classical expectation-maximization algorithm. The algorithm uses the PaRIS algorithm to achieve an efficient algorithm.

Integrating Social and Auxiliary Semantics for Multifaceted Topic Modeling in Twitter

Microblogging platforms, such as Twitter, have already played an important role in recent cultural, social and political events. Discovering latent topics from social streams is therefore important for many downstream applications, such as clustering, classification or recommendation. However, traditional topic models that rely on the bag-of-words assumption are insufficient to uncover the rich semantics and temporal aspects of topics in Twitter. In particular, microblog content is often influenced by external information sources, such as Web documents linked from Twitter posts, and often focuses on specific entities, such as people or organizations. These external sources provide useful semantics to understand microblogs and we generally refer to these semantics as auxiliary semantics . In this article, we address the mentioned issues and propose a unified framework for Multifaceted Topic Modeling from Twitter streams. We first extract social semantics from Twitter by modeling the social chatter associated with hashtags. We further extract terms and named entities from linked Web documents to serve as auxiliary semantics during topic modeling. The Multifaceted Topic Model (MfTM) is then proposed to jointly model latent semantics among the social terms from Twitter, auxiliary terms from the linked Web documents and named entities. Moreover, we capture the temporal characteristics of each topic. An efficient online inference method for MfTM is developed, which enables our model to be applied to large-scale and streaming data. Our experimental evaluation shows the effectiveness and efficiency of our model compared with state-of-the-art baselines. We evaluate each aspect of our framework and show its utility in the context of tweet clustering.

RISE: An Incremental Trust-Region Method for Robust Online Sparse Least-Squares Estimation

Many point estimation problems in robotics, computer vision, and machine learning can be formulated as instances of the general problem of minimizing a sparse nonlinear sum-of-squares objective function. For inference problems of this type, each input datum gives rise to a summand in the objective function, and therefore performing online inference corresponds to solving a sequence of sparse nonlinear least-squares minimization problems in which additional summands are added to the objective function over time. In this paper, we present Robust Incremental least-Squares Estimation (RISE), an incrementalized version of the Powell's Dog-Leg numerical optimization method suitable for use in online sequential sparse least-squares minimization. As a trust-region method, RISE is naturally robust to objective function nonlinearity and numerical ill-conditioning and is provably globally convergent for a broad class of inferential cost functions (twice-continuously differentiable functions with bounded sublevel sets). Consequently, RISE maintains the speed of current state-of-the-art online sparse least-squares methods while providing superior reliability.

Two-stage online inference model for traffic pattern analysis and anomaly detection

In this paper, we propose a method for modeling trajectory patterns with both regional and velocity observations through the probabilistic topic model. By embedding Gaussian models into the discrete topic model framework, our method uses continuous velocity as well as regional observations unlike existing approaches. In addition, the proposed framework combined with Hidden Markov Model can cover the temporal transition of the scene state, which is useful in checking a violation of the rule that some conflict topics (e.g. two cross-traffic patterns) should not occur at the same time. To achieve online learning even with the complexity of the proposed model, we suggest a novel learning scheme instead of collapsed Gibbs sampling. The proposed two-stage greedy learning scheme is not only efficient at reducing the search space but also accurate in a way that the accuracy of online learning becomes not worse than that of the batch learning. To validate the performance of our method, experiments were conducted on various datasets. Experimental results show that our model explains satisfactorily the trajectory patterns with respect to scene understanding, anomaly detection, and prediction.

Topological Mapping and Scene Recognition With Lightweight Color Descriptors for an Omnidirectional Camera

Scene recognition problems for mobile robots have been extensively studied. This is important for tasks such as visual topological mapping. Usually, sophisticated key-point-based descriptors are used, which can be computationally expensive. In this paper, we describe a lightweight novel scene recognition method using an adaptive descriptor, which is based on color features and geometric information that are extracted from an uncalibrated omnidirectional camera. The proposed method enables a mobile robot to perform online registration of new scenes onto a topological representation automatically and solve the localization problem to topological regions simultaneously, all in real time. We adopt a Dirichlet process mixture model (DPMM) to describe the online inference process. It is based on an approximation of conditional probabilities of the new measurements given incrementally estimated reference models. It enables online inference speeds of up to 50 Hz for a normal CPU. We compare it with state-of-the-art key-point descriptors and show the advantage of the proposed algorithm in terms of performance and computational efficiency. A real-world experiment is carried out with a mobile robot equipped with an omnidirectional camera. Finally, we show the results on extended datasets.

Dynamic joint sentiment-topic model

Social media data are produced continuously by a large and uncontrolled number of users. The dynamic nature of such data requires the sentiment and topic analysis model to be also dynamically updated, capturing the most recent language use of sentiments and topics in text. We propose a dynamic Joint Sentiment-Topic model (dJST) which allows the detection and tracking of views of current and recurrent interests and shifts in topic and sentiment. Both topic and sentiment dynamics are captured by assuming that the current sentiment-topic-specific word distributions are generated according to the word distributions at previous epochs. We study three different ways of accounting for such dependency information: (1) sliding window where the current sentiment-topic word distributions are dependent on the previous sentiment-topic-specific word distributions in the last S epochs; (2) skip model where history sentiment topic word distributions are considered by skipping some epochs in between; and (3) multiscale model where previous long- and short- timescale distributions are taken into consideration. We derive efficient online inference procedures to sequentially update the model with newly arrived data and show the effectiveness of our proposed model on the Mozilla add-on reviews crawled between 2007 and 2011.

State of charge estimation of lithium-ion batteries using the open-circuit voltage at various ambient temperatures

Ambient temperature is a significant factor that influences the accuracy of battery SOC estimation, which is critical for remaining driving range prediction of electric vehicles (EVs) and optimal charge/discharge control of batteries. A widely used method to estimate SOC is based on an online inference of open-circuit voltage (OCV). However, the fact that the OCV–SOC is dependent on ambient temperature can result in errors in battery SOC estimation. To address this problem, this paper presents an SOC estimation approach based on a temperature-based model incorporated with an OCV–SOC–temperature table. The unscented Kalman filtering (UKF) was applied to tune the model parameters at each sampling step to cope with various uncertainties arising from the operation environment, cell-to-cell variation, and modeling inaccuracy. Two dynamic tests, the dynamic stress test (DST) and the federal urban driving schedule (FUDS), were used to test batteries at different temperatures. Then, DST was used to identify the model parameters while FUDS was used to validate the performance of the SOC estimation. The estimation was made covering the major working range from 25% to 85% SOC. The results indicated that our method can provide accurate SOC estimation with smaller root mean squared errors than the method that does not take into account ambient temperature. Thus, our approach is effective and accurate when battery operates at different ambient temperatures. Since the developed method takes into account the temperature factor as well as the complexity of the model, it could be effectively applied in battery management systems for EVs.

Model-based clustering of high-dimensional data streams with online mixture of probabilistic PCA

Model-based clustering is a popular tool which is renowned for its probabilistic foundations and its flexibility. However, model-based clustering techniques usually perform poorly when dealing with high-dimensional data streams, which are nowadays a frequent data type. To overcome this limitation of model-based clustering, we propose an online inference algorithm for the mixture of probabilistic PCA model. The proposed algorithm relies on an EM-based procedure and on a probabilistic and incremental version of PCA. Model selection is also considered in the online setting through parallel computing. Numerical experiments on simulated and real data demonstrate the effectiveness of our approach and compare it to state-of-the-art online EM-based algorithms.

Influence relation estimation based on lexical entrainment in conversation

In conversations, people tend to mimic their companions’ behavior depending on their level of trust. This phenomenon is known as entrainment. We propose a probabilistic model for estimating influences among speakers from conversation data involving multiple people by modeling lexical entrainment. The proposed model estimates word use as a function of the weighted sum of the earlier word use of other speakers. The weights represent influences between speakers. The influences can be efficiently estimated by using the expectation maximization (EM) algorithm. We also develop its online inference procedures for sequentially modeling the dynamics of influence relations. Experiments performed on two meeting data sets one in Japanese and one in English demonstrate the effectiveness of the proposed method.

Sequential Modeling of Topic Dynamics with Multiple Timescales

We propose an online topic model for sequentially analyzing the time evolution of topics in document collections. Topics naturally evolve with multiple timescales. For example, some words may be used consistently over one hundred years, while other words emerge and disappear over periods of a few days. Thus, in the proposed model, current topic-specific distributions over words are assumed to be generated based on the multiscale word distributions of the previous epoch. Considering both the long- and short-timescale dependency yields a more robust model. We derive efficient online inference procedures based on a stochastic EM algorithm, in which the model is sequentially updated using newly obtained data; this means that past data are not required to make the inference. We demonstrate the effectiveness of the proposed method in terms of predictive performance and computational efficiency by examining collections of real documents with timestamps.