Probabilistic Perspective Research Articles

A unifying probabilistic perspective for spectral dimensionality reduction: insights and new models

We introduce a new perspective on spectral dimensionality reduction which views these methods as Gaussian Markov random fields (GRFs). Our unifying perspective is based on the maximum entropy principle which is in turn inspired by maximum variance unfolding. The resulting model, which we call maximum entropy unfolding (MEU) is a nonlinear generalization of principal component analysis. We relate the model to Laplacian eigenmaps and isomap. We show that parameter fitting in the locally linear embedding (LLE) is approximate maximum likelihood MEU. We introduce a variant of LLE that performs maximum likelihood exactly: Acyclic LLE (ALLE). We show that MEU and ALLE are competitive with the leading spectral approaches on a robot navigation visualization and a human motion capture data set. Finally the maximum likelihood perspective allows us to introduce a new approach to dimensionality reduction based on L1 regularization of the Gaussian random field via the graphical lasso.

Structured learning of local features for human action classification and localization

Human action recognition is a promising yet non-trivial computer vision field with many potential applications. Current advances in bag-of-feature approaches have brought significant insights into recognizing human actions within complex context. It is, however, a common practice in literature to consider action as merely an orderless set of local salient features. This representation has been shown to be oversimplified, which inherently limits traditional approaches from robust deployment in real-life scenarios. In this work, we propose and show that, by taking into account global configuration of local features, we can greatly improve recognition performance. We first introduce a novel feature selection process called Sparse Hierarchical Bayes Filter to select only the most contributive features of each action type based on neighboring structure constraints. We then present the application of structured learning in human action analysis. That is, by representing human action as a complex set of local features, we can incorporate different spatial and temporal feature constraints into the learning tasks of human action classification and localization. In particular, we tackle the problem of action localization in video using structured learning with two alternatives: one is Dynamic Conditional Random Field from probabilistic perspective; the other is Structural Support Vector Machine from max-margin point of view. We evaluate our modular classification-localization framework on various testbeds, in which our proposed framework is proven to be highly effective and robust compared against bag-of-feature methods.

The bei passive and its discourse motivations

This study examines the variation between the active and the bei passive in Mandarin Chinese from a probabilistic perspective. The variables considered include discourse continuity factors and adversity. Two different models were built for the active-agentless passive variation and the active-agentive passive variation. Four factors were found to have significant effect: agent thematicity, patient thematicity, adversity, and referential distance. In contrast, the effect of topic persistence and local environment is not significant. The accuracy of prediction for the active-agentless passive variation is significantly higher than the accuracy for the active-agentive passive variation. Overall, the bei passive, either agentless or agentive, is more likely to be chosen over its active counterpart, if it is adversative, has a non-thematic agent, a thematic patient, and a shorter referential distance for the patient.

Information causality from an entropic and a probabilistic perspective

The information causality principle is a generalisation of the no-signalling principle which implies some of the known restrictions on quantum correlations. But despite its clear physical motivation, information causality is formulated in terms of a rather specialised game and figure of merit. We explore different perspectives on information causality, discussing the probability of success as the figure of merit, a relation between information causality and the non-local `inner-product game', and the derivation of a quadratic bound for these games. We then examine an entropic formulation of information causality with which one can obtain the same results, arguably in a simpler fashion.

Auto-Regressive Processes Explained by Self-Organized Maps. Application to the Detection of Abnormal Behavior in Industrial Processes

This paper analyzes the expected time evolution of an auto-regressive (AR) process using self-organized maps (SOM). It investigates how a SOM captures the time information given by the AR input process and how the transitions from one neuron to another one can be understood under a probabilistic perspective. In particular, regions of the map into which the AR process is expected to move are identified. This characterization allows detecting anomalous changes in the AR process structure or parameters. On the basis of the theoretical results, an anomaly detection method is proposed and applied to a real industrial process.

Bayesian Visual Reranking

Visual reranking has been proven effective to refine text-based video and image search results. It utilizes visual information to recover “true” ranking list from the noisy one generated by text-based search, by incorporating both textual and visual information. In this paper, we model the textual and visual information from the probabilistic perspective and formulate visual reranking as an optimization problem in the Bayesian framework, termed Bayesian visual reranking. In this method, the textual information is modeled as a likelihood, to reflect the disagreement between reranked results and text-based search results which is called ranking distance. The visual information is modeled as a conditional prior, to indicate the ranking score consistency among visually similar samples which is called visual consistency. Bayesian visual reranking derives the best reranking results by maximizing visual consistency while minimizing ranking distance. To model the ranking distance more precisely, we propose a novel pair-wise method which measure the ranking distance based on the disagreement in terms of pair-wise orders. For visual consistency, we study three different regularizers to mine the best way for its modeling. We conduct extensive experiments on both video and image search datasets. Experimental results demonstrate the effectiveness of our proposed Bayesian visual reranking.

Serviceability limit state CPT-based design for vertically loaded shallow footings on sand

In this study, reliability-based design of vertically-loaded shallow foundations on cohesionless soil at the serviceability limit state is addressed and related to cone penetration resistance. A special database of 30 large footings on 12 different sands is investigated in a statistical and probabilistic perspective to quantify the load-displacement-capacity response and derive a direct design method based on cone penetrometer testing (CPT). A reference load-displacement model is identified through a comparative assessment of fitting of model types to load test data. The uncertainties in the load-displacement model and in the model inputs are modeled probabilistically. The probability of exceedance of a target probability of occurrence of a pre-established allowable settlement is computed in the form of a reliability index through Monte Carlo simulation. A partial load factor is defined and calibrated based on simulation outputs. Analytical expressions for the load factor as a function of the ratio of allowable settlement to footing dimensions are derived using least squares regression. The final design format allows simple and practical utilization on the part of the engineer while ensuring the minimization of excess conservatism.

A risk concept applicable for both probabilistic and non-probabilistic perspectives

In engineering risk assessments, probability is the common tool used to describe the epistemic uncertainties about unknown quantities. Probability is considered a main component of risk. However, a number of alternative approaches exist for representing and describing uncertainties in risk assessments, including possibility theory and evidence theory. For these approaches and theories a probability-based risk definition cannot in general serve as a conceptual framework for risk assessments. A broader risk perspective is required. The purpose of the present paper is to present such a perspective and show how both the probabilistic and the alternative approaches and theories can be supported by this perspective. The key feature of this perspective is that uncertainty replaces probability in the definition of risk.

Random walk and the heat equation

The heat equation can be derived by averaging over a very large number of particles. Traditionally, the resulting PDE is studied as a deterministic equation, an approach that has brought many significant results and a deep understanding of the equation and its solutions. By studying the heat equation by considering the individual random particles, however, one gains further intuition into the problem. While this is now standard for many researchers, this approach is generally not presented at the undergraduate level. In this book, Lawler introduces the heat equation and the closely related notion of harmonic functions from a probabilistic perspective. The theme of the first two chapters of the book is the relationship between random walks and the heat equation. The first chapter discusses the discrete case, random walk and the heat equation on the integer lattice; and the second chapter discusses the continuous case, Brownian motion and the usual heat equation. Relationships are shown between the two. For example, solving the heat equation in the discrete setting becomes a problem of diagonalization of symmetric matrices, which becomes a problem in Fourier series in the continuous case. Random walk and Brownian motion are introduced and developed from first principles. The latter two chapters discuss different topics: martingales and fractal dimension, with the chapters tied together by one example, a random Cantor set. The idea of this book is to merge probabilistic and deterministic approaches to heat flow. It is also intended as a bridge from undergraduate analysis to graduate and research perspectives. The book is suitable for advanced undergraduates, particularly those considering graduate work in mathematics or related areas.

Stochastic Reachability: From Markov Chains to Stochastic Hybrid Systems

AbstractStochastic hybrid systems represent established classes of realistic models of hybrid dynamics subject to random perturbations, autonomous uncontrollable transitions, nondeterminism or uncertainty. Stochastic reachability analysis is a key factor in the verification and deployment of stochastic hybrid systems. In this paper, we tackle the stochastic reachability problem from a purely probabilistic perspective. Using the connection between stochastic reachability and optimal stopping, the reach probabilities are estimated using only the probabilistic parameters (transition probabilities, infinitesimal generator) of stochastic hybrid systems. The technique is illustrated first for Markov chains, then for other more complex Markov models.

PIGS, PARTIES, AND NOISE: HOW STOCHASTICITY IMPACTS THE ROBUSTNESS OF THE TSEMBAGA CULTURAL PRACTICES

We present a probabilistic perspective on sustainable resource usage. A mathematical model is introduced to describe the interplay between a population and its renewable resource base. The amount of effort a society chooses to exert in harvesting its resource is formalized in the model. Using an indigenous population of slash and burn farmers as a case study, we derive a system of stochastic differential equations from a system of ordinary differential equations introduced by another author. The cultural mechanisms that help to stabilize the population in the deterministic system actually decrease the expected survival time in the stochastic system.

The Logical Problem of Language Acquisition: A Probabilistic Perspective

Natural language is full of patterns that appear to fit with general linguistic rules but are ungrammatical. There has been much debate over how children acquire these "linguistic restrictions," and whether innate language knowledge is needed. Recently, it has been shown that restrictions in language can be learned asymptotically via probabilistic inference using the minimum description length (MDL) principle. Here, we extend the MDL approach to give a simple and practical methodology for estimating how much linguistic data are required to learn a particular linguistic restriction. Our method provides a new research tool, allowing arguments about natural language learnability to be made explicit and quantified for the first time. We apply this method to a range of classic puzzles in language acquisition. We find some linguistic rules appear easily statistically learnable from language experience only, whereas others appear to require additional learning mechanisms (e.g., additional cues or innate constraints).

Cops and Robbers from a distance

Cops and Robbers is a pursuit and evasion game played on graphs that has received much attention. We consider an extension of Cops and Robbers, distance k Cops and Robbers, where the cops win if at least one of them is of distance at most k from the robber in G . The cop number of a graph G is the minimum number of cops needed to capture the robber in G . The distance k analogue of the cop number, written c k ( G ) , equals the minimum number of cops needed to win at a given distance k . We study the parameter c k from algorithmic, structural, and probabilistic perspectives. We supply a classification result for graphs with bounded c k ( G ) values and develop an O ( n 2 s + 3 ) algorithm for determining if c k ( G ) ≤ s for s fixed. We prove that if s is not fixed, then computing c k ( G ) is NP-hard. Upper and lower bounds are found for c k ( G ) in terms of the order of G . We prove that ( n k ) 1 / 2 + o ( 1 ) ≤ c k ( n ) = O ( n log ( 2 n k + 1 ) log ( k + 2 ) k + 1 ) , where c k ( n ) is the maximum of c k ( G ) over all n -vertex connected graphs. The parameter c k ( G ) is investigated asymptotically in random graphs G ( n , p ) for a wide range of p = p ( n ) . For each k ≥ 0 , it is shown that c k ( G ) as a function of the average degree d ( n ) = p n forms an intriguing zigzag shape.

Review of David Temperley,Music and Probability(MIT Press, 2007)

[1] In Music and Probability, David Temperley presents a meaningful analysis of the cognitive resources implied in music perception, providing a sound and coherent series of models based on a probabilistic perspective.[2] Cognitive sciences aim at understanding how our minds-and perhaps even artificial ones-are able to combine information about the external world with internal mental representations, in order to perform certain actions and achieve specific goals. Cognition gives us a more or less realistic, accurate perspective of the existence and behavior of the outside world. By processing this information we are able to plan our actions, solve problems, and obtain desired results (Von Eckardt 1993).[3] Originally, cognitive sciences employed a symbolic approach. According to this paradigm, minds are viewed as symbolic processors, and syntactic rules-rules that correlate with the form of information only, not its contents-were enough in principle to represent knowledge and the way humans think and solve problems (Gardner 1985). Since computers were particularly well suited to perform syntactic analysis, it seemed conceivable that one could turn them into cognitive agents (Turing 1950).[4] Due to its formal structure, music was one of the early targets in cognitive research. Several researchers and experimental musicians were convinced that music could be analyzed mathematically as some sort of code, and could therefore be artificially generated (Meyer 1967). Avant-garde artists dreamed of ensembles of humans and computers improvising in jam sessions. Those interested in cognitive processes and music believed we could understand those processes better by assimilating them into the way computers and humans processed syntax (Jackendoff 1983).[5] When this paradigm proved to be too limiting (Dreyfus 1992, Varela 1991), emphasis was directed towards statistical and probabilistic approaches, the same as in Music and Probability. The older models, however, were black box approaches to mathematical generation of music. There was no attempt to give any psychological reality to models, despite the fact that this has been one of the main aims of the cognitive sciences since their inception (Johnson-Laird 1983).[6] Much of this earlier research mirrored cognitive science's understanding of neural networks at the time (Rumelhart 1989). Neural networks were trained to recognize certain patterns and structures, and they did astonishing things that symbolic AI (Artificial Intelligence) could not. For example, they could recognize faces and speech, and they could play backgammon better than humans. Yet cognitive scientists could not explain how these networks worked. They found it very difficult to extract useful information, which would help them to better understand how humans perceive or generate music. An early appreciation of such difficulties within a probabilistic study of music can be found in Cohen (1962).[7] Early statistical and probabilistic approaches only showed that, when humans recognize a musical passage, some sort of statistical process occurred in their neurons. Although the process was thought to be equivalent in some sense to that of neural networks, it was impossible to develop an algorithm that could predict what the cognitive processes would be like. This is a general problem that most probabilistic and neural networks models share, as stated in Clark (1989).[8] Music and Probability is different from earlier probabilistic studies of music cognition. Temperley clearly understands mathematics, music, and cognitive sciences, and he successfully and convincingly combines them in his book. He leads the reader into a deeper interaction with cognitive processes. The reader learns how the brain uses the mathematical nature of music to perceive and create music.Music and Probability? Main Contents[9] This book serves as an introductory and systematic course on the probabilistic analysis of music, and on how to use that approach in music theory as well as in the cognitive sciences. …

The centrality of metrical structure in signaling information structure: A probabilistic perspective

This article introduces a new way to explain how information structure is signaled prosodically in English. I claim that METRICAL STRUCTURE plays a central role (Ladd 2008, Truckenbrodt 1995). Information structure (defined as in Steedman 1991 and Vallduví & Vilkuna 1998) places strong constraints on the PROBABIILISTIC mapping of words onto metrical prosodic structure—that is, foci usually align with nuclear accents and theme/rheme units with prosodic phrases, and themes are less metrically prominent than rhemes. It is shown that focus position, scope, and pragmatic interpretation are then derived by manipulating EXPECTED PROMINENCE within metrical structure. Broadly, the more prominent a word than expected, the more likely a contrastive reading; the less prominent, the more likely a givenness reading. Both constructed and naturally occurring examples from the Switchboard corpus are used.

Bayesian approach to model‐based signal processing: An overview.

Although available for a long time with the advent of high‐speed/high‐throughput computing, the development of Bayesian processing techniques has evolved recently in acoustical signal processing. Bayesian signal processing is concerned with the estimation of the underlying probability distribution of a random signal in order to perform statistical inferences such as the conditional mean estimation. Knowledge of this distribution provides all of the essential information available required for problem solution. The usual limitations of nonlinear approximations and non‐gaussian processes prevalent in classical algorithms (e.g., Kalman filters) are no longer a restriction to perform Bayesian inference. This approach enables the next generation of processors called particle filters that are sequential Monte Carlo methods providing an estimate of the underlying discrete probability distribution. In this overview, Bayesian signal processing is presented from a probabilistic perspective starting with Bayes rule and evolving to the development of a bootstrap particle filter perhaps one of the most common and simplest constructs available. The relationship of Bayesian processing to the concept of maximum entropy is discussed. Maximum entropy and its applicability in Bayesian processing is also mentioned briefly.

Kernel density estimation-based real-time prediction for respiratory motion

Effective delivery of adaptive radiotherapy requires locating the target with high precision in real time. System latency caused by data acquisition, streaming, processing and delivery control necessitates prediction. Prediction is particularly challenging for highly mobile targets such as thoracic and abdominal tumors undergoing respiration-induced motion. The complexity of the respiratory motion makes it difficult to build and justify explicit models. In this study, we honor the intrinsic uncertainties in respiratory motion and propose a statistical treatment of the prediction problem. Instead of asking for a deterministic covariate–response map and a unique estimate value for future target position, we aim to obtain a distribution of the future target position (response variable) conditioned on the observed historical sample values (covariate variable). The key idea is to estimate the joint probability distribution (pdf) of the covariate and response variables using an efficient kernel density estimation method. Then, the problem of identifying the distribution of the future target position reduces to identifying the section in the joint pdf based on the observed covariate. Subsequently, estimators are derived based on this estimated conditional distribution. This probabilistic perspective has some distinctive advantages over existing deterministic schemes: (1) it is compatible with potentially inconsistent training samples, i.e., when close covariate variables correspond to dramatically different response values; (2) it is not restricted by any prior structural assumption on the map between the covariate and the response; (3) the two-stage setup allows much freedom in choosing statistical estimates and provides a full nonparametric description of the uncertainty for the resulting estimate. We evaluated the prediction performance on ten patient RPM traces, using the root mean squared difference between the prediction and the observed value normalized by the standard deviation of the observed data as the error metric. Furthermore, we compared the proposed method with two benchmark methods: most recent sample and an adaptive linear filter. The kernel density estimation-based prediction results demonstrate universally significant improvement over the alternatives and are especially valuable for long lookahead time, when the alternative methods fail to produce useful predictions.

An atomic and probabilistic perspective on twin nucleation in Mg

We discuss the nucleation of deformation twins in Mg from a fundamental perspective. Atomistic simulations reveal twinning mechanisms and suggest that twin nucleation most likely occurs at grain boundaries (GBs). We observe twin nucleation from symmetrical tilt grain boundaries using molecular dynamics and reveal that the nucleation pathway depends on the tilt angle and the GB defect state. In particular, twin nucleation is preferred at GBs with low misorientation angles, in agreement with electron back-scattering diffraction (EBSD) analyses. A probabilistic description of twin nucleation is then proposed with the aim of linking atomic-scale information with meso-scale EBSD statistical analyses.

A Unifying Probabilistic Perspective for Spectral Dimensionality Reduction

We introduce a new perspective on spectral dimensionality reduction which views these methods as Gaussian Markov random fields (GRFs). Our unifying perspective is based on the maximum entropy princ...

Multimodel Ensemble ENSO Prediction with CCSM and CFS

Abstract Results are described from a large sample of coupled ocean–atmosphere retrospective forecasts during 1982–98. The prediction system is based on the National Center for Atmospheric Research (NCAR) Community Climate System Model, version 3 (CCSM3.0), and a state-of-the-art ocean data assimilation system made available by the National Oceanic and Atmospheric Administration (NOAA) Geophysical Fluid Dynamics Laboratory (GFDL). The retrospective forecasts are initialized in January, April, July, and November of each year, and ensembles of 6 forecasts are run for each initial month, yielding a total of 408 1-yr predictions. In generating the ensemble members, perturbations are added to the atmospheric initial state only. The skill of the prediction system is analyzed from both a deterministic and a probabilistic perspective, it is then compared to the operational NOAA Climate Forecast System (CFS), and the forecasts are combined with CFS to produce a multimodel prediction system. While the skill scores for each model are highly dependent on lead time and initialization month, the overall level of skill of the individual models is quite comparable. The multimodel combination (i.e., the unweighted average of the forecast), while not always the most skillful, is generally as skillful as the best model, using either deterministic or probabilistic skill metrics.