General Probabilistic Framework Research Articles

Regular variation in Hilbert spaces and principal component analysis for functional extremes

Motivated by the increasing availability of data of functional nature, we develop a general probabilistic and statistical framework for extremes of regularly varying random elements X in L2[0,1]. We place ourselves in a Peaks-Over-Threshold framework where a functional extreme is defined as an observation X whose L2-norm ‖X‖ is comparatively large. Our goal is to propose a dimension reduction framework resulting into finite dimensional projections for such extreme observations. Our contribution is double. First, we investigate the notion of Regular Variation for random quantities valued in a general separable Hilbert space, for which we propose a novel concrete characterization involving solely stochastic convergence of real-valued random variables. Second, we propose a notion of functional Principal Component Analysis (PCA) accounting for the principal ‘directions’ of functional extremes. We investigate the statistical properties of the empirical covariance operator of the angular component of extreme functions, by upper-bounding the Hilbert–Schmidt norm of the estimation error for finite sample sizes. Numerical experiments with simulated and real data illustrate this work.

Learning fair models without sensitive attributes: A generative approach

Most existing fair classifiers rely on sensitive attributes to achieve fairness. However, for many scenarios, we cannot obtain sensitive attributes due to privacy and legal issues. The lack of sensitive attributes challenges many existing fair classifiers. Though we lack sensitive attributes, for many applications, there usually exists features/information of various formats that are relevant to sensitive attributes. For example, a person’s purchase history can reflect his/her race, which would help for learning fair classifiers on race. However, the work on exploring relevant features for learning fair models without sensitive attributes is rather limited. Therefore, in this paper, we study a novel problem of learning fair models without sensitive attributes by exploring relevant features. We propose a probabilistic generative framework to effectively estimate the sensitive attribute from the training data with relevant features in various formats and utilize the estimated sensitive attribute information to learn fair models. Experimental results on real-world datasets show the effectiveness of our framework in terms of both accuracy and fairness. Our source code is available at: https://github.com/huaishengzhu/FairWS.

A LCCA framework for the management of bridges based on data fusion from visual inspections and SHM systems.

AbstractBridges' integrity, safety and serviceability are fundamental targets that need to be ensured during the structural lifetime, owing to the crucial role that such infrastructure components play in modern communities. Recently, a particular attention is being paid to the management of bridges since many of them have reached the end of their expected service life. To this purpose, the Italian Ministry of Sustainable Mobility issued in 2020 the “Guidelines for risk classification and management, safety assessment and monitoring of bridges” that regulate visual inspections, risk assessment based on multilevel protocols and consequent monitoring and maintenance activities. In this context, structural health monitoring (SHM) systems play a fundamental role to guarantee safety and adequate performance levels. However, the synergy between visual inspections and SHM is yet to be fully understood and exploited. The main goal of this research work is to contribute to filling this gap by proposing a general probabilistic framework, based on Life‐Cycle Cost Analysis (LCCA), able to fuse information from SHM data and visual inspections, with the aim to plan over time short‐term and long‐term interventions and to optimize fund allocation by considering all the stored information.

PyBEAM: A Bayesian approach to parameter inference for a wide class of binary evidence accumulation models.

Many decision-making theories are encoded in a class of processes known as evidence accumulation models (EAM). These assume that noisy evidence stochastically accumulates until a set threshold is reached, triggering a decision. One of the most successful and widely used of this class is the Diffusion Decision Model (DDM). The DDM however is limited in scope and does not account for processes such as evidence leakage, changes of evidence, or time varying caution. More complex EAMs can encode a wider array of hypotheses, but are currently limited by computational challenges. In this work, we develop the Python package PyBEAM (Bayesian Evidence Accumulation Models) to fill this gap. Toward this end, we develop a general probabilistic framework for predicting the choice and response time distributions for a general class of binary decision models. In addition, we have heavily computationally optimized this modeling process and integrated it with PyMC, a widely used Python package for Bayesian parameter estimation. This 1) substantially expands the class of EAM models to which Bayesian methods can be applied, 2) reduces the computational time to do so, and 3) lowers the entry fee for working with these models. Here we demonstrate the concepts behind this methodology, its application to parameter recovery for a variety of models, and apply it to a recently published data set to demonstrate its practical use.

X-Metric: An N-Dimensional Information-Theoretic Framework for Groupwise Registration and Deep Combined Computing.

This paper presents a generic probabilistic framework for estimating the statistical dependency and finding the anatomical correspondences among an arbitrary number of medical images. The method builds on a novel formulation of the N-dimensional joint intensity distribution by representing the common anatomy as latent variables and estimating the appearance model with nonparametric estimators. Through connection to maximum likelihood and the expectation-maximization algorithm, an information-theoretic metric called X-metric and a co-registration algorithm named X-CoReg are induced, allowing groupwise registration of the N observed images with computational complexity of O(N). Moreover, the method naturally extends for a weakly-supervised scenario where anatomical labels of certain images are provided. This leads to a combined-computing framework implemented with deep learning, which performs registration and segmentation simultaneously and collaboratively in an end-to-end fashion. Extensive experiments were conducted to demonstrate the versatility and applicability of our model, including multimodal groupwise registration, motion correction for dynamic contrast enhanced magnetic resonance images, and deep combined computing for multimodal medical images. Results show the superiority of our method in various applications in terms of both accuracy and efficiency, highlighting the advantage of the proposed representation of the imaging process.

Learning Probabilistic Piecewise Rigid Atlases of Model Organisms via Generative Deep Networks.

Atlases are crucial to imaging statistics as they enable the standardization of inter-subject and inter-population analyses. While existing atlas estimation methods based on fluid/elastic/diffusion registration yield high-quality results for the human brain, these deformation models do not extend to a variety of other challenging areas of neuroscience such as the anatomy of C. elegans worms and fruit flies. To this end, this work presents a general probabilistic deep network-based framework for atlas estimation and registration which can flexibly incorporate various deformation models and levels of keypoint supervision that can be applied to a wide class of model organisms. Of particular relevance, it also develops a deformable piecewise rigid atlas model which is regularized to preserve inter-observation distances between neighbors. These modeling considerations are shown to improve atlas construction and key-point alignment across a diversity of datasets with small sample sizes including neuron positions in C. elegans hermaphrodites, fluorescence microscopy of male C. elegans, and images of fruit fly wings. Code is accessible at https://github.com/amin-nejat/Deformable-Atlas.

GFlowNets for AI-driven scientific discovery

GFlowNets provide a general probabilistic framework for accelerating the computational phase of the scientific discovery process, which is crucial for tackling pressing challenges posed by global pandemics and the climate crisis.

A probabilistic model of diffusion through a semi-permeable barrier

Diffusion through semi-permeable structures arises in a wide range of processes in the physical and life sciences. Examples at the microscopic level range from artificial membranes for reverse osmosis to lipid bilayers regulating molecular transport in biological cells to chemical and electrical gap junctions. There are also macroscopic analogues such as animal migration in heterogeneous landscapes. It has recently been shown that one-dimensional diffusion through a barrier with constant permeability κ 0 is equivalent to snapping out Brownian motion (BM). The latter sews together successive rounds of partially reflecting BMs that are restricted to either the left or the right of the barrier. Each round is killed when its Brownian local time exceeds an exponential random variable parameterized by κ 0 . A new round is then immediately started in either direction with equal probability. In this article, we use a combination of renewal theory, Laplace transforms and Green’s function methods to show how an extended version of snapping out BM provides a general probabilistic framework for modelling diffusion through a semi-permeable barrier. This includes modifications of the diffusion process away from the barrier (e.g. stochastic resetting) and non-Markovian models of membrane absorption that kill each round of partially reflected BM. The latter leads to time-dependent permeabilities.

The microstructure of stochastic volatility models with self-exciting jump dynamics

We provide a general probabilistic framework within which we establish scaling limits for a class of continuous-time stochastic volatility models with self-exciting jump dynamics. In the scaling limit, the joint dynamics of asset returns and volatility is driven by independent Gaussian white noises and two independent Poisson random measures that capture the arrival of exogenous shocks and the arrival of self-excited shocks, respectively. Various well-studied stochastic volatility models with and without self-exciting price/volatility co-jumps are obtained as special cases under different scaling regimes. We analyze the impact of external shocks on the market dynamics, especially their impact on jump cascades and show in a mathematically rigorous manner that many small external shocks may trigger endogenous jump cascades in asset returns and stock price volatility.

Full Bayesian inference in hidden Markov models of plant growth

Accurately modeling the growth process of plants in interaction with their environment is important for predicting their biophysical characteristics, referred to as phenotype prediction. Most models are described by discrete dynamic systems in general state-space representation with important domain-specific characteristics: First, plant model parameters have usually clear functional meanings and may be of genetic origins, thus necessitating a precise estimation. Second, critical growth variables, specifically biomass production and dynamic allocation to organs, are hidden variables not accessible to measure. Finally, the difficulty to assess the local plant environment may imply the introduction of process noises in models. Therefore, a precise understanding of the system’s behavior requires the joint estimation of functional parameters, hidden states, and noise parameters. In this paper we describe how a full Bayesian method of estimation can accurately estimate all these key model variables using Markov chain Monte Carlo (MCMC) techniques. In the presence of both process and observation noises, it requires to use adequate particle MCMC (PMCMC) algorithms to efficiently sample the hidden states which, consequently, allows for a precise estimation of all noise parameters involved. Thanks to the Bayesian framework, appropriate choices of prior distributions for the noise parameters have enabled analytical posterior distributions and only simple updates are required. Furthermore, this estimation strategy can be easily generalized and adapted to different types of plant growth models, such as organ-scale or compartmental, provided that they are formulated as hidden Markov models. Our estimation method improves on those classically used in plant growth modeling in several aspects: First, by building upon a general probabilistic framework the estimation results allow proper statistical analyses. It is useful in prediction, no only for uncertainty and risk analysis (e.g., for crop yield prediction) but also to analyze the results of experimental trials, for example, to compare genotypes in breeding. Moreover, the care taken in the estimation of hidden variables opens new perspectives in the understanding of inner growth processes, notably the balance and interaction between biomass production and allocation (referred to as source-sink dynamics). Applications of this estimation procedure are demonstrated on the GreenLab model for Arabidopsis thaliana and the Log-Normal Allocation and Senescence (LNAS) model for sugar beet, on both synthetic and real data.

Constrained Probabilistic Movement Primitives for Robot Trajectory Adaptation

Placing robots outside controlled conditions requires versatile movement representations that allow robots to learn new tasks and adapt them to environmental changes. The introduction of obstacles or the placement of additional robots in the workspace, the modification of the joint range due to faults or range-of-motion constraints are typical cases where the adaptation capabilities play a key role for safely performing the robot's task. Probabilistic movement primitives (ProMPs) have been proposed for representing adaptable movement skills, which are modelled as Gaussian distributions over trajectories. These are analytically tractable and can be learned from a small number of demonstrations. However, both the original ProMP formulation and the subsequent approaches only provide solutions to specific movement adaptation problems, e.g., obstacle avoidance, and a generic, unifying, probabilistic approach to adaptation is missing. In this paper we develop a generic probabilistic framework for adapting ProMPs. We unify previous adaptation techniques, for example, various types of obstacle avoidance, via-points, mutual avoidance, in one single framework and combine them to solve complex robotic problems. Additionally, we derive novel adaptation techniques such as temporally unbound via-points and mutual avoidance. We formulate adaptation as a constrained optimisation problem where we minimise the Kullback-Leibler divergence between the adapted distribution and the distribution of the original primitive while we constrain the probability mass associated with undesired trajectories to be low. We demonstrate our approach on several adaptation problems on simulated planar robot arms and 7-DOF Franka-Emika robots in a dual robot arm setting.

Towards resilience of offshore wind farms: A framework and application to asset integrity management

This paper develops a general probabilistic framework for resilience modeling and analysis of offshore wind farm (OWF), and illustrates how such a framework may be implemented within the modeling techniques and tools commonly applied in the industry. Based on this framework the significance of prevailing uncertainties and the effects of different decision alternatives relevant in the context of asset integrity management (AIM) are studied and discussed. In the framework, OWFs are modeled as system-of-systems by a hierarchical model where the life-cycle performances of each system, as well as the dependencies between these systems, are represented probabilistically. The quantification of resilience is undertaken based on a scenario-based modeling of life cycle benefits and costs in which resilience failure is defined as the exhaustion of the economic capacity accumulated by the system over time. Moreover, this paper introduces resilience-informed decision-making for OWF in the context of AIM. The proposed framework is applied to the OWFs populated with NREL 5MW offshore wind turbines (OWTs). Events of typhoon-induced waves and winds are considered as the two random environmental load processes affecting the OWF’s dynamic responses and for which their resilience performances are carried out. Finally, the resilience performances of the OWFs are studied and discussed for a range of decision alternatives relevant to AIM.

Semi-supervised Latent Block Model with pairwise constraints

Co-clustering aims at simultaneously partitioning both dimensions of a data matrix. It has demonstrated better performances than one-sided clustering for high-dimensional data. The Latent Block Model (LBM) is a probabilistic model for co-clustering based on mixture models that has proven useful for a broad class of data. In this paper, we propose to leverage prior knowledge in the form of pairwise semi-supervision in both row and column space to improve the clustering performances of the algorithms derived from this model. We present a general probabilistic framework for incorporating must link and cannot link relationships in the LBM based on Hidden Markov Random Fields. We instantiate this framework on a model for count data and present two inference algorithms based on Variational and Classification EM. Extensive experiments on simulated data and on real-world attributed networks confirm the interest of our approach and demonstrate the effectiveness of our algorithms.

Modeling Diffusion in Social Networks Using Network Properties

Diffusion of items occurs in social networks due to spreading of items through word of mouth and exogenous factors. These items may be news, products, videos, advertisements or contagious viruses. Previous research has studied diffusion process at both the macro and micro levels. The former models the number of item adopters in the diffusion process while the latter determines which individuals adopt item. In this paper, we establish a general probabilistic framework, which can be used to derive macro-level diffusion models, including the well known Bass Model (BM). Using this framework, we develop several other models considering the social network’s degree distribution coupled with the assumption of linear influence by neighboring adopters in the diffusion process. Through some evaluation on synthetic data, this paper shows that degree distribution actually changes during the diffusion process. We therefore introduce a multi-stage diffusion model to cope with variable degree distribution. By conducting experiments on both synthetic and real datasets, we show that our proposed diffusion models can recover the diffusion parameters from the observed diffusion data, which allows us to model diffusion with high accuracy.

Group recommendation with noisy subjective preferences

Social choice theory provides a principled framework for the aggregation of individuals' preferences in support of group decision‐making and recommendation. Much of this work, however, either assumes that individuals' subjective preferences (and thus, their votes) are correctly specified by the individuals themselves, or alternatively that the votes of individuals are noisy estimates of some underlying ground truth over rankings of alternatives. We argue that neither model appropriately addresses some of the issues which arise in the context of group‐recommendation domains where individuals have subjective preferences but for some reason (eg, the high cognitive burden, concerns about privacy, etc.) may instead vote using a noisy estimate of their subjective preference rankings. In this paper, we propose a general probabilistic framework for modeling noisy subjective preferences, and explore the accuracy and reliability of four well‐studied voting rules under various noise models. Our results demonstrate that there is no single reliable method amongst the examined methods. Specifically, we observe the change in noise distribution can flip one method from being the most reliable to the least.

RAFnet: Recurrent attention fusion network of hyperspectral and multispectral images

Hyperspectral imaging can facilitate a better understanding of more knowledge under real scenes, compared with traditional image systems. However, due to hardware limitations, only low resolution hyperspectral (LrHs) and high resolution multispectral (HrMs) images can generally be acquired. This paper proposed a recurrent attention fusion network (RAFnet) under a variational probabilistic generative framework, in order to fuse the LrHs and HrMs images together to generate a high resolution hyperspectral (HrHs) image in an unsupervised manner. In specific, two variational autoencoders are designed to preserve both spectral and spatial information of LrHs and HrMs images, coupled through a shared decoder to generate hyperspectral images. Considering the spectra of each hyperspectral pixel is intrinsically a sequence based data structure, we construct a hierarchical recurrent neural network to extract the abundant spectral information. Moreover, self-attention and relation-attention mechanisms are adopted to capture long temporal dependencies through the spectral domain. The effectiveness and efficiency are evaluated based on several publicly available hyperspectral datasets, compared with many state-of-the-art methods for the unsupervised fusion task.

Modelling correlated damage of spatially distributed building portfolios under scenario tropical cyclones

Probabilistic damage assessment of a community under a scenario tropical cyclone needs to consider the collective damage of individual structures within the community. This involves modelling the spatial correlations of wind speeds and structural vulnerability between individual structures. Ignoring the spatial correlations can lead to unconservative assessment. However, the methods to model these two spatial correlations and their effects on damage assessment are limited. In this paper, a general probabilistic framework is proposed to analyze the correlated damage of spatially distributed building portfolios using joint fragility functions. Two cases are investigated: (1) the correlation of fragility functions of different buildings is given, and (2) the correlation of structural resistance is given from which the correlation of fragilities is computed. To demonstrate the framework, an example is provided to analyze the roof sheathing damage of a residential building portfolio under a cyclone event.

Task-based support in search engines

Web search has become a key technology on which people rely daily for getting information about almost everything. The evolution of the search experience has also shaped the expectations of people about it. Many users seem to expect today's web search engines to behave like a kind of "wise interpreter," capable of understanding the meaning behind a search query, realizing its current context, and responding to it directly and appropriately. Search by meaning, or semantic search, encompasses a large portion of information retrieval (IR) research devoted to study more meaningful representations of the information need expressed by the user query. Entity cards, direct displays, and verticals are examples of how major commercial search engines have indeed responded to user expectations, capitalizing on query understanding. Search is usually performed with a specific goal underlying the query. In many cases, this goal consists of a nontrivial task to be completed. Current search engines support a small set of basic tasks, and most of the knowledge-intensive workload for supporting more complex tasks is left to the user. Task-based search can be viewed as an information access paradigm that aims to enhance search engines with functionalities for recognizing the underlying tasks in searches and providing support for task completion. The research presented in this thesis focuses on utilizing and extending methods and techniques from semantic search in the next stage of the evolution of search engines, namely, to support users in achieving their tasks. Our work can be grouped in three grand themes: (1) Entity type information for entity retrieval : we conduct a systematic evaluation and analysis of methods for type-aware entity retrieval, in terms of three main dimensions. Also, we revisit the problem of hierarchical target type identification, present a state-of-the-art supervised learning method, and analyze the usage of automatically identified target entity types for type-aware entity retrieval; (2) Entity-oriented search intents : we propose a categorization scheme for entity-oriented search intents, and study the distributions of entity intent categories per entity type. We further develop a method for constructing a knowledge base of entity-oriented search intents; and (3) Task-based search : we design a probabilistic generative framework for task-based query suggestion, and principledly estimate each of its components. Furthermore, we introduce the problems of query-based task recommendation and mission-based task recommendation, and establish respective methods as suitable baselines.

Latency Analysis for Sequential Detection in Low-Complexity Binary Radio Systems

We consider the problem of making a quick decision in favor of one of two possible physical signal models while the numerical measurements are acquired by sensing devices featuring minimal digitization complexity. Therefore, the digital data streams available for statistical processing are binary and exhibit temporal and spatial dependencies. To handle the intractable multivariate binary data model, we first consider sequential tests for exponential family distributions. Within this generic probabilistic framework, we identify adaptive approximations for the log-likelihood ratio and the Kullback-Leibler divergence. The results allow designing sequential detectors for binary radio systems and analyzing their average run-time along classical arguments of Wald. In particular, the derived tests exploit the spatio-temporal correlation structure of the analog sensor signals engraved into the binary measurements. As an application, we consider the specification of binary sensing architectures for cognitive radio and GNSS spectrum monitoring where our results characterize the sequential detection latency as a function of the temporal oversampling and the number of antennas. Finally, we evaluate the efficiency of the proposed algorithms and illustrate the accuracy of our analysis via Monte-Carlo simulations.

A probabilistic framework to incorporate mixed-data type features: Matrix factorization with multimodal side information

Recommender systems that exclusively rely on past interactions between the users and the items underperform in settings with very few observations. Their performance at such extreme sparsity can be improved by exploiting the side information of the users and the items including the demographics and the item descriptions. However, such side information is mostly heterogeneous and multimodal, including both numerical and categorical features to yield a non-trivial incorporation process. Researchers have addressed this problem mainly by converting the categorical features into numerical ones or forming numerical similarity matrices. This paper presents a different approach in the form of a novel Bayesian probabilistic generative framework which can effectively incorporate multimodal side information into the matrix factorization (MF-MSI) model. With the help of local quadratic bounds on the categorical likelihoods, we derive a scalable and computationally efficient iterative optimization method based on the variational EM to learn the posterior distributions of latent variables for both the users and the items. A comprehensive experimental study on both simulated and real benchmark datasets demonstrate the proof-of-concept where the additional side information improves both the prediction accuracy and the ranking performance over more than a dozen popular baseline models. Finally, the proposed MF-MSI model claims state-of-the-art performance in the majority of the test scenarios when compared to more recently introduced recommender systems which can also utilize the side information via different techniques.