Multimodal Probability Distributions Research Articles

Bayesian Ensemble Kalman Filter for Gaussian Mixture Models

AbstractInverse theory and data assimilation methods are commonly used in earth and environmental science studies to predict unknown variables, such as the physical properties of underground rocks, from a set of measured geophysical data, like geophysical seismic or electromagnetic data. A new Bayesian approach based on the ensemble Kalman filter using Gaussian mixture models is presented to overcome the assumption of Gaussian distribution of the unknown variables commonly used in the data assimilation literature and to generalize the algorithm to inverse problems with multimodal probability distributions. In applications of subsurface characterization, the multimodality of the unknown variables is generally due to the presence of different rock types, also known as geological facies. In the proposed method, the weights of the Gaussian mixture model represent the facies proportions, and they follow a Markov chain model. The proposed Bayesian model generates the unknown model parameters conditioned on measured data using a Markov chain Monte Carlo sampler. The validity of the method is demonstrated on a data assimilation problem where the goal is to estimate the posterior distribution of the unknown rock density from a set of repeated measurements of acoustic wave velocity measured at different times. The proposed method provides accurate estimates with efficient computational times.

Multi-Modal Pose Representations for 6-DOF Object Tracking

Pose estimation methods for robotics should return a distribution of poses rather than just a single pose estimate. Motivated by this, in this work we investigate multi-modal pose representations for reliable 6-DoF object tracking. A neural network architecture for simultaneous object segmentation and estimation of fiducial points of the object on RGB images is proposed. Given a priori probability distribution of object poses a particle filter is employed to estimate the posterior probability distribution of object poses. An advanced observation model relying on matching the projected 3D model with the segmented object and a distance transform-based object representation is used to weight samples representing the probability distribution. Afterwards, the object pose determined by the PnP algorithm is included in the probability distribution via replacing a particle with the smallest weight. Next, a k-means++ algorithm is executed to determine modes in a multi-modal probability distribution. A multi-swarm particle swarm optimization is then executed to determine the finest modes in the probability distribution. A subset of particles for final pose optimization is found in a multi-criteria analysis using the TOPSIS algorithm. They are verified using conflicting criteria that are determined on the basis of object keypoints, segmented object, and the distance transform. On the challenging YCB-Video dataset it outperforms recent algorithms for both object pose estimation and object pose tracking.

Adaptive filtering under multi-peak noise

In this paper, we assume the multi-peak noise in adaptive filtering follows a multimodal probability distribution (MPD), and model it as a Gaussian mixture model (GMM). Then a new gradient ascent algorithm is proposed based on the maximum likelihood function. Under the MPD assumption, multiple means and variances of the modes can be estimated by the expectation-maximization (EM) algorithm, and a corresponding closed-form solution can be obtained. Since the GMM can exactly approximate multi-peak probability density function (PDF), the closed-form solution of the proposed MPD algorithm can approximate the optimal solution of the adaptive filtering problem under multi-peak additive noise. According to the interaction between the distance and magnitudes of adjacent peaks, multi-peak noise can be divided into significant multi-peak (SMP) noise, hidden multi-peak (HMP) noise (e.g., Rayleigh), and invisible multi-peak (IMP) noise (e.g., Uniform). Simulations demonstrate that the proposed MPD algorithm outperforms the maximum correntropy criterion (MCC) and minimum error entropy (MEE) under SMP and HMP noise, and outperforms the least mean fourth (LMF) under IMP noise. The experimental steady-state errors align well with the theoretical values.

Probabilistic multi-modal expected trajectory prediction based on LSTM for autonomous driving

Autonomous vehicles (AVs) need to adequately predict the trajectory space of surrounding vehicles (SVs) in order to make reasonable decision-making and improve driving safety. In this paper, we build the driving behavior intention recognition module and traffic vehicle expected trajectory prediction module by deep learning. On the one hand, the driving behavior intention recognition module identifies the probabilities of lane keeping, left lane changing, right lane changing, left acceleration lane changing, and right acceleration lane changing of the predicted vehicle. On the other hand, the expected trajectory prediction module adopts an encoder-decoder architecture, in which the encoder encodes the historical environment information of the surrounding agents as a context vector, and the decoder and MDN network combine the context vector and the identified driving behavior intention to predict the probability distribution of future trajectories. Additionally, our model produces the multiple behaviors and trajectories that may occur in the next 6 s for the predicted vehicle (PV). The proposed model is trained, validated and tested with the HighD dataset. The experimental results show that the constructed probabilistic multi-modal expected trajectory prediction possesses high accuracy in the intention recognition module with full consideration of interactive information. At the same time, the multi-modal probability distribution generated by the anticipated trajectory prediction model is more consistent with the real trajectories, which significantly improves the trajectory prediction accuracy compared with other approaches and has apparent advantages in predicting long-term domain trajectories.

POMDP Motion Planning Algorithm Based on Multi-Modal Driving Intention

On highways, the interaction with surrounding vehicles is very crucial for the decision-making and planning of autonomous vehicles. However, the multi-modal driving intentions of surrounding vehicles have brought great challenges. Aiming at the multi-modal driving intention of surrounding vehicles, a multi-modal driving risk field based on dynamic collision region is proposed, and MDI-POMDP decision framework is established, which integrating behavior decision and motion planning. Firstly, the multi-modal probability distribution of driving intention is fused to establish a driving risk field. Moreover, combined with the longitudinal safety distance model and lateral driving direction, the concept of dynamic collision area is proposed in the driving risk field. Then, MDI-POMDP is formulated to analyze the influence of the uncertainty on planning, which is caused by the multi-modal driving intention of surrounding vehicles. In the following, with the help of the previous state, a time-dependent DRL algorithm RDPG is designed to enhance the current observation, to solve the optimal driving policy under partial observation and generate the optimal trajectory. Furthermore, the simulation results show that the performance of our proposed motion planning algorithm is outstanding, compared with the states-of-the-art methods. And our algorithm has the powerful ability to model the multi-modality of driving intention, to ensure the traffic safety.

Reliability analysis based on active learning Kriging model for problems involving multimodally distributed random variables

A novel strategy, integrating the λ Mixture Model (λMM) and the Active Learning Kriging model (ALK), is proposed to cope with the problems involving multimodally distributed random variables. First, the λMMs are applied to represent the multimodal probability distributions of random variables, because of their excellent multimodal property. Next, the Rejection Sampling (RS) is adopted to draw the candidate points for the proposed AK-RS, which avoids calculating the inverse Cumulative Distribution Function (CDF) of complex λMM and ensures the independence between samples. Following, an advanced Errorbased Stopping Criterion (ESC) is introduced, to further improve the efficiency of the proposed approach. The study of three cases shows that, the AK-RS can superbly fit the Probability Density Functions (PDFs) of observations, while this method can dramatically enhance the computational efficiency and ensure the achievement of the target accuracy.

Multimodal probabilistic modeling of melt pool geometry variations in additive manufacturing

Laser Powder Bed Fusion (LPBF) is commonly used to fabricate metal additive manufacturing (AM) parts. Advanced sensing empowers the collection of in-situ signals (e.g., melt pools) for real-time monitoring of AM processes. The variations of melt pool geometry are closely pertinent to mechanical and functional properties of final AM builds. Most of previous works are concerned about the extraction of melt pool features (e.g., geometric statistics, size, shape descriptors) for AM process monitoring. However, very little has been done to investigate the emission physics of photons that leads to the generation of melt pool images. To fill the gap, this work presents a novel approach to simulate the emission of photons for statistical estimation and modeling of the multimodal probability distribution function (PDF) of a melt pool. Specifically, the complex geometry of a melt pool is represented as a multimodal PDF. Photons emitted to generate melt pool images are treated as independent and identically distributed (iid) random samples drawn from the multimodal PDF. Further, we propose a new Gaussian mixture model to represent and estimate the multimodal PDF via the expectation–maximization procedures. In addition, we investigate how process conditions influence the geometric variations of melt pools. Experimental results show that the proposed methodology effectively builds the multimodal distribution model of melt pool geometric variations. Most importantly, the proposed methodology is flexible enough to take raw images as the input and shows great potentials to be generally applicable for different types of melt pool images and AM processes.

Fault permeability from stochastic modeling of clay smears

AbstractIn normally consolidated, shallow (depth <~3 km) siliciclastic sequences, faults develop clay smears. Existing models include the dependence of permeability on the clay fraction, but improved predictions of fault permeability should account for uncertainty and anisotropy. We introduce PREDICT, a methodology that computes probability distributions for the directional components (dip-normal, strike-parallel, and dip-parallel) of the fault permeability tensor from statistical samples for a set of geological variables. These variables, which include geometrical, compositional, and mechanical properties, allow multiple discretizations of the fault core to be populated with sand and clay smears, which can be used to upscale the permeability to a coarser scale (e.g., suitable for reservoir modeling). We validated our implementation with experimental data and applied PREDICT to several stratigraphic sequences. We show that fault permeability is controlled by the clay smear configuration and, crucially, that it typically exhibits multimodal probability distributions due to the existence of holes. The latter is a unique feature of our algorithm, which can be used to build fault permeability scenarios to manage and mitigate risk in subsurface applications.

Method for spectrum holes detection based on mode analysis of spectral samples histogram

The subject of this article is the process of detection and estimation frequency boundaries of spectrum holes under conditions of high spectrum occupancy when using receivers with narrow instantaneous bandwidths. The work increases the probability of spectrum holes correct detection in conditions of high occupancy of the radio frequency spectrum and variable noise levels by developing a method to distinguish signal and noise samples based on the analysis of the histogram of spectral sample modes. The tasks to be solved are: development of a method for separation signal and noise samples in the frequency domain; development of a methodology to find the minimum mode of a multimodal probability distribution; determination of frequency boundaries of spectrum holes; formulation of recommendations for the practical implementation of developed method. The methods used are: methods of probability theory and mathematical statistics, methods of statistical modeling. The essence of the proposed method is to distinguish the set of energy spectrum samples using a threshold, obtained for the value of the histogram mode, which corresponds to noise, and to determine the frequency boundaries of spectrum holes. The following results were obtained: an expression for calculating the threshold value for separation signal and noise samples in the frequency domain using the value, which corresponds to the noise mode of the frequency samples of the probability density function. It was found that the noise mode has the smallest value among other modes, since noise samples have a smaller value compared to the signal ones. A technique for estimating the value of the noise mode has been developed, which consists of a histogram of energy spectrum frequency samples and finding the partition interval that corresponds to the value of the minimal mode. An approach was proposed to determine the frequency boundaries of noise samples in the presence of one signal in the analyzed band. Conclusions. The developed method allows detecting spectrum holes with a probability of at least 0.9 at signal-to-noise ratio values of at least 5 dB for the spectrum with a rectangular shape envelope and 12 dB for other envelopes under occupancy of up to 80%.

Atlas of surface currents in the Mediterranean and Canary–Iberian–Biscay waters

ABSTRACT Sea surface currents probably are the most relevant essential ocean variable affecting diverse societal challenges concerning the marine environmental (as, for example, safe and efficient navigation, marine pollution and ecological connectivity). This work introduces a climatological Atlas (monthly resolution) of currents in the Mediterranean and Canary–Iberian–Biscay basins, based on today's state of the art reanalyses of the ocean circulation. The focus is on surface and subsurface reanalyses (here understood as and m, respectively) provided by the Copernicus Marine Environment Monitoring Service (CMEMS). The climatological values are computed from the median of the empirical probability density functions and the Atlas also includes the variance matrix and a bimodality index to have quantitative information on their variability. For both domains, the subsurface climatological fields are reasonably consistent with circulation schemes proposed in the previous literature but clearly improving the time and space resolution of the emerging patterns. For the Canary–Iberian–Biscay domain, the monthly climatological surface currents capture accurately the characteristic seasonal signal and its transition between a favourable and non-favourable upwelling regime. In the Mediterranean basin, differences between the near-surface and the 15 m velocity fields suggest a non-negligible role of winds over the variability of the uppermost ocean layer, specially in the Eastern Mediterranean basin. This is, up to our knowledge, the first time that such near-surface climatological patterns are computed. It has been found that, in general, the resulting patterns agree with surface drifter trajectories. In several regions, interannual variability foster bimodal and multimodal probability distributions. The Atlas has been conceived with the purpose of providing a first quantitative assessment on the surface circulation, thus being a complementary tool of real-time ocean forecasting systems. The Atlas is distributed following the FAIR principles and is accompanied with a digital version, with enhanced visualization capabilities for both research and assessment.

Defense of Massive False Data Injection Attack via Sparse Attack Points Considering Uncertain Topological Changes

False data injection attack (FDIA) is a typical cyber-attack aiming at falsifying measurement data for state estimation (SE), which may incur catastrophic consequences on cyber-physical system operation. In this paper, we develop a deep learning based methodology for detection, localization, and data recovery of FDIA on power systems in a coherent and holistic manner. However, the multi-modal probability distributions of both measurements and state variables in SE due to ever-changing operating points and structural/topological changes pose great challenges in detecting and localizing FDIA. To address this challenge, we first propose an enhanced attack model to launch massive FDIA on limited access points. Second, we train an auto-encoder (AE) with a Bayesian change verification (BCV) classifier using <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$N-1$</tex> contingencies to detect FDIA with unseen <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$N-k$</tex> operational topologies. Third, to avoid model collapse caused by multi-modal measurement distribution, an AE-based generative adversarial network (GAN) is derived to generate a diverse candidate set of normal measurement vectors with various operational topologies. Finally, we develop a pattern match algorithm to localize and recover the falsified measurements and state variables by comparing the falsified measurement vectors with the normal measurement vectors in the candidate set. Case studies with IEEE benchmark systems and a modified 415-bus China Southern Grid system are provided to validate the proposed methodology. It shows that the proposed methodology achieves an average 95% accuracy for detection, over 80% accuracy for localization of FDIA, and recovers the measurement and state variables close to their true values.

Discerning media bias within a network of political allies and opponents: The idealized example of a biased coin

Perceptions of political bias in the media are formed directly, through the independent consumption of the published outputs of a media organization, and indirectly, through observing the collective responses of political allies and opponents to the same published outputs. A network of Bayesian learners is constructed to model this system, in which the bias perceived by each agent obeys a probability density function, which is updated according to Bayes’s theorem given data about the published outputs and the beliefs of the agent’s political allies and opponents. The Bayesian framework allows for uncertain beliefs, multimodal probability distribution functions, and antagonistic interactions with opponents, not just cooperation with allies. Numerical simulations are performed to test the idealized example of inferring the bias of a coin. It is found that some agents converge on the wrong conclusion faster than others converge on the right conclusion under a surprisingly broad range of conditions, when antagonistic interactions are present which “lock out” some agents from the truth, e.g. in Barabási–Albert networks. It is also found that structurally unbalanced networks routinely experience turbulent nonconvergence, where some agents fail to achieve a steady-state belief, e.g. when they are allies of two agents who are opponents themselves. The subtle phenomenon of long-term intermittency is also explored.

Deep Samplable Observation Model for Global Localization and Kidnapping

Global localization and kidnapping are two challenging problems in robot localization. The popular method, Monte Carlo Localization (MCL) addresses the problem by iteratively updating a set of particles with a “sampling-weighting” loop. Sampling is decisive to the performance of MCL [1]. However, traditional MCL can only sample from a uniform distribution over the state space. Although variants of MCL propose different sampling models, they fail to provide an accurate distribution or generalize across scenes. To better deal with these problems, we present a distribution proposal model named Deep Samplable Observation Model (DSOM). DSOM takes a map and a 2D laser scan as inputs and outputs a conditional multimodal probability distribution of the pose, making the samples more focusing on the regions with higher likelihood. With such samples, the convergence is expected to be more effective and efficient. Considering that the learning-based sampling model may fail to capture the accurate pose sometimes, we furthermore propose the Adaptive Mixture MCL (AdaM MCL), which deploys a trusty mechanism to adaptively select updating mode for each particle to tolerate this situation. Equipped with DSOM, AdaM MCL can achieve more accurate estimation, faster convergence and better scalability than previous methods in both synthetic and real scenes. Even in real environments with long-term changes, AdaM MCL is able to localize the robot using DSOM trained only by simulation observations from a SLAM map or a blueprint map. Source code for this paper is available here: https://github.com/Runjian-Chen/AdaM_MCL.

Multimodal Deep Generative Models for Trajectory Prediction: A Conditional Variational Autoencoder Approach

Human behavior prediction models enable robots to anticipate how humans may react to their actions, and hence are instrumental to devising safe and proactive robot planning algorithms. However, modeling complex interaction dynamics and capturing the possibility of many possible outcomes in such interactive settings is very challenging, which has recently prompted the study of several different approaches. In this work, we provide a self-contained tutorial on a conditional variational autoencoder (CVAE) approach to human behavior prediction which, at its core, can produce a multimodal probability distribution over future human trajectories conditioned on past interactions and candidate robot future actions. Specifically, the goals of this tutorial paper are to review and build a taxonomy of state-of-the-art methods in human behavior prediction, from physics-based to purely data-driven methods, provide a rigorous yet easily accessible description of a data-driven, CVAE-based approach, highlight important design characteristics that make this an attractive model to use in the context of model-based planning for human-robot interactions, and provide important design considerations when using this class of models.

Deep Convolutional Mixture Density Network for Inverse Design of Layered Photonic Structures.

Machine learning (ML) techniques, such as neural networks, have emerged as powerful tools for the inverse design of nanophotonic structures. However, this innovative approach suffers some limitations. A primary one is the nonuniqueness problem, which can prevent ML algorithms from properly converging because vastly different designs produce nearly identical spectra. Here, we introduce a mixture density network (MDN) approach, which models the design parameters as multimodal probability distributions instead of discrete values, allowing the algorithms to converge in cases of nonuniqueness without sacrificing degenerate solutions. We apply our MDN technique to inversely design two types of multilayer photonic structures consisting of thin films of oxides, which present a significant challenge for conventional ML algorithms due to a high degree of nonuniqueness in their optical properties. In the 10-layer case, the MDN can handle transmission spectra with high complexity and under varying illumination conditions. The 4-layer case tends to show a stronger multimodal character, with secondary modes indicating alternative solutions for a target spectrum. The shape of the distributions gives valuable information for postprocessing and about the uncertainty in the predictions, which is not available with deterministic networks. Our approach provides an effective solution to the inverse design of photonic structures and yields more optimal searches for the structures with high degeneracy and spectral complexity.

Symbolic Learning and Reasoning With Noisy Data for Probabilistic Anchoring.

Robotic agents should be able to learn from sub-symbolic sensor data and, at the same time, be able to reason about objects and communicate with humans on a symbolic level. This raises the question of how to overcome the gap between symbolic and sub-symbolic artificial intelligence. We propose a semantic world modeling approach based on bottom-up object anchoring using an object-centered representation of the world. Perceptual anchoring processes continuous perceptual sensor data and maintains a correspondence to a symbolic representation. We extend the definitions of anchoring to handle multi-modal probability distributions and we couple the resulting symbol anchoring system to a probabilistic logic reasoner for performing inference. Furthermore, we use statistical relational learning to enable the anchoring framework to learn symbolic knowledge in the form of a set of probabilistic logic rules of the world from noisy and sub-symbolic sensor input. The resulting framework, which combines perceptual anchoring and statistical relational learning, is able to maintain a semantic world model of all the objects that have been perceived over time, while still exploiting the expressiveness of logical rules to reason about the state of objects which are not directly observed through sensory input data. To validate our approach we demonstrate, on the one hand, the ability of our system to perform probabilistic reasoning over multi-modal probability distributions, and on the other hand, the learning of probabilistic logical rules from anchored objects produced by perceptual observations. The learned logical rules are, subsequently, used to assess our proposed probabilistic anchoring procedure. We demonstrate our system in a setting involving object interactions where object occlusions arise and where probabilistic inference is needed to correctly anchor objects.

Long-run market configurations in a dynamic quality-ladder model with externalities

We study the impact of standard-setting by introducing an externality that increases product compatibility in the presence of asymmetric returns to investment in a dynamic quality-ladder-type model. We classify the long-run, multi-modal probability distributions over different market structures that arise from this model. In some cases, the lagging firm may remain in the market in the long-run depending on the strength of the externality. In the case where only the laggard invests in compatibility, it is possible that the laggard becomes a monopolist if the leader has a relatively low R&D capability and the two firms are almost symmetric in this same regard. This variety of multi-modal long-run distributions may have important consequences for the estimation and the simulation of this class of dynamic models.

Experimental Uncertainty Quantification of Flutter Derivatives for a PK Section Girder and Its Application on Probabilistic Flutter Analysis

Aerodynamic flutter instability could lead to the catastrophic collapse of flexible long-span bridges and needs to be prevented. This study investigated the variability of eight experimentally measured flutter derivatives (FDs) for a commonly used a Pasco–Kennewick (PK) section girder using a free vibration technique in a wind tunnel, which provided some new insights into the identification of FDs and supplemented the reliability analysis for buffeting and flutter performance. The aerodynamic uncertainties in terms of the dispersion of flutter coefficients were studied and two implications behind the extraction of FDs were discussed. The statistical and probabilistic behaviors of FDs at different reduced wind speeds were examined before generating sufficient independent random samples using the Monte Carlo (MC) approach. The intercorrelations between experimental FDs quantified by correlation coefficients were also examined and were employed to randomly sample the correlated FDs. The probabilistic flutter solutions in terms of critical wind speed propagated from the FDs uncertainties were studied utilizing step-by-step analysis, which resulted in several multimodal probability distributions of flutter onset. In addition, the role played by each FD on the variation of critical wind speed was discussed.

Sequential approximate reliability-based design optimization for structures with multimodal random variables

For practical engineering design problems, random variables tend to follow multimodal probability distributions when working at multiple operating conditions. For example, the structural fatigue stress of a steel bridge carrying both highway and railway traffic obeys the bimodal distribution. The existing popular reliability-based design optimization (RBDO) methods are mainly used to treat random variables with only unimodal distributions, which, therefore, tends to result in relatively large computational errors when multimodal random variables are involved. In this paper, a sequential approximate RBDO method is firstly proposed for engineering design involving multimodal random variables. The probability density function (PDF) of the response function is firstly calculated to assess the reliability of each probabilistic constraint, due to the existence of multimodal random variables. Then, a deterministic optimization problem is established to calculate candidate design points, based on the approximation that the response PDF demonstrates only transitional deformation in the optimization process. Three numerical examples and one engineering application of a thermal laptop design are presented to demonstrate the effectiveness of the proposed method.

Population dynamics of driven autocatalytic reactive mixtures.

Motivated by the effect of electroautocatalysis (explicit concentration dependence) on the stability of electrochemically driven phase-separating single particles, we apply the Fokker-Planck equation to describe the population dynamics of a general ensemble of chemically reactive particles. For phase-separating ensembles, we show that mosaic instability (from a homogeneous initial state to a multimodal probability distribution) may be suppressed or enhanced by autoinhibitory or autocatalytic reactions, respectively. In some cases, autocatalysis may induce two distinct populations in thermodynamically stable single-phase ensembles. Asymmetric reaction kinetics also results in qualitatively different population dynamics upon reversing the reaction direction. In the limit of negligible fluctuations, we use the method of characteristics and linearization to study the evolution of the concentration variance as well as the condition for mosaic instability, in good agreement with the full numerical solution. Applications include Li-ion batteries characterized by in situ x-ray diffraction.