Hierarchical Inference Research Articles

A Hierarchical Inference Model for Internet-of-Things

Internet-of-Things (IoT) has connected billions of devices to the Internet. These devices are already collecting zettabytes ( $10^{21}$ ) of data. However, the current IoT framework suffers from limited sensor energy, communication bandwidth, and server storage. These limitations impede the ability to send all the sensor data to the server all the time. Compact smart sensors provide a way to address this challenge. As opposed to the conventional sense-and-transmit sensors, emerging smart sensors can collect data, extract features, derive local inferences, and transmit only inference outcomes and possibly some raw data associated with rare events instead of all the raw data. This can dramatically cut down on the amount of sensor data transmitted, and hence its communication energy and network traffic. However, edge or server inference models trained with conventional machine learning approaches do not account for the fact that the smart sensors in the system have already performed a local inference. These approaches need all the sensor data and hence only cater to the traditional sense-and-transmit paradigm. This undoes the energy benefits brought about by smart sensors. In this paper, we propose a hierarchical inference model for IoT applications based on hierarchical learning and local inferences. Our model is able to take advantage of inference already performed on smart sensors, while at the same time accommodating conventional sense-and-transmit sensors in the IoT system. It also generalizes sensor-level inference to inference at other edge nodes by exploiting the intrinsically sensor/edge-grouped IoT data structure. We train classifiers hierarchically, aligned with the sensor-edge-server IoT paradigm. We verify our approach with seven IoT applications, demonstrating that the model is accurate, efficient, and generally applicable. We derive four edge-level inference models and four server-level inference models for these applications. For the four edge-level inference models, we reduce the number of bits transmitted from the sensor by $3.2\times$ - $42.7\times$ while at the same time also improving the classification accuracy by 0.3-6.7 percent. For the four server-level inference models, we reduce the number of edge-to-server bits transmitted by $17\times$ - $60\times$ , with classification accuracy change in the $-0.4$ - $+0.1$ percent range.

Towards Bayesian hierarchical inference of equilibrium climate sensitivity from a combination of CMIP5 climate models and observational data

Despite decades of research, large multi-model uncertainty remains about the Earth’s equilibrium climate sensitivity to carbon dioxide forcing as inferred from state-of-the-art Earth system models (ESMs). Statistical treatments of multi-model uncertainties are often limited to simple ESM averaging approaches. Sometimes models are weighted by how well they reproduce historical climate observations. Here, we propose a novel approach to multi-model combination and uncertainty quantification. Rather than averaging a discrete set of models, our approach samples from a continuous distribution over a reduced space of simple model parameters. We fit the free parameters of a reduced-order climate model to the output of each member of the multi-model ensemble. The reduced-order parameter estimates are then combined using a hierarchical Bayesian statistical model. The result is a multi-model distribution of reduced-model parameters, including climate sensitivity. In effect, the multi-model uncertainty problem within an ensemble of ESMs is converted to a parametric uncertainty problem within a reduced model. The multi-model distribution can then be updated with observational data, combining two independent lines of evidence. We apply this approach to 24 model simulations of global surface temperature and net top-of-atmosphere radiation response to abrupt quadrupling of carbon dioxide, and four historical temperature data sets. Our reduced order model is a 2-layer energy balance model. We present probability distributions of climate sensitivity based on (1) the multi-model ensemble alone and (2) the multi-model ensemble and observations.

HCN J = 4–3, HNC J = 1–0, H13CN J = 1–0, and HC3N J = 10–9 Maps of the Galactic Center Region. I. Spatially Resolved Measurements of Physical Conditions and Chemical Composition

Abstract This supplement paper presents the maps of HCN J = 4–3, HNC J = 1–0, J = 1–0, and HC3N J = 10–9 for the Galactic central molecular zone (CMZ), which have been obtained using the Atacama Submillimeter Telescope Experiment and Nobeyama Radio Observatory 45 m telescope. Three-dimensional maps (2D in space and 1D in velocity) of the gas kinetic temperature (T kin), hydrogen volume density ( ), and fractional abundances of eight molecules (HCN, HNC, , HCO+, , SiO, CS, and ) have been constructed from our and archival data. We have developed a method with hierarchical Bayesian inference for this analysis, which has successfully suppressed the artificial correlations among the parameters created by systematic errors due to the deficiency in the simple one-zone excitation analysis and the calibration uncertainty. The typical values of T kin and are and , respectively, and the presence of an additional cold, low-density component is also indicated. The distribution of high-temperature regions is poorly correlated with known active star-forming regions, while a few of them coincide with shocked clouds. Principal component analysis has identified two distinct groups in the eight analyzed molecules: one group with large PC1 and PC2 scores, and the other with a large T kin dependence, which could be explained using two regimes of shock chemistry with fast ( ) and slow ( ) velocity shocks, respectively. This supports the idea that the mechanical sputtering of dust grains and the mechanical heating play primary roles in the chemical and thermal processes in CMZ clouds.

Sound source localization and speech enhancement with sparse Bayesian learning beamforming.

Speech localization and enhancement involves sound source mapping and reconstruction from noisy recordings of speech mixtures with microphone arrays. Conventional beamforming methods suffer from low resolution, especially with a limited number of microphones. In practice, there are only a few sources compared to the possible directions-of-arrival (DOA). Hence, DOA estimation is formulated as a sparse signal reconstruction problem and solved with sparse Bayesian learning (SBL). SBL uses a hierarchical two-level Bayesian inference to reconstruct sparse estimates from a small set of observations. The first level derives the posterior probability of the complex source amplitudes from the data likelihood and the prior. The second level tunes the prior towards sparse solutions with hyperparameters which maximize the evidence, i.e., the data probability. The adaptive learning of the hyperparameters from the data auto-regularizes the inference problem towards sparse robust estimates. Simulations and experimental data demonstrate that SBL beamforming provides high-resolution DOA maps outperforming traditional methods especially for correlated or non-stationary signals. Specifically for speech signals, the high-resolution SBL reconstruction offers not only speech enhancement but effectively speech separation.

Web-based statistical tools for the analysis and design of clinical trials that incorporate historical controls

A collection of web-based statistical tools (http://research.mdacc.tmc.edu/SmeeactWeb/) are described that enable investigators to incorporate historical control data into analysis of randomized clinical trials using Bayesian hierarchical modeling as well as implement adaptive designs that balance posterior effective sample sizes among the study arms and thus maximize power. With balanced allocation guided by “dynamic” Bayesian hierarchical modeling, the design offers the potential to assign more patients to experimental therapies and thereby enhance efficiency while limiting bias and controlling average type I error. The tools effectuate analysis and design for static (non-hierarchical Bayesian analysis) and two types of dynamic (hierarchical Bayesian inference using empirical Bayes and spike-and-slab hyperprior) methods for Gaussian data models, as well as a dynamic method for time-to-failure endpoints based on a piecewise constant hazard model. The site also offers interfaces to facilitate calibration of the model hyperparameters. These allow users to test different parameters in the presence of the historical data on the basis of their resultant frequentist properties, including bias and mean squared error. All calculations are performed on a central computational server. The user may upload data, choose trial settings, run computations in real-time, and review the results using only a web browser. The back-end web module, computation module, and MCMC sampling module are developed in the C#, R, and C++ languages, respectively, and a communication module is also available to ensure the continued connection between the client computer and the back-end server during the Bayesian computations. The statistical tools are described and demonstrated with examples.

Enactivism and predictive processing: a non-representational view

This paper starts by considering an argument for thinking that predictive processing (PP) is representational. This argument suggests that the Kullback–Leibler (KL)-divergence provides an accessible measure of misrepresentation, and therefore, a measure of representational content in hierarchical Bayesian inference. The paper then argues that while the KL-divergence is a measure of information, it does not establish a sufficient measure of representational content. We argue that this follows from the fact that the KL-divergence is a measure of relative entropy, which can be shown to be the same as covariance (through a set of additional steps). It is well known that facts about covariance do not entail facts about representational content. So there is no reason to think that the KL-divergence is a measure of (mis-)representational content. This paper thus provides an enactive, non-representational account of Bayesian belief optimisation in hierarchical PP.

On parametrized cold dense matter equation-of-state inference

Constraining the equation of state of cold dense matter in compact stars is a major science goal for observing programmes being conducted using X-ray, radio, and gravitational wave telescopes. We discuss Bayesian hierarchical inference of parametrised dense matter equations of state. In particular we generalise and examine two inference paradigms from the literature: (i) direct posterior equation of state parameter estimation, conditioned on observations of a set of rotating compact stars; and (ii) indirect parameter estimation, via transformation of an intermediary joint posterior distribution of exterior spacetime parameters (such as gravitational masses and coordinate equatorial radii). We conclude that the former paradigm is not only tractable for large-scale analyses, but is principled and flexible from a Bayesian perspective whilst the latter paradigm is not. The thematic problem of Bayesian prior definition emerges as the crux of the difference between these paradigms. The second paradigm should in general only be considered as an ill-defined approach to the problem of utilising archival posterior constraints on exterior spacetime parameters; we advocate for an alternative approach whereby such information is repurposed as an approximative likelihood function. We also discuss why conditioning on a piecewise-polytropic equation of state model - currently standard in the field of dense matter study - can easily violate conditions required for transformation of a probability density distribution between spaces of exterior (spacetime) and interior (source matter) parameters.

A Bayesian hierarchical approach to galaxy–galaxy lensing

We present a Bayesian hierarchical inference formalism to study the relation between the properties of dark matter halos and those of their central galaxies using weak gravitational lensing. Unlike traditional methods, this technique does not resort to stacking the weak lensing signal in bins, and thus allows for a more efficient use of the information content in the data. Our method is particularly useful for constraining scaling relations between two or more galaxy properties and dark matter halo mass, and can also be used to constrain the intrinsic scatter in these scaling relations. We show that, if observational scatter is not properly accounted for, the traditional stacking method can produce biased results when exploring correlations between multiple galaxy properties and halo mass. For example, this bias can affect studies of the joint correlation between galaxy mass, halo mass, and galaxy size, or galaxy color. In contrast, our method easily and efficiently handles the intrinsic and observational scatter in multiple galaxy properties and halo mass. We test our method on mocks with varying degrees of complexity. We find that we can recover the mean halo mass and concentration, each with a $0.1$ dex accuracy, and the intrinsic scatter in halo mass with a $0.05$ dex accuracy. In its current version, our method will be most useful for studying the weak lensing signal around central galaxies in groups and clusters, as well as massive galaxies samples with $\log{M_*} > 11$, which have low satellite fractions.

Nested Hierarchical Functional Data Modeling and Inference for the Analysis of Functional Plant Phenotypes

ABSTRACTIn a plant science Root Image Study, the process of seedling roots bending in response to gravity is recorded using digital cameras, and the bending rates are modeled as functional plant phenotype data. The functional phenotypes are collected from seeds representing a large variety of genotypes and have a three-level nested hierarchical structure, with seeds nested in groups nested in genotypes. The seeds are imaged on different days of the lunar cycle, and an important scientific question is whether there are lunar effects on root bending. We allow the mean function of the bending rate to depend on the lunar day and model the phenotypic variation between genotypes, groups of seeds imaged together, and individual seeds by hierarchical functional random effects. We estimate the covariance functions of the functional random effects by a fast penalized tensor product spline approach, perform multi-level functional principal component analysis (FPCA) using the best linear unbiased predictor of the principal component scores, and improve the efficiency of mean estimation by iterative decorrelation. We choose the number of principal components using a conditional Akaike information criterion and test the lunar day effect using generalized likelihood ratio test statistics based on the marginal and conditional likelihoods. We also propose a permutation procedure to evaluate the null distribution of the test statistics. Our simulation studies show that our model selection criterion selects the correct number of principal components with remarkably high frequency, and the likelihood-based tests based on FPCA have higher power than a test based on working independence. Supplementary materials for this article are available online.

Predictive Coding in Area V4: Dynamic Shape Discrimination under Partial Occlusion.

The primate visual system has an exquisite ability to discriminate partially occluded shapes. Recent electrophysiological recordings suggest that response dynamics in intermediate visual cortical area V4, shaped by feedback from prefrontal cortex (PFC), may play a key role. To probe the algorithms that may underlie these findings, we build and test a model of V4 and PFC interactions based on a hierarchical predictive coding framework. We propose that probabilistic inference occurs in two steps. Initially, V4 responses are driven solely by bottom-up sensory input and are thus strongly influenced by the level of occlusion. After a delay, V4 responses combine both feedforward input and feedback signals from the PFC; the latter reflect predictions made by PFC about the visual stimulus underlying V4 activity. We find that this model captures key features of V4 and PFC dynamics observed in experiments. Specifically, PFC responses are strongest for occluded stimuli and delayed responses in V4 are less sensitive to occlusion, supporting our hypothesis that the feedback signals from PFC underlie robust discrimination of occluded shapes. Thus, our study proposes that area V4 and PFC participate in hierarchical inference, with feedback signals encoding top-down predictions about occluded shapes.

Information Sharing Assessment in Supply Chain: Hierarchical Fuzzy Rule-Based System

As one of three prominent flows (material, money, information) in a supply chain, the information flow plays a key role in supply chain performance and also in reaching high degrees of integrity and cooperation between supply chain members. Regarding the vagueness and multi-dimensional nature of information sharing, evaluating the information sharing level in supply chain is a complex problem. This paper aims to develop a new rule-based inference system to evaluate the level of information sharing in a supply chain. To this end, first, the influencing parameters are obtained from the literature and are screened and refined by the experts, using Lexicography method. Then, they are categorised into a multi-level hierarchical structure. Finally, a hierarchical inference system, which consists of a set of fuzzy rule bases, is constructed to infer the information sharing degree. The use of fuzzy rule bases is due to the lack of explicit functions between the influencing factors as well as ambiguity of these parameters. Also, in this research, the input parameters’ measurement process is defined and some relevant indices are designed. The developed assessment system is a four-leveled hierarchical inference system, built of six fuzzy rule bases. The inference system has eight input parameters, the values of which are calculated utilising six designed indices and two quick questionnaires. Eventually, the developed inference system and the measuring method are implemented in a real case.

Forecasting Monthly Electricity Demands by Wavelet Neuro-Fuzzy System Optimized by Heuristic Algorithms

Electricity load forecasting plays a paramount role in capacity planning, scheduling, and the operation of power systems. Reliable and accurate planning and prediction of electricity load are therefore vital. In this study, a novel approach for forecasting monthly electricity demands by wavelet transform and a neuro-fuzzy system is proposed. Firstly, the most appropriate inputs are selected and a dataset is constructed. Then, Haar wavelet transform is utilized to decompose the load data and eliminate noise. In the model, a hierarchical adaptive neuro-fuzzy inference system (HANFIS) is suggested to solve the curse-of-dimensionality problem. Several heuristic algorithms including Gravitational Search Algorithm (GSA), Cuckoo Optimization Algorithm (COA), and Cuckoo Search (CS) are utilized to optimize the clustering parameters which help form the rule base, and adaptive neuro-fuzzy inference system (ANFIS) optimize the parameters in the antecedent and consequent parts of each sub-model. The proposed approach was applied to forecast the electricity load of Hanoi, Vietnam. The constructed models have shown high forecasting performances based on the performance indices calculated. The results demonstrate the validity of the approach. The obtained results were also compared with those of several other well-known methods including autoregressive integrated moving average (ARIMA) and multiple linear regression (MLR). In our study, the wavelet CS-HANFIS model outperformed the others and provided more accurate forecasting.

A dust spectral energy distribution model with hierarchical Bayesian inference – I. Formalism and benchmarking

This article presents a new dust SED model, named HerBIE, aimed at eliminating the noise-induced correlations and large scatter obtained when performing least-squares fits. The originality of this code is to apply the hierarchical Bayesian approach to full dust models, including realistic optical properties, stochastic heating and the mixing of physical conditions in the observed regions. We test the performances of our model by applying it to synthetic observations. We explore the impact on the recovered parameters of several effects: signal-to-noise ratio, SED shape, sample size, the presence of intrinsic correlations, the wavelength coverage and the use of different SED model components. We show that this method is very efficient: the recovered parameters are consistently distributed around their true values. We do not find any clear bias, even for the most degenerate parameters, or with extreme signal-to-noise ratios.

Архітектура програмної системи ієрархічного нечіткого логічного виведення

Архітектура програмної системи ієрархічного нечіткого логічного виведення

The Neuronal Basis of Predictive Coding Along the Auditory Pathway: From the Subcortical Roots to Cortical Deviance Detection.

In this review, we attempt to integrate the empirical evidence regarding stimulus-specific adaptation (SSA) and mismatch negativity (MMN) under a predictive coding perspective (also known as Bayesian or hierarchical-inference model). We propose a renewed methodology for SSA study, which enables a further decomposition of deviance detection into repetition suppression and prediction error, thanks to the use of two controls previously introduced in MMN research: the many-standards and the cascade sequences. Focusing on data obtained with cellular recordings, we explain how deviance detection and prediction error are generated throughout hierarchical levels of processing, following two vectors of increasing computational complexity and abstraction along the auditory neuraxis: from subcortical toward cortical stations and from lemniscal toward nonlemniscal divisions. Then, we delve into the particular characteristics and contributions of subcortical and cortical structures to this generative mechanism of hierarchical inference, analyzing what is known about the role of neuromodulation and local microcircuitry in the emergence of mismatch signals. Finally, we describe how SSA and MMN are occurring at similar time frame and cortical locations, and both are affected by the manipulation of N-methyl-D-aspartate receptors. We conclude that there is enough empirical evidence to consider SSA and MMN, respectively, as the microscopic and macroscopic manifestations of the same physiological mechanism of deviance detection in the auditory cortex. Hence, the development of a common theoretical framework for SSA and MMN is all the more recommendable for future studies. In this regard, we suggest a shared nomenclature based on the predictive coding interpretation of deviance detection.

An adaptive map-matching algorithm based on hierarchical fuzzy system from vehicular GPS data.

An improved hierarchical fuzzy inference method based on C-measure map-matching algorithm is proposed in this paper, in which the C-measure represents the certainty or probability of the vehicle traveling on the actual road. A strategy is firstly introduced to use historical positioning information to employ curve-curve matching between vehicle trajectories and shapes of candidate roads. It improves matching performance by overcoming the disadvantage of traditional map-matching algorithm only considering current information. An average historical distance is used to measure similarity between vehicle trajectories and road shape. The input of system includes three variables: distance between position point and candidate roads, angle between driving heading and road direction, and average distance. As the number of fuzzy rules will increase exponentially when adding average distance as a variable, a hierarchical fuzzy inference system is then applied to reduce fuzzy rules and improve the calculation efficiency. Additionally, a learning process is updated to support the algorithm. Finally, a case study contains four different routes in Beijing city is used to validate the effectiveness and superiority of the proposed method.

Deep Gaussian mixture models

Deep learning is a hierarchical inference method formed by subsequent multiple layers of learning able to more efficiently describe complex relationships. In this work, deep Gaussian mixture models (DGMM) are introduced and discussed. A DGMM is a network of multiple layers of latent variables, where, at each layer, the variables follow a mixture of Gaussian distributions. Thus, the deep mixture model consists of a set of nested mixtures of linear models, which globally provide a nonlinear model able to describe the data in a very flexible way. In order to avoid overparameterized solutions, dimension reduction by factor models can be applied at each layer of the architecture, thus resulting in deep mixtures of factor analyzers.

Hierarchical Probabilistic Inference of the Color–Magnitude Diagram and Shrinkage of Stellar Distance Uncertainties

Abstract We present a hierarchical probabilistic model for improving geometric stellar distance estimates using color–magnitude information. This is achieved with a data-driven model of the color–magnitude diagram, not relying on stellar models but instead on the relative abundances of stars in color–magnitude cells, which are inferred from very noisy magnitudes and parallaxes. While the resulting noise-deconvolved color–magnitude diagram can be useful for a range of applications, we focus on deriving improved stellar distance estimates relying on both parallax and photometric information. We demonstrate the efficiency of this approach on the 1.4 million stars of the Gaia TGAS sample that also have AAVSO Photometric All Sky Survey magnitudes. Our hierarchical model has 4 million parameters in total, most of which are marginalized out numerically or analytically. We find that distance estimates are significantly improved for the noisiest parallaxes and densest regions of the color–magnitude diagram. In particular, the average distance signal-to-noise ratio (S/N) and uncertainty improve by 19% and 36%, respectively, with 8% of the objects improving in S/N by a factor greater than 2. This computationally efficient approach fully accounts for both parallax and photometric noise and is a first step toward a full hierarchical probabilistic model of the Gaia data.

Hierarchical Bayesian Inference and Learning in Spiking Neural Networks.

Numerous experimental data from neuroscience and psychological science suggest that human brain utilizes Bayesian principles to deal the complex environment. Furthermore, hierarchical Bayesian inference has been proposed as an appropriate theoretical framework for modeling cortical processing. However, it remains unknown how such a computation is organized in the network of biologically plausible spiking neurons. In this paper, we propose a hierarchical network of winner-take-all circuits which can carry out hierarchical Bayesian inference and learning through a spike-based variational expectation maximization (EM) algorithm. Particularly, we show how the firing activities of spiking neurons in response to the input stimuli and the spike-timing-dependent plasticity rule can be understood, respectively, as variational E-step and M-step of variational EM. Finally, we demonstrate the utility of this spiking neural network on the MNIST benchmark for unsupervised classification of handwritten digits.

Soybean response to nitrogen application across the United States: A synthesis-analysis

The effects of supplemental nitrogen (N) on soybean [Glycine max (L.) Merr.] seed yield have been the focus of much research over the past four decades. However, most experiments were region-specific and focused on the effect of a single N-related management choice, thus resulting in a limited inference space. Here, we composited data from individual experiments conducted across the US that examined the effect of N fertilization on soybean yield. The combined database included 207 environments (experiment×year combinations) for a total of 5991 N-treated soybean yields. We used hierarchical modeling and conditional inference tree analysis on the combined dataset to establish the relationship and contribution of several N management choices on soybean yield. The N treatment variables were: N-application (single or split), N-method (soil incorporated, foliar, etc.), N-timing (pre-plant, at a reproductive stage, etc.), and N-rate (from a 0N control to as much as 560kgha−1). Of the total yield variability, 68% was associated with the effect of environment, whereas only a small fraction of that variability (<1%) was attributable to each N variable. Averaged over all experiments, a single N application and the split N application were 60 and 110kgha−1 greater yielding than the zero N control treatment, respectively. A split N application with more than one method (e.g., soil incorporated and foliar) resulted in 120kgha−1 greater yield than zero N plots. Split N application between planting and reproductive stages (Rn) resulted in greater yield than zero N and single application during a Rn; however, the effect was not significantly different than N application at other growth stages. Increasing the N rate increased the environment average soybean yield; however, 93% of the environment-specific N-rate responses were not significant which suggested a minimal effect of N across the examined region. A large yield variability was observed among environments within the same N rates, which was attributed to growing environment differences (e.g., in-season weather conditions, soil type etc.) and non-N related management (e.g., irrigation). Conditional inference tree analysis identified N-timing and N-rate to be conditional to irrigation, and to seeding rates >420,000seedsha−1, indicating that N management decisions should take into account major, non-N related management practices. Overall, the analysis revealed that N management decisions had a measurable, but small, effect on soybean yield. Given the growing pressure for increasing food production, it is imperative to further examine all soybean N decisions (application method, timing, and rate) in environment- and cropping system-specific randomized trials in important agricultural regions.