Hierarchical Inference Research Articles

Resolution for label-based formulas in hierarchical representation

Order-sorted logic iucludes many and partially ordered sorts as a sort-hierarchy. In the field of knowledge representation and reasoning, it is useful to develop reasoning systems fbr terminological knowledge, together with assertional knowledge. However, the expression of sort-hierarchies cannot sufficiently capture the lexical diversity of terminological knowledge. In addition to sorts, various kinds of symbols: constants, functions and predicates are semantically and hierarchically associated with each other. This is because natural language words identifying these symbols can be employed in the description of terminological knowledge. In this paper, we present a label-based language for consistently handling the variety of hierarchical relationships among symbol names. For this language we develop a sorted resolution system whose reasoning power is enhanced by adding hierarchical inference rules with labeled substitutions.

Hierarchical Bayes multiple-breed inference with an application to genetic evaluation of a Nelore-Hereford population.

The primary objective of this study was to demonstrate the utility of a hierarchical Bayes implementation of a multiple-breed animal model (MBAM) to estimate breed composition means and additive genetic variances as well as on variances due to the segregation between breeds. The MBAM and a conventional animal model (AM) were both applied to five simulated data sets derived from each of two different populations. Population I consisted of crosses between two breeds having a twofold difference in genetic variance and a nonzero segregation variance. Population II had the same population structure as Population I, except that the two breeds had the same genetic variance with no segregation variance; that is, Population II was essentially single breed in its genetic architecture. For Population I, posterior means of all variance components obtained by MBAM were unbiased, with 95% posterior probability intervals (PPI) having the expected coverage based on five replicates. The MBAM showed a slightly superior performance over the AM for genetic predictions in Population I, although there was no evidence that the use of the MBAM translated into greater genetic gains relative to the use of the AM. Nevertheless, the MBAM was clearly demonstrated to have superior fit to the data using pseudo-Bayes factors (PBF) as the basis for model choice. As expected, the MBAM and AM performed equally well in Population II. A data set consisting of 22,717 postweaning gain (PWG) records of a Nelore-Hereford population (40,082 animals in the pedigree) also was analyzed. The MBAM inference on Nelore and Hereford additive heritabilities (h2A) substantially differed. Herefords had a posterior mean h2A of 0.20 with a 95% PPI of 0.15 to 0.25, whereas the corresponding values for the Nelores were 0.07 and 0.04 to 0.11, respectively. The posterior mean genetic variance due to the segregation between these breeds was 8.4 kg2, with a 95% PPI of 2.3 to 24.8 kg2, and represented 35.4% of the Nelore but only 9.9% of the Hereford posterior mean genetic variance. The posterior mean h2A using the AM was 0.15, presumed common across the two breeds, with a 95% PPI of 0.11 to 0.19. The PBF heavily favored the MBAM over the AM for the PWG data. Accordingly, the MBAM represents a viable alternative to AM for multiple-breed genetic evaluations, providing the necessary flexibility in modeling heteroskedastic genetic variances of breed composition groups.

BITE: A Bayesian Intensity Estimator

BITE is a software package designed for the analysis of event history data using flexible hierarchical models and Bayesian inference, with a particular emphasis on the application of flexible intensities as a description of the distribution of lifetimes. BITE provides a framework for combining flexible baseline hazard rates and observed data into intensity processes. Inclusion of covariate information is possible, and data can be non-informatively and independently filtered, or censored. The model and the data are described by a command language and data are stored into text files. Markov chain Monte Carlo methods are used for numerical approximation of expectations with respect to the posterior. Output consists of (i) parameter values stored during simulations, (ii) estimated expectations of functionals of parameters, or (iii) graphs (created with Splus or R software packages) presenting point-wise expectations (and credibility intervals) of the baseline hazard rates.

The influence of hierarchy on probability judgment

Consider the task of predicting which soccer team will win the next World Cup. The bookmakers may judge Brazil to be the team most likely to win, but also judge it most likely that a European rather than a Latin American team will win. This is an example of a non-aligned hierarchy structure: the most probable event at the subordinate level (Brazil wins) appears to be inconsistent with the most probable event at the superordinate level (a European team wins). In this paper we exploit such structures to investigate how people make predictions based on uncertain hierarchical knowledge. We distinguish between aligned and non-aligned environments, and conjecture that people assume alignment. Participants were exposed to a non-aligned training set in which the most probable superordinate category predicted one outcome, whereas the most probable subordinate category predicted a different outcome. In the test phase participants allowed their initial probability judgments about category membership to shift their final ratings of the probability of the outcome, even though all judgments were made on the basis of the same statistical data. In effect people were primed to focus on the most likely path in an inference tree, and neglect alternative paths. These results highlight the importance of the level at which statistical data are represented, and suggest that when faced with hierarchical inference problems people adopt a simplifying heuristic that assumes alignment.

Hierarchical Bayesian inference in the visual cortex.

Traditional views of visual processing suggest that early visual neurons in areas V1 and V2 are static spatiotemporal filters that extract local features from a visual scene. The extracted information is then channeled through a feedforward chain of modules in successively higher visual areas for further analysis. Recent electrophysiological recordings from early visual neurons in awake behaving monkeys reveal that there are many levels of complexity in the information processing of the early visual cortex, as seen in the long-latency responses of its neurons. These new findings suggest that activity in the early visual cortex is tightly coupled and highly interactive with the rest of the visual system. They lead us to propose a new theoretical setting based on the mathematical framework of hierarchical Bayesian inference for reasoning about the visual system. In this framework, the recurrent feedforward/feedback loops in the cortex serve to integrate top-down contextual priors and bottom-up observations so as to implement concurrent probabilistic inference along the visual hierarchy. We suggest that the algorithms of particle filtering and Bayesian-belief propagation might model these interactive cortical computations. We review some recent neurophysiological evidences that support the plausibility of these ideas.

Evidential reasoning for object recognition

The authors present a framework to guide development of evidential reasoning in object recognition systems. Principles of evidential reasoning processes for open-world object recognition are proposed and applied to build evidential reasoning capabilities. The principles summarize research and findings by the authors up through the mid-1990s, including seminal results in object-centered computer vision, figure-ground discrimination, and the application of hierarchical Bayesian inference, Bayesian networks, and decision graphs to evidential reasoning for object recognition.

A Procedure for Assessing the Influence of Problem Domain on Effort Estimation Consistency

By and large, given the inherent subjectivity in defining and measuring factors used in algorithmic effort estimation methods, when algorithmic methods produce consistent estimates it seems reasonable to assume that this is in part due to estimator experience. Further, software development factors are usually assumed to have different degrees of influence on actual effort. For example, no specific allowances for program language or problem domain were made in the original COCOMO model or in Albrecht's Function Points, whilst allowances for development mode in COCOMO and function type complexity for Albrecht's Function Points are crucial. However, work has been conducted that concluded that 4GLs are associated with higher productivity than 3GLs. Clearly, we can support such conclusions about productivity, since, for example, it usually requires less effort to develop a database using a purposely designed DBMS product than it does using a 3GL. However, in general, for a given problem domain an appropriate development language and platform will be selected. Hence, we might feel that an appropriate development language will not be a factor that influences estimate consistency unduly, given that an estimator has experience of the problem domain. However, algorithmic methods usually require calibration to different problem domains. Calibration may be needed because the method was originally designed using data from another type of domain. Furthermore, estimators' estimation consistency within problem domains may be affected for one or more reasons. Intuitively, reasons might include: estimators lack estimation experience in some domains; or the development team(s) may have different levels of experience in different domains, which the estimator finds difficult to take into account. We demonstrate how, in general, the influence of problem domain may be assessed using a Hierarchical Bayesian inference procedure. We also show how values can be derived to account for variations in estimate consistency in problem domains.

A proposed hierarchical fuzzy inference system for the diagnosis of arthritic diseases.

Development of computer-based medical inference systems is always confronted with some difficulties. In this paper, difficulties of designing an inference system for the diagnosis of arthritic diseases are described, including variations of disease manifestations under various situations and conditions. Furthermore, the need for a huge knowledge base would result in low efficiency of the inference system. We proposed a hierarchical model of the fuzzy inference system as a possible solution. With such a model, the diagnostic process is divided into two levels. The first level of the diagnosis reduces the scope of diagnosis to be processed by the second level. This will reduce the amount of input and mapping for the whole diagnostic process. Fuzzy relational theory is the core of this system and it is used in both levels to improve the accuracy.

다형질 Threshold 개체모형에서 Missing 기록을 포함한 이산형 자료에 대한 Bayesian 분석

한우의 근내지방도 또는 임신 여부 등과 같이 이산형 분포의 성질을 갖는 다수의 형질들에 대한 유전모수 및 종축의 유전능력을 평가하기 위한 방법으로써 Threshold 모형하에서 Bayesian 추론방법의 일종인 Gibbs sampling방법을 모의실험을 통하여 알아보았으며 기록이 누락된 다수의 형질을 포함하는 다형질 Threshold 개체모형에서의 종축평가 방법론을 제시하였다. 이산형 형질의 관측치에 대응하는 임의의 잠재변수는 기록을 갖고 있는 형질들에 대한 사전정보를 고려한 사후조건확률분포에서 Gibbs sampling을 할 때 모수에 근접하는 확률분포를 얻을 수 있었으며 이러한 이산형 기록들에 대한 육종가 추정치는 선형모형에서 보다 Threshold 모형에서의 추정치가 실제 모수에 더욱 근접하는 것을 알 수 있었다. 따라서 기록이 누락된 개체들에 대한 이산형 분포를 갖는 형질들에 대하여 선형분포를 갖는 형질들과 함께 동시 유전분석할 때 Threshod 모형이 일반 선형모형 보다 적합함을 알 수 있었다. Genetic variance and covariance components of the linear traits and the ordered categorical traits, that are usually observed as dichotomous or polychotomous outcomes, were simultaneously estimated in a multivariate threshold animal model with concepts of arbitrary underlying liability scales with Bayesian inference via Gibbs sampling algorithms. A multivariate threshold animal model in this study can be allowed in any combination of missing traits with assuming correlation among the traits considered. Gibbs sampling algorithms as a hierarchical Bayesian inference were used to get reliable point estimates to which marginal posterior means of parameters were assumed. Main point of this study is that the underlying values for the observations on the categorical traits sampled at previous round of iteration and the observations on the continuous traits can be considered to sample the underlying values for categorical data and continuous data with missing at current cycle (see appendix). This study also showed that the underlying variables for missing categorical data should be generated with taking into account for the correlated traits to satisfy the fully conditional posterior distributions of parameters although some of papers (Wang et al., 1997; VanTassell et al., 1998) presented that only the residual effects of missing traits were generated in same situation. In present study, Gibbs samplers for making the fully Bayesian inferences for unknown parameters of interests are played rolls with methodologies to enable the any combinations of the linear and categorical traits with missing observations. Moreover, two kinds of constraints to guarantee identifiability for the arbitrary underlying variables are shown with keeping the fully conditional posterior distributions of those parameters. Numerical example for a threshold animal model included the maternal and permanent environmental effects on a multiple ordered categorical trait as calving ease, a binary trait as non-return rate, and the other normally distributed trait, birth weight, is provided with simulation study.

NETEST: Estimating a Terrorist Network's Structure—Graduate Student Best Paper Award, CASOS 2002 Conference

Since the events of September 11, 2001, the United States has found itself engaged in an unconventional and asymmetric form of warfare against elusive terrorist organizations. Defense and investigative organizations require innovative solutions that will assist them in determining the membership and structure of these organizations. Data on covert organizations are often in the form of disparate and incomplete inferences of memberships and connections between members. NETEST is a tool that combines multi-agent technology with hierarchical Bayesian inference models and biased net models to produce accurate posterior representations of a network. Bayesian inference models produce representations of a network's structure and informant accuracy by combining prior network and accuracy data with informant perceptions of a network. Biased net theory examines and captures the biases that may exist in a specific network or set of networks. Using NETEST, an investigator has the power to estimate a network's size, determine its membership and structure, determine areas of the network where data is missing, perform cost/benefit analysis of additional information, assess group level capabilities embedded in the network, and pose “what if” scenarios to destabilize a network and predict its evolution over time.

Determination of an optimal dosage regimen using a Bayesian decision analysis of efficacy and adverse effect data.

One of the aims of Phase II clinical trials is to determine the dosage regimen(s) that will be investigated during a confirmatory Phase III clinical trial. During Phase II, pharmacodynamic data are collected that enables the efficacy and safety of the drug to be assessed. It is proposed in this paper to use Bayesian decision analysis to determine the optimal dosage regimen based on efficacy and toxicity of the drug oxybutynin used in the treatment of urinary urge incontinence. Such an approach results in a general framework allowing modeling, inference and decision making to be carried out. For oxybutynin, the repeated measurement efficacy and toxicity data were modeled using nonlinear hierarchical models and inferences were based on posterior probabilities. The optimal decision in this problem was to determine the dosage regimen that maximized the posterior expected utility given the prior information on the model parameters and the patient response data. The utility function was defined using clinical opinion on the satisfactory levels of efficacy and toxicity and then combined by weighting the relative importance of each pharmacodynamic response. Markov chain Monte Carlo (MCMC) methodology implemented in Win-BUGS 1.3 was used to obtain posterior estimates of the model parameters, probabilities and utilities.

Empirical and Hierarchical Bayesian Estimation of Ancestral States

Several methods have been proposed to infer the states at the ancestral nodes on a phylogeny. These methods assume a specific tree and set of branch lengths when estimating the ancestral character state. Inferences of the ancestral states, then, are conditioned on the tree and branch lengths being true. We develop a hierarchical Bayes method for inferring the ancestral states on a tree. The method integrates over uncertainty in the tree, branch lengths, and substitution model parameters by using Markov chain Monte Carlo. We compare the hierarchical Bayes inferences of ancestral states with inferences of ancestral states made under the assumption that a specific tree is correct. We find that the methods are correlated, but that accommodating uncertainty in parameters of the phylogenetic model can make inferences of ancestral states even more uncertain than they would be in an empirical Bayes analysis.

Fuzzy configuration of matching runtime implementation strategies

With applications currently growing in complexity and range, increasing numbers of configuration problems are arising in compilers. Already many software systems offer multiple specialized implementation strategies and substrategies, differing in terms of applicability and/or cost, depending on the application context. Configurations then have to be created from the different strategies available in accordance with the application characteristics, the global optimization objective, and potential constraints on the strategies' combinability. In many cases, this results in a combinatorial, i.e., discrete, optimization problem. Proper solutions for automating the configuration while limiting the complexity of the solution search are still being sought. We address here the field of parallel/distributed processing and the configuration of runtime implementation strategies, such as for communication or dynamic load balancing. We present a rule-based approach, integrating fuzzy methodologies for the classification of application characteristics and for gradual selection preference in rules. In this way we exploit available knowledge about the correlation of the problem and solution space, and apply soft computing methods to obtain an approximate, rather than perfect, solution approach, thus helping to limit the configuration complexity. Our approach extends standard fuzzy inference by a multistage organization, and – with proper organization of rules, characteristics and strategies – performs hierarchical fuzzy inference. The approach is demonstrated on concrete configuration examples in parallel compilers.

Visual learning with navigation as an example

The state-based learning method presented is applicable to virtually any vision-based control problem. We use navigation as an example. In a controlled environment, we can define a few known landmarks before system design, and the navigation system can employ landmark detectors. Such navigation systems typically employ a model-based design method. However these methods have difficulties dealing with learning in complex, changing environments. To overcome these limitations, we have developed Shoslif (Self-organizing Hierarchical Optimal Subspace Learning and Inference Framework), a model-free, learning-based approach. Shoslif introduces mechanisms such as automatic feature derivation, a self-organizing tree structure to reach a very low logarithmic time complexity, one-instance learning, and incremental learning without forgetting prior memorized information. In addition, we have created a state-based version of Shoslif that lets humans teach robots to use past history and local views that are useful for disambiguation. Shoslif-N is a prototype autonomous navigation system using Shoslif. We have tested Shoslif-N primarily indoors. Indoor navigation encounters fewer lighting changes than outdoor navigation. However, it offers other, considerable challenges for vision-based navigation. Shoslif-N has shown that it can navigate in real time reliably in an unaltered indoor environment for an extended amount of time and distance, without any special image-processing hardware.

Skeletal growth estimation using radiographic image processing and analysis

An automated knowledge-based vision system for skeletal growth estimation in children is reported in this paper. Images were obtained from hand radiographs of 32 male and 25 female children of age 1-16 yr. Phalanx bones were automatically localized and segmented using hierarchical inferences and active shape models, respectively. A number of shape descriptors were obtained from the segmented bone contour to quantify skeletal growth. From these descriptors, a feature vector was selected for a regression model and a Bayesian estimator. The estimation accuracy was 84% for females and 82% for males. This level of accuracy is comparable to that of expert pediatric radiologists, which suggests that the proposed approach has a potential application in pediatric medicine.

State-based SHOSLIF for indoor visual navigation

In this paper, we investigate vision-based navigation using the self-organizing hierarchical optimal subspace learning and inference framework (SHOSLIF) that incorporates states and a visual attention mechanism. With states to keep the history information and regarding the incoming video input as an observation vector, the vision-based navigation is formulated as an observation-driven Markov model (ODMM). The ODMM can be realized through recursive partitioning regression. A stochastic recursive partition tree (SRPT), which maps an preprocessed current input raw image and the previous state into the current state and the next control signal, is used for efficient recursive partitioning regression. The SRPT learns incrementally: each learning sample is learned or rejected "on-the-fly." The purposed scheme has been successfully applied to indoor navigation.

Hierarchical discriminant analysis for image retrieval

A self-organizing framework for object recognition is described. We describe a hierarchical database structure for image retrieval. The self-organizing hierarchical optimal subspace learning and inference framework (SHOSLIF) system uses the theories of optimal linear projection for optimal feature derivation and a hierarchical structure to achieve logarithmic retrieval complexity. A space-tessellation tree is generated using the most expressive features (MEF) and most discriminating features (MDF) at each level of the tree. The major characteristics of the analysis include: (1) avoiding the limitation of global linear features by deriving a recursively better-fitted set of features for each of the recursively subdivided sets of training samples; (2) generating a smaller tree whose cell boundaries separate the samples along the class boundaries better than the principal component analysis, thereby giving a better generalization capability (i.e., better recognition rate in a disjoint test); (3) accelerating the retrieval using a tree structure for data pruning, utilizing a different set of discriminant features at each level of the tree. We allow for perturbations in the size and position of objects in the images through learning. We demonstrate the technique on a large image database of widely varying real-world objects taken in natural settings, and show the applicability of the approach for variability in position, size, and 3D orientation. This paper concentrates on the hierarchical partitioning of the feature spaces.

Decision Support Under Incomplete and Uncertain Information

The present paper deals with problems of influence diagrams construction and its solution for decision support systems that function under uncertain and incomplete information available. The scheme of goal-directed generation of influence diagrams is considered. A method for using belief network algorithms to solve influence diagram problems is proposed This method is based on the transformation of an influence diagram into a belief network. An algorithm for hierarchical probabilistic inference is presented that is based on belief network representation in the form of a dependence tree according to which probabilities are determined in the order of their increase in the dependence tree level hierarchy. The suggested technique of influence diagram construction and analysis has been used for the task of logistic transport chain development.

Genetic distances and ordination: the land snail Helix aspersa in north Africa as a test case.

We examined the efficiencies of ordination methods in the treatment of gene frequency data at intraspecific level, using metric and nonmetric distance measures (Nei's and Rogers' genetic distances, chi 2 distance). We assessed initial processes responsible for the geographical distribution of the Mediterranean land snail Helix aspersa. Seventeen enzyme loci from 30 North African snail populations were considered in the present analysis. Five combinations of distance/multivariate analysis were compared: correspondence analysis (CA), nonmetric multidimensional scaling (NMDS) on Nei's, Rogers', and chi 2 distances, and principal coordinates analysis on Rogers' distances. Configuration of the objects resulting from ordination was projected onto three-dimensional graphics with the minimum spanning tree or the relative neighborhood graph superimposed. Pre- and postordination or clustering distance matrices were compared by means of correlation methods. As expected, all combinations led to a clear west versus east pattern of variation. However, the intraregional relationships and degree of connectivity between pairs of operational taxonomic units were not necessarily constant from one method to another. Ordination methods when applied with Nei's and Rogers' distances provided the best fit, with original distances (r = 0.98) compared with UPGMA clustering (r approximately 0.75). The Nei/NMDS combination seems to be a good compromise (distortion index dt = 10%) between Rogers/NMDS, which produces a more confusing pattern of differentiation (dt = 24%), and chi 2/CA, which tends to distort large distances (dt = 31%). NMDS obviously provides a powerful method to summarize relationships between populations, when neither hierarchical structure nor phylogenetic inference are required. These findings led the discussion on the good performance of NMDS, the appropriate distances to be used, and the potential application of this method to other types of allelic data (such as microsatellite loci) or data on nucleotide sequences of genes.

Identification of Nonlinear System Described by Functional Expansion on Basis of Object-Oriented Inference Engine

In the identification of system whose a priori information of structure is poor, adequate candidate models are difficultly set up. This paper proposes the system identification method in which the object-oriented inference engine may be applied to search for the optimal model. The candidate models are generated one by one by using the search tree, and are evaluated by Bayesian theorem in consideration of the reliability of the a posteriori probability. Various simulation show that the model of true structure is discriminated clearly from the others as data increase. The system described by a functional expansion may be identified, because the flexible search of models may be possible on the basis of hierarchical inference.