Multinomial Likelihood Research Articles

A Bayesian method for query approximation

This study presents statistical techniques to obtain local approximate query answers for aggregate multivariate materialized views thus eliminating the need for repetitive scanning of the source data. In widely distributed management information systems, detailed data do not necessarily reside in the same physical location as the decision-maker; thus, requiring scanning of the source data as needed by the query demand. Decision-making, business intelligence and data analysis could involve multiple data sources, data diversity, aggregates and large amounts of data. Management often confronts delays in information acquisition from remote sites. Management decisions usually involve analyses that require the most precise summary data available. These summaries are readily available from data warehouses and can be used to estimate or approximate data in exchange for a quicker response. An approach to supporting aggregate materialized view management is proposed that reconstructs data sets locally using posterior parameter estimates based on sufficient statistics in a log-linear model with a multinomial likelihood.

Dietary composition, plasticity, and prey selection of Pallas's cats

Dietary plasticity and diet composition of Pallas’s cat (Otocolobus manul) was quantified in central Mongolia. Diet of Pallas’s cat was assessed by scat analyses, and prey surveys were used to estimate prey availability. Prey selection was calculated using multinomial likelihood ratio tests. Analysis of 146 scats identified 249 prey items. Pallas’s cats ate a broad range of small mammals, insects, birds, reptiles, and carrion, but Daurian pikas (Ochotona dauurica) were the most frequently consumed prey. Multinomial likelihood ratio tests indicated nonrandom selection of prey species. Pikas were selected disproportionately to their availability, and other more numerous prey items were used less than expected, indicating feeding specialization. Specialization on pikas appears to optimize energy intake per unit foraging by Pallas’s cats, because pikas are 2–4 times larger than other small mammal prey. We argue that pika control programs in China and Mongolia potentially threaten Pallas’s cat populations, because no other prey species could fill the niche occupied by the pika in terms of size and year-round availability. DOI: 10.1644/09-MAMM-A-342.1.

Assessment of Substitution Model Adequacy Using Frequentist and Bayesian Methods

In order to have confidence in model-based phylogenetic methods, such as maximum likelihood (ML) and Bayesian analyses, one must use an appropriate model of molecular evolution identified using statistically rigorous criteria. Although model selection methods such as the likelihood ratio test and Akaike information criterion are widely used in the phylogenetic literature, model selection methods lack the ability to reject all models if they provide an inadequate fit to the data. There are two methods, however, that assess absolute model adequacy, the frequentist Goldman-Cox (GC) test and Bayesian posterior predictive simulations (PPSs), which are commonly used in conjunction with the multinomial log likelihood test statistic. In this study, we use empirical and simulated data to evaluate the adequacy of common substitution models using both frequentist and Bayesian methods and compare the results with those obtained with model selection methods. In addition, we investigate the relationship between model adequacy and performance in ML and Bayesian analyses in terms of topology, branch lengths, and bipartition support. We show that tests of model adequacy based on the multinomial likelihood often fail to reject simple substitution models, especially when the models incorporate among-site rate variation (ASRV), and normally fail to reject less complex models than those chosen by model selection methods. In addition, we find that PPSs often fail to reject simpler models than the GC test. Use of the simplest substitution models not rejected based on fit normally results in similar but divergent estimates of tree topology and branch lengths. In addition, use of the simplest adequate substitution models can affect estimates of bipartition support, although these differences are often small with the largest differences confined to poorly supported nodes. We also find that alternative assumptions about ASRV can affect tree topology, tree length, and bipartition support. Our results suggest that using the simplest substitution models not rejected based on fit may be a valid alternative to implementing more complex models identified by model selection methods. However, all common substitution models may fail to recover the correct topology and assign appropriate bipartition support if the true tree shape is difficult to estimate regardless of model adequacy.

A self-consistency approach to multinomial logit model with random effects

The computation in the multinomial logit mixed effects model is costly especially when the response variable has a large number of categories, since it involves high-dimensional integration and maximization. Tsodikov and Chefo (2008) developed a stable MLE approach to problems with independent observations, based on generalized self-consistency and quasi-EM algorithm developed in Tsodikov (2003). In this paper, we apply the idea to clustered multinomial response to simplify the maximization step. The method transforms the complex multinomial likelihood to Poisson-type likelihood and hence allows for the estimates to be obtained iteratively solving a set of independent low-dimensional problems. The methodology is applied to real data and studied by simulations. While maximization is simplified, numerical integration remains the dominant challenge to computational efficiency.

Skill Importance in Women's Volleyball

The purpose of this paper is to demonstrate two methods to quantify skill importance for teams in general, and women's volleyball in particular. A division I women's volleyball team rated each skill (serve, pass, set, etc.) and recorded rally outcomes during all home games in a competitive season. The skills were only rated when the ball was on the home team's side of the net. Events followed one of these three patterns: serve-outcome, pass-set-attack outcome, or block-dig-set-attack-outcome. These sequences of events were assumed to be first-order Markov chains, meaning the quality of the performance of the current skill only depended on the quality of the performance of the previous skill. We analyze the volleyball data using two different techniques: one uses a Markovian transition matrix, while the other is an implementation of logistic regression. To estimate the Markovian transition matrix, we assumed a multinomial likelihood with a Dirichlet prior on the transition probabilities. The logistic regression model also uses a Bayesian approach. The posterior distributions of parameters associated with skill performance are used to calculate importance scores. Importance scores produced by the two methods are reasonably consistent across skills. The importance scores indicate, among other things, that the team would have been well rewarded by improving transition offense. Importance scores can be used to assist coaches in allocating practice time, developing new strategies, and optimizing team performance relative to player selection.

Children’s asthma hospitalizations and relative risk due to nitrogen dioxide (NO2): Effect modification by race, ethnicity, and insurance status

This study explores the role of race, ethnicity, and insurance status in modifying the effects of air pollution on children's asthma hospitalizations in Phoenix, Arizona (US) between 2001 and 2003. While controlling for weather, interactions between nitrous dioxide (NO(2)) and race, ethnicity, and insurance status are used to predict relative risk for subgroups of children. The generalized logit regression model for nominal categorical data within a multinomial likelihood framework was used. This model is specifically suited to small counts and the reporting of 95% confidence intervals for the odds ratio of hospital admission for one group as compared to another. The odds ratio is known to approximate relative risk for rare events. Several significant findings were found for race, ethnicity, and insurance status as modulators for the effect of NO(2) on children's risk for asthma hospitalization: (1) children without insurance have 1.4 (95% CI: 1.1-1.8) times higher risk of asthma admissions than those with private insurance at exceedances of 0.02 parts per million (ppm) of NO(2) above the seasonal mean; the same finding holds for children without insurance as compared to those with Medicaid; (2) black children have 2.1 (95% CI: 1.3-3.3) times higher risk of hospitalization than Hispanic children at seasonal mean NO(2) levels, but this disproportionate risk shrinks to 1.7 with exceedances of 0.02 ppm of NO(2) above the seasonal mean. Specific to finding (1) among those children without health insurance, Hispanic children have 2.1 (95% CI: 1.1-3.8) times higher risk of hospitalization than white children. Among all Hispanic children, those without health insurance have 1.9 (95% CI: 1.3-3.0) times greater risk than those with private insurance; the same finding holds for Hispanic children without insurance as compared to Hispanic children with Medicaid. Specific to finding (2), among children with private insurance, the disproportionate risk of black children as compared to Hispanic children is magnified by a factor of 1.3 (95% CI: 1.0-1.8) for exceedances of 0.02 ppm of NO(2) above the seasonal mean. Although we cannot confirm a cause-effect relationship, this analysis suggests that increasing insurance enrollment for all children, and specifically Hispanic children, may reduce their disproportionate risk from exceedances of air pollution. There are few black children in Phoenix, so further studies are needed to investigate the increasing risk of black children with private insurance as compared to Hispanics at exceedances of NO(2).

Data augmentation, frequentist estimation, and the Bayesian analysis of multinomial logit models

This article describes a convenient method of selecting Metropolis– Hastings proposal distributions for multinomial logit models. There are two key ideas involved. The first is that multinomial logit models have a latent variable representation similar to that exploited by Albert and Chib (J Am Stat Assoc 88:669–679, 1993) for probit regression. Augmenting the latent variables replaces the multinomial logit likelihood function with the complete data likelihood for a linear model with extreme value errors. While no conjugate prior is available for this model, a least squares estimate of the parameters is easily obtained. The asymptotic sampling distribution of the least squares estimate is Gaussian with known variance. The second key idea in this paper is to generate a Metropolis–Hastings proposal distribution by conditioning on the estimator instead of the full data set. The resulting sampler has many of the benefits of so-called tailored or approximation Metropolis–Hastings samplers. However, because the proposal distributions are available in closed form they can be implemented without numerical methods for exploring the posterior distribution. The algorithm converges geometrically ergodically, its computational burden is minor, and it requires minimal user input. Improvements to the sampler’s mixing rate are investigated. The algorithm is also applied to partial credit models describing ordinal item response data from the 1998 National Assessment of Educational Progress. Its application to hierarchical models and Poisson regression are briefly discussed.

A Likelihood Model That Accounts for Censoring Due to Fetal Loss Can Accurately Test the Effects of Maternal and Fetal Genotype on the Probability of Miscarriage

Objective: Heritable maternal and fetal thrombophilia and/or hypofibrinolysis are important causes of miscarriage. Under the constraint that fetal genotype is observed only after a live birth, estimating risk is complicated. Censoring prevents use of published statistical methodology. We propose techniques to determine whether increases in miscarriage are due to the fetal genotype, maternal genotype, or both. Methods: We propose a study to estimate the risk of miscarriage contributed by an allele, expressed in either dominant or recessive fashion. Using a multinomial likelihood, we derive maximum likelihood estimates of risk for different genotype groups. We describe likelihood ratio tests and a planned hypothesis testing strategy. Results: Parameter estimation is accurate (bias <0.0011, root mean squared error <0.0780, n = 500). We used simulation to estimate power for studies of three gene mutations: the 4G hypofibrinolytic mutation in the plasminogen activator inhibitor gene (PAI-1), the prothrombin G20210A mutation, and the Factor V Leiden mutation. With 500 families, our methods have approximately 90% power to detect an increase in the miscarriage rate of 0.2, above a background rate of 0.2. Conclusion: Our statistical method can determine whether increases in miscarriage are due to fetal genotype, maternal genotype, or both despite censoring.

Pattern recognition with a Bayesian kernel combination machine

In this paper, we describe a Bayesian classification method that informatively combines diverse sources of information and multiple feature spaces for multiclass problems. The proposed method is based on recent advances in kernel approaches where the integration of multiple object descriptors, or feature spaces, is achieved via kernel combination. Each kernel constructs a similarity metric between objects in a particular feature space and then having a common metric across modalities an overall combination can be constructed. We follow a hierarchical Bayesian approach, which introduces prior distributions over random variables and we construct a Gibbs sampling Markov chain Monte Carlo (MCMC) solution which is naturally derived from the employed multinomial probit likelihood. The methodology is the basis for possible deterministic approximations such as variational or maximum-a-posteriori estimators, and it is compared against the well-known classifier combination methods on the classification of handwritten numerals. The results of the proposed method show a significant improvement over the best individual classifier and match the performance of the best multiple classifier combination, whilst reducing the computational requirements of combining classifiers and offering additional information on the significance of the contributing sources.

Generalized self-consistency: Multinomial logit model and Poisson likelihood

A generalized self-consistency approach to maximum likelihood estimation (MLE) and model building was developed in Tsodikov [2003. Semiparametric models: a generalized self-consistency approach. J. Roy. Statist. Soc. Ser. B Statist. Methodology 65(3), 759–774] and applied to a survival analysis problem. We extend the framework to obtain second-order results such as information matrix and properties of the variance. Multinomial model motivates the paper and is used throughout as an example. Computational challenges with the multinomial likelihood motivated Baker [1994. The Multinomial–Poisson transformation. The Statist. 43, 495–504] to develop the Multinomial–Poisson (MP) transformation for a large variety of regression models with multinomial likelihood kernel. Multinomial regression is transformed into a Poisson regression at the cost of augmenting model parameters and restricting the problem to discrete covariates. Imposing normalization restrictions by means of Lagrange multipliers [Lang, J., 1996. On the comparison of multinomial and Poisson log-linear models. J. Roy. Statist. Soc. Ser. B Statist. Methodology 58, 253–266] justifies the approach. Using the self-consistency framework we develop an alternative solution to multinomial model fitting that does not require augmenting parameters while allowing for a Poisson likelihood and arbitrary covariate structures. Normalization restrictions are imposed by averaging over artificial “missing data” (fake mixture). Lack of probabilistic interpretation at the “complete-data” level makes the use of the generalized self-consistency machinery essential.

A note on multinomial maximum likelihood estimation under ordered restrictions and the EM algorithm

A frequently occurring problem is to find the maximum likelihood estimation (MLE) of p subject to p∈C (C⊂ P the probability vectors in Rk). The problem has been discussed by many authors and they mainly focused when p is restricted by linear constraints or log-linear constraints. In this paper, we construct the relationship between the the maximum likelihood estimation of p restricted by p∈C and EM algorithm and demonstrate that the maximum likelihood estimator can be computed through the EM algorithm (Dempster et al. in J R Stat Soc Ser B 39:1–38, 1997). Several examples are analyzed by the proposed method.

Artificial neural network for the joint modelling of discrete cause-specific hazards

Artificial neural network (ANN) based regression methods have been introduced for modelling censored survival data to account for complex prognostic patterns. In the framework of ANN extensions of generalized linear models for survival data, PLANN is a partial logistic ANN, suitable for smoothed discrete hazard estimation as a function of time and covariates. An extension of PLANN for competing risks analysis (PLANNCR) is now proposed for discrete or grouped survival times, resorting to the multinomial likelihood. PLANNCR is built by assigning input nodes to the explanatory variables with the time interval treated as an ordinal variable. The logistic function is used as activation for the hidden nodes of the network, whereas the softmax, which corresponds to the canonical link of generalized linear models for polytomous regression, is adopted for multiple output nodes, to provide a smoothed estimation of discrete conditional event probabilities for each event. The Kullback-Leibler distance is used as error function for the target vectors, amounting to half of the deviance of a multinomial logistic regression model. PLANNCR can jointly model non-linear, non-proportional and non-additive effects on cause-specific hazards (CSHs). The degree of smoothing is modulated by the number of hidden nodes and penalization of the error function (weight decay). Model optimisation is achieved by quasi-Newton algorithms, while non-linear cross-validation (NCV) and the Network Information Criterion (NIC) were adopted for model selection. PLANNCR was applied to data on 1793 women with primary invasive breast cancer, histologically N-, who underwent surgery at the Milan Cancer Institute between 1981 and 1986. Differential effects of covariates and time on the shape of the CSH for the three main failure causes, namely intra-breast tumor recurrences, distant metastases and contralateral breast cancer, have been enlightened. PLANNCR can be suitably adopted in an exploratory framework for a thorough evaluation of the disease dynamics in the presence of competing risks.

A new likelihood for simultaneously estimating von Bertalanffy growth parameters, gear selectivity, and natural and fishing mortality

Gear selectivity and the cumulative effects of size-selective fishing produce bias in the length-at-age samples used to estimate the von Bertalanffy growth parameters. In fished populations, fast-growing young fish and slow-growing old fish are overrepresented in size–age samples. To account for such effects, we treated size-at-age observations as multinomial samples, with expected catches in each size–age category dependent on growth parameters, growth variation, size selectivity, abundance at age, and the history of exploitation. Using simulated data sets, estimated growth parameters using the multinomial likelihood were unbiased when fishing mortality was not too high and the shape of the vulnerability function was correct. In contrast, estimated growth parameters using a least squares approach overestimated the metabolic growth coefficient (K) and underestimated mean asymptotic length (L∞). Models that do not explicitly account for the effects of fishing and size selectivity underestimated L∞ and overestimated K. We estimate growth parameters for northern pikeminnow (Ptychocheilus oregonensis) as an example of the method and document a stunted "pigmy" population with an L∞ of 175-mm fork length, attributing its small size to effects of high density and (or) a short growing season.

Missing data in the 2 x 2 table: patterns and likelihood-based analysis for cross-sectional studies with supplemental sampling.

Standard measures of crude association in the context of a cross-sectional study are the risk difference, relative risk and odds ratio as derived from a 2x 2 table. Most such studies are subject to missing data on disease, exposure, or both, introducing bias into the usual complete-case analysis. We describe several scenarios distinguished by the manner in which missing data arise, and for each we adjust the natural multinomial likelihood to properly account for missing data. The situations presented allow for increasing levels of generality with regard to the missing data mechanism. The final case, quite conceivable in epidemiologic studies, assumes that the probability of missing exposure depends on true exposure and disease status, as well as upon whether disease status is missing (and conversely for the probability of missing disease information). When parameters relating to the missing data process are inestimable without strong assumptions, we propose maximum likelihood analysis subsequent to collecting supplemental data in the spirit of a validation study. Analytical results give insight into the bias inherent in complete-case analysis for each scenario, and numerical results illustrate the performance of likelihood-based point and interval estimates in the most general case. Adjustment for potential confounders via stratified analysis is also discussed.

Simultaneous estimation of intrarater and interrater agreement for multiple raters under order restrictions for a binary trait.

It is valuable in many studies to assess both intrarater and interrater agreement. Most measures of intrarater agreement do not adjust for unequal estimates of prevalence between the separate rating occasions for a given rater and measures of interrater agreement typically ignore data from the second set of assessments when raters make duplicate assessments. In the event when both measures are assessed there are instances where interrater agreement is larger than at least one of the corresponding intrarater agreements. This implies that a rater agrees less with him/herself and more with another rater. In the situation of multiple raters making duplicate assessments on all subjects, the authors propose properties for an agreement measure based on the odds ratio for a dichotomous trait: (i) estimate a single prevalence across two reading occasions for each rater; (ii) estimate pairwise interrater agreement from all available data; (iii) bound the pairwise interrater agreement above by the corresponding intrarater agreements. Estimation of odds ratios under these properties is done by maximizing the multinomial likelihood with constraints using generalized log-linear models in combination with a generalization of the Lemke-Dykstra iterative-incremental algorithm. An example from a mammography examination reliability study is used to demonstrate the new method.

A general model for tagging on multiple component fisheries: an integration of age-dependent reporting rates and mortality estimation

A major objective of analyzing multiple year tag return data in fisheries is to estimate fishing and natural mortality rates which may vary by age class and calendar year. To do this one needs to be able to estimate the reporting rates for the tags recovered. Some fisheries such as that for Southern Bluefin Tuna (Thunnus maccoyii) have multiple components with potentially different reporting rates for the tag returns. In this paper we develop a general model for multi-cohort, multi-year tag return analyses where there are multiple components to the fishery with potentially different reporting rates. We require the assumption that one component has a reporting rate of 100% (i.e., this could be the component of a boat based fishery where scientific observers are present). We show further how it is possible to partition the overall likelihood developed into two conditionally independent components. The first component of the likelihood is the standard multinomial likelihood that allows estimation of fishing and natural mortality rates. It uses the tag return matrix summed over all the components of the fishery. It requires an average reporting rate for the tag returns (where the average reporting rate is a weighted average of the individual reporting rates of the different components). The second component is also multinomial for the individual component tag returns and allows us to estimate individual component reporting rates. However, this requires that we augment our second component tag return likelihood with a catch data likelihood for the corresponding components. The methodology is illustrated on some Southern Bluefin Tuna tagging and catch data. We also discuss important model assumptions and give suggestions for future research including the integration of tag-return and catch at age data analyses.

Testing for elevated disease rates using smoothed estimates.

A method of testing for elevated disease rates using smoothed estimates is introduced for counts aggregated to a specific administrative level. A test statistic is developed based on an approximation to the smoothed estimate from a penalized multinomial likelihood and an approximate critical value is derived. Empirical investigation reveals that smoothing can increase the ability to detect elevated rates. The methods are applied to data on gastric cancer in Nova Scotia, Canada.

A Penalized Multinomial Approach to Smoothed Estimation of Disease Incidence

Raw estimates of disease rates over a geographical region are frequently quite variable, even though one may reasonably expect adjacent communities to have similar true rates. Smoother estimates are obtained by incorporating a penalty into a multinomial likelihood estimation procedure. For each pair of locations, this penalty increases with the difference between the rates and decreases with the distance between the two sites. The resulting estimates have smaller mean squared error than the raw estimates. Expansions are developed which demonstrate the contributions of the smoothing constant, spatial configuration, risk population and raw estimates to the amount of smoothing. Simulations and an example involving gastric cancer data illustrate the proposed method.

Additive logistic regression: a statistical view of boosting (With discussion and a rejoinder by the authors)

Boosting is one of the most important recent developments in classification methodology. Boosting works by sequentially applying a classification algorithm to reweighted versions of the training data and then taking a weighted majority vote of the sequence of classifiers thus produced. For many classification algorithms, this simple strategy results in dramatic improvements in performance. We show that this seemingly mysterious phenomenon can be understood in terms of well-known statistical principles, namely additive modeling and maximum likelihood. For the two-class problem, boosting can be viewed as an approximation to additive modeling on the logistic scale using maximum Bernoulli likelihood as a criterion. We develop more direct approximations and show that they exhibit nearly identical results to boosting. Direct multiclass generalizations based on multinomial likelihood are derived that exhibit performance comparable to other recently proposed multiclass generalizations of boosting in most situations, and far superior in some. We suggest a minor modification to boosting that can reduce computation, often by factors of 10 to 50. Finally, we apply these insights to produce an alternative formulation of boosting decision trees. This approach, based on best-first truncated tree induction, often leads to better performance, and can provide interpretable descriptions of the aggregate decision rule. It is also much faster computationally, making it more suitable to large-scale data mining applications.

Estimation of weighted multinomial probabilities under log-convex constraints

Many problems in Statistics involve maximizing a multinomial likelihood over a restricted region. In this paper, we consider instead maximizing a weighted multinomial likelihood. We show that a dual problem always exits which is frequently more tractable and that a solution to the dual problem leads directly to a solution of the primal problem. Moreover, the form of the dual problem suggests an iterative algorithm for solving the MLE problem when the constraint region can be written as a finite intersection of cones. We show that this iterative algorithm is guaranteed to converge to the true solution and show that when the cones are isotonic, this algorithm is a version of Dykstra's algorithm (Dykstra, J. Amer. Statist. Assoc. 78 (1983) 837–842) for the special case of least squares projection onto the intersection of isotonic cones. We give several meaningful examples to illustrate our results. In particular, we obtain the nonparametric maximum likelihood estimator of a monotone density function in the presence of selection bias.