Higher-order Inference Research Articles

Anisotropic span embeddings and the negative impact of higher-order inference for coreference resolution: An empirical analysis

Abstract Coreference resolution is the task of identifying and clustering mentions that refer to the same entity in a document. Based on state-of-the-art deep learning approaches, end-to-end coreference resolution considers all spans as candidate mentions and tackles mention detection and coreference resolution simultaneously. Recently, researchers have attempted to incorporate document-level context using higher-order inference (HOI) to improve end-to-end coreference resolution. However, HOI methods have been shown to have marginal or even negative impact on coreference resolution. In this paper, we reveal the reasons for the negative impact of HOI coreference resolution. Contextualized representations (e.g., those produced by BERT) for building span embeddings have been shown to be highly anisotropic. We show that HOI actually increases and thus worsens the anisotropy of span embeddings and makes it difficult to distinguish between related but distinct entities (e.g., pilots and flight attendants). Instead of using HOI, we propose two methods, Less-Anisotropic Internal Representations (LAIR) and Data Augmentation with Document Synthesis and Mention Swap (DSMS), to learn less-anisotropic span embeddings for coreference resolution. LAIR uses a linear aggregation of the first layer and the topmost layer of contextualized embeddings. DSMS generates more diversified examples of related but distinct entities by synthesizing documents and by mention swapping. Our experiments show that less-anisotropic span embeddings improve the performance significantly (+2.8 F1 gain on the OntoNotes benchmark) reaching new state-of-the-art performance on the GAP dataset.

Coverage error optimal confidence intervals for local polynomial regression

This paper studies higher-order inference properties of nonparametric local polynomial regression methods under random sampling. We prove Edgeworth expansions for t statistics and coverage error expansions for interval estimators that (i) hold uniformly in the data generating process, (ii) allow for the uniform kernel, and (iii) cover estimation of derivatives of the regression function. The terms of the higher-order expansions, and their associated rates as a function of the sample size and bandwidth sequence, depend on the smoothness of the population regression function, the smoothness exploited by the inference procedure, and on whether the evaluation point is in the interior or on the boundary of the support. We prove that robust bias corrected confidence intervals have the fastest coverage error decay rates in all cases, and we use our results to deliver novel, inference-optimal bandwidth selectors. The main methodological results are implemented in companion R and Stata software packages.

Viewing images of foods evokes taste quality-specific activity in gustatory insular cortex

Previous studies have shown that the conceptual representation of food involves brain regions associated with taste perception. The specificity of this response, however, is unknown. Does viewing pictures of food produce a general, nonspecific response in taste-sensitive regions of the brain? Or is the response specific for how a particular food tastes? Building on recent findings that specific tastes can be decoded from taste-sensitive regions of insular cortex, we asked whether viewing pictures of foods associated with a specific taste (e.g., sweet, salty, and sour) can also be decoded from these same regions, and if so, are the patterns of neural activity elicited by the pictures and their associated tastes similar? Using ultrahigh-resolution functional magnetic resonance imaging at high magnetic field strength (7-Tesla), we were able to decode specific tastes delivered during scanning, as well as the specific taste category associated with food pictures within the dorsal mid-insula, a primary taste responsive region of brain. Thus, merely viewing food pictures triggers an automatic retrieval of specific taste quality information associated with the depicted foods, within gustatory cortex. However, the patterns of activity elicited by pictures and their associated tastes were unrelated, thus suggesting a clear neural distinction between inferred and directly experienced sensory events. These data show how higher-order inferences derived from stimuli in one modality (i.e., vision) can be represented in brain regions typically thought to represent only low-level information about a different modality (i.e., taste).

Motor versus body awareness: Voxel-based lesion analysis in anosognosia for hemiplegia and somatoparaphrenia following right hemisphere stroke

Anosognosia for hemiplegia (AHP) is informative about the neurocognitive basis of motor awareness. However, it is frequently associated with concomitant symptoms, such as hemispatial neglect and disturbances in the sense of body ownership (DSO). Although double dissociations between these symptoms have been reported, there is ongoing debate about whether they are manifestations of independent abnormalities, or a single neurocognitive deficit. We aimed to investigate the specificity of lesions associated with AHP by surpassing four, existing methodological limitations: (a) recruit a relatively large sample of patients (total N = 70) in a multi-centre study; (b) identify lesions associated with AHP in grey and white matter using voxel-based methods; (c) take into account the duration of AHP and concomitant neglect symptoms; and (d) compare lesions against a control hemiplegic group, patients suffering from AHP and DSO, and a few, rare patients with selective DSO. Results indicated that acute AHP is associated with a wide network, mainly including: (1) the Rolandic operculum, (2) the insula and (3) the superior temporal gyri. Subcortically, damage mainly involved the basal ganglia and white matter, mostly the superior corona radiate, arcuate fasciculus and the part of the ventral, superior longitudinal fasciculus. Persistent symptoms were linked with wider damage involving fronto-temporal cortex and long white matter tracts. A shift in the latero-medial direction (mainly involving the basal ganglia and surrounding white matter) emerged when DSO was taken accounted for. These results suggest that while bodily awareness is processed by areas widely distributed across the brain, intact subcortical structures and white matter tracts may be necessary to support basic feelings of owning and controlling contralateral body parts. An accurate and ‘up-to-date’ awareness of our motor abilities, however, may rely also on intact processing in cortical areas which presumably allow higher-order inferences about the current state of the body.

Higher order inference for stress–strength reliability with independent Burr-type X distributions

In this paper, a small-sample asymptotic method is proposed for higher order inference in the stress–strength reliability model, R=P(Y<X), where X and Y are distributed independently as Burr-type X distributions. In a departure from the current literature, we allow the scale parameters of the two distributions to differ, and the likelihood-based third-order inference procedure is applied to obtain inference for R. The difficulty of the implementation of the method is in obtaining the the constrained maximum likelihood estimates (MLE). A penalized likelihood method is proposed to handle the numerical complications of maximizing the constrained likelihood model. The proposed procedures are illustrated using a sample of carbon fibre strength data. Our results from simulation studies comparing the coverage probabilities of the proposed small-sample asymptotic method with some existing large-sample asymptotic methods show that the proposed method is very accurate even when the sample sizes are small.

Accurate Directional Inference for Vector Parameters in Linear Exponential Families

We consider inference on a vector-valued parameter of interest in a linear exponential family, in the presence of a finite-dimensional nuisance parameter. Based on higher-order asymptotic theory for likelihood, we propose a directional test whose p-value is computed using one-dimensional integration. The work simplifies and develops earlier research on directional tests for continuous models and on higher-order inference for discrete models, and the examples include contingency tables and logistic regression. Examples and simulations illustrate the high accuracy of the method, which we compare with the usual likelihood ratio test and with an adjusted version due to Skovgaard. In high-dimensional settings, such as covariance selection, the approach works essentially perfectly, whereas its competitors can fail catastrophically.

Higher order inference on a treatment effect under low regularity conditions

We describe a novel approach to nonparametric point and interval estimation of a treatment effect in the presence of many continuous confounders. We show that the problem can be reduced to that of point and interval estimation of the expected conditional covariance between treatment and response given the confounders. Our estimators are higher order U-statistics. The approach applies equally to the regular case where the expected conditional covariance is root-n estimable and to the irregular case where slower nonparametric rates prevail.

Higher order inference for the consensus mean in inter-laboratory studies

In inter-laboratory studies, a fundamental problem of interest is inference concerning the consensus mean, when the measurements are made by several laboratories which may exhibit different within-laboratory variances, apart from the between laboratory variability. A heteroscedastic one-way random model is very often used to model this scenario. Under such a model, a modified signed log-likelihood ratio procedure is developed for the interval estimation of the common mean. Furthermore, simulation results are presented to show the accuracy of the proposed confidence interval, especially for small samples. The results are illustrated using an example on the determination of selenium in non-fat milk powder by combining the results of four methods. Here, the sample size is small, and the confidence limits for the common mean obtained by different methods produce very different results. The confidence interval based on the modified signed log-likelihood ratio procedure appears to be quite satisfactory.

The second order ancillary is rotation based

This paper concerns the approximate ancillary needed for higher order asymptotic likelihood inference. The existence of an exact or approximate ancillary is the key to such inference. The exact ancillary is, however, only available for transformation model and an approximate ancillary is typically very difficult to find. Fraser (1988) revealed that the high order inference can be obtained by using only the tangent directions to the second order ancillary, thus avoiding its specification. The present work discuses the second order ancillary that has the observed contour tangent to the sensitivity directions v = ∂ y / ∂ θ | { y 0 ; θ ^ ( y 0 ) } corresponding to a pivot and derived by keeping the pivotal variable fixed at the observed fitted value; these directions are always available ( Fraser and Reid, 1995, 2001). For scalar parameter case, such an approximate ancillary is well defined and can be described as a rotation. Our approach provides insights into the second order ancillary outlined by the sensitivity directions in the vector parameter case.

Conditional properties of unconditional parametric bootstrap procedures for inference in exponential families

Higher-order inference about a scalar parameter in the presence of nuisance parameters can be achieved by bootstrapping, in circumstances where the parameter of interest is a component of the canonical parameter in a full exponential family. The optimal test, which is approximated, is a conditional one based on conditioning on the sufficient statistic for the nuisance parameter. A bootstrap procedure that ignores the conditioning is shown to have desirable conditional properties in providing third-order relative accuracy in approximation of p-values associated with the optimal test, in both continuous and discrete models. The bootstrap approach is equivalent to third-order analytical approaches, and is demonstrated in a number of examples to give very accurate approximations even for very small sample sizes.

A Meta-Analysis on the Correlation Between the Implicit Association Test and Explicit Self-Report Measures

Theoretically, low correlations between implicit and explicit measures can be due to (a) motivational biases in explicit self reports, (b) lack of introspective access to implicitly assessed representations, (c) factors influencing the retrieval of information from memory, (d) method-related characteristics of the two measures, or (e) complete independence of the underlying constructs. The present study addressed these questions from a meta-analytic perspective, investigating the correlation between the Implicit Association Test (IAT) and explicit self-report measures. Based on a sample of 126 studies, the mean effect size was .24, with approximately half of the variability across correlations attributable to moderator variables. Correlations systematically increased as a function of (a) increasing spontaneity of self-reports and (b) increasing conceptual correspondence between measures. These results suggest that implicit and explicit measures are generally related but that higher order inferences and lack of conceptual correspondence can reduce the influence of automatic associations on explicit self-reports.

Implementation of higher-order asymptotics to S-plus

Higher-order asymptotics is an active area of development in theoretical statistics. However, most existing work in higher-order asymptotics is directed to the theoretical aspects. This paper attempts to incorporate higher-order inference procedures to S-plus, a widely used software in statistics. Algorithm is developed in the settings of generalized linear models and nonlinear regression models. The proposed algorithm generalizes the standard S-plus functions “glim” and “nls” in the sense that both the first-order and higher-order p-values are provided, and its manipulation is straightforward.

Hemispheric differences in simple reaction time to auditory stimuli of different frequencies.

To determine whether reaction time varies as a function of tone frequency and/or ear stimulated 27 adult subjects were presented with two two-tone series (1,000 Hz vs 2,500 Hz and 1,500 Hz vs 4,000 Hz) in a simple reaction-time paradigm. The analyses clearly indicated that the higher tone in each series and stimulation to the right ear resulted in significantly shorter reaction times. These results were interpreted as indicating that cerebral asymmetries are apparent in tasks and with stimuli that do not seem to require higher-order inferences regrading the functional organization of the cerebral hemispheres.