Incremental Interpretation Research Articles

A deep neural network with modified random forest incremental interpretation approach for diagnosing diabetes in smart healthcare

Artificial intelligence (AI) applications based on deep learning for diagnosing type-II diabetes are sometimes difficult to understand and communicate even as patients are eager to understand the rationale behind the diagnostic results. Accordingly, recent studies have used multiple simple rules to adequately explain the diagnostic process and results to patients. However, this can cause patient confusion as the rules vary. Hence, this study proposes a deep neural network (DNN) with random forest (RF) and modified random forest incremental interpretation (MRFII) approach for diagnosing diabetes. This method first entails constructing a DNN to predict the probability of a patient having diabetes. To make the prediction result explainable, an RF is built to explain the process and results in terms of multiple simple decision rules. Additionally, to eliminate patient confusion, the MRFII is proposed to sort and aggregate the decision rules for a specific patient. A certainty mechanism is also established to feed back the explanation results from RF to improve the effectiveness of the DNN. The proposed method was applied to a diabetes dataset from the National Institute of Diabetes and Digestive and Kidney Diseases, and the results showed that this approach provided a more concise and accurate explanation than existing explainable artificial intelligence (XAI) techniques for the same purpose.

A modified random forest incremental interpretation method for explaining artificial and deep neural networks in cycle time prediction

Methods based on artificial neural network (ANN) or deep neural network (DNN) applications have been proposed to predict job cycle time effectively. However, the predicting mechanism of ANNs (or DNNs) is often difficult to understand and communicate. This problem has hindered their acceptability (and practicability). Furthermore, existing explainable artificial intelligence (XAI) techniques use simpler decision rules to approximate the prediction process and/or outcomes of ANNs (or DNNs). However, these decision rules may not conform to domain knowledge, which limits their usefulness. This study proposes a modified random forest incremental interpretation (RFII) approach to address this issue. In the proposed methodology, the input–output relationship of an ANN (or DNN) is fitted by decision rules using a random forest approach. Then, domain knowledge is extracted, and decision rules are checked against this knowledge. Decision rules with poor conformance to domain knowledge are excluded. Subsequently, the remaining decision rules are aggregated in a way that is convenient for user interpretation. The modified RFII method has been applied to a real case in the literature to demonstrate its applicability.

Preschoolers modulate contrastive inferences during online language comprehension.

This study examined 4- and 5-year-olds' incremental interpretation of size adjectives, focusing on whether contrastive inferences are modulated by speaker behavior. Children (N=120, 59 females, mostly White, tested between July, 2018 and August, 2019) encountered either a conventional or unconventional speaker who labeled objects in a correspondingly typical or atypical way. Critical utterances contained size adjectives (e.g., "Look at the big duck"). With conventional speakers, gaze measures indicated that children rapidly used the adjective to differentiate members of a contrasting pair, indicating that even 4-year-olds derive contrastive inferences. With unconventional speakers, contrastive inferences were delayed in processing. The findings demonstrate that preschoolers adjust their use of pragmatic cues when presented with evidence disconfirming their default assumptions about a speaker.

Anticipatory Processing in a Verb-Initial Mayan Language: Eye-Tracking Evidence During Sentence Comprehension in Tseltal.

We present a visual world eye-tracking study on Tseltal (a Mayan language) and investigate whether verbal information can be used to anticipate an upcoming referent. Basic word order in transitive sentences in Tseltal is Verb-Object-Subject (VOS). The verb is usually encountered first, making argument structure and syntactic information available at the outset, which should facilitate anticipation of the post-verbal arguments. Tseltal speakers listened to verb-initial sentences with either an object-predictive verb (e.g., "eat") or a general verb (e.g., "look for") (e.g., "Ya slo'/sle ta stukel on te kereme," Is eating/is looking (for) by himself the avocado the boy/ "The boy is eating/is looking (for) an avocado by himself") while seeing a visual display showing one potential referent (e.g., avocado) and three distractors (e.g., bag, toy car, coffee grinder). We manipulated verb type (predictive vs. general) and recorded participants' eye movements while they listened and inspected the visual scene. Participants' fixations to the target referent were analyzed using multilevel logistic regression models. Shortly after hearing the predictive verb, participants fixated the target object before it was mentioned. In contrast, when the verb was general, fixations to the target only started to increase once the object was heard. Our results suggest that Tseltal hearers pre-activate semantic features of the grammatical object prior to its linguistic expression. This provides evidence from a verb-initial language for online incremental semantic interpretation and anticipatory processing during language comprehension. These processes are comparable to the ones identified in subject-initial languages, which is consistent with the notion that different languages follow similar universal processing principles.

Incremental pragmatic interpretation of gradable adjectives: The role of standards of comparison

While for relative gradable adjectives the value on the underlying measurement scale that serves as a standard of comparison is contextually determined, for absolute gradable adjectives this is typically taken to be a fixed, context-invariant value (Rotstein & Winter 2004; Kennedy & McNally 2005). The present study investigates how lexical-semantic factors, such as the type of standard of comparison invoked by gradable adjectives, affect the incremental computation of scalar implicatures triggered by such adjectives. Our study shows that the incremental computation of scalar implicatures is facilitated by the immediate visual context but only for relative adjectives. Minimum standard absolute adjectives, which impose a lower bound on their corresponding measurement scales, robustly trigger upper-bounded interpretations independently of the availability of contrastive visual information. Our findings indicate that different kinds of scalar meaning are computed incrementally and potentially in parallel. Overall, these findings shed new light on theories of scalar implicatures and highlight the need for a model of adjective meaning that incorporates semantic and pragmatic factors (see also Gotzner 2021; and for related ideas in the domain of quantifiers see Franke & Bergen 2020 and Cremers, Wilcox & Spector 2022, and Magri (2017) for Hirschberg scales).

Why bother? What our eyes tell about psych verb (non) causative constructions

We present an eyetracking study that investigates how linking is achieved during real-time comprehension of Spanish sentences with causative psych verbs and alternative case marking. This group of verbs lead to verbs’ argument structures that require direct or inverse syntax-to-semantics linking according to the type of case marking assigned to their object. The study aimed at disentangling whether processing inverse linking was more costly than direct linking, and exploring how incremental argument interpretation takes place when lexemes that accept several case markings are used. Results showed that during incremental comprehension, inverse linking is more difficult than direct linking, irrespective of word order. As for argument interpretation, the current study partially replicated the results of previous studies conducted in this language using different verb types. Findings are discussed under the light of different psycholinguistic models addressing case marking processing and incremental linking.

Person-based prominence guides incremental interpretation: Evidence from obviation in Ojibwe

Distinctions related to person and animacy have long been known to impact both the grammar and incremental processing in a way that can be described through “prominence” scales. We put the generalizability of these scales to the test by examining the processing effects of a typologically uncommon distinction known as obviation, which is found in Ojibwe, an Indigenous language of North America. Obviation contrasts the single most discourse-salient animate third person (proximate) with other non-salient third persons (obviative). Using a visual world paradigm, we show that obviation influences parsing and interpretation commitments under incremental ambiguity: Proximate nouns are assumed to be the agent of an action, while obviative nouns do not lead to strong incremental commitments. This result parallels previous findings in other languages with distinctions related to animacy and person, supporting a theory where the effect of prominence information in processing is the result of a common set of constraints derived from the alignment of scales related to person, syntactic position, and thematic role.

Temporary ambiguity and memory for the context of spoken language.

Spoken language is interpreted incrementally, with listeners considering multiple candidate meanings as words unfold over time. Due to incremental interpretation, when a speaker refers to something in the world, there is often temporary ambiguity regarding which of several candidate items in the referential context the speaker is referring to. Subsequent tests of recognition memory show that listeners have good memory for referenced items, but that listeners also sometimes recognize non-referenced items from the referential context that share features with items that were mentioned. Predicted or inferred (but not experienced) interpretations of what was said are also sometimes retained in memory. While these findings indicate that multiple items from the referential context may be encoded in memory, the mechanisms supporting memory for the context of language use remain poorly understood. This paper tests the hypothesis that a consequence of temporary ambiguity in spoken language is enhanced memory for the items in the referential context. Two experiments demonstrate that periods of temporary referential ambiguity boost memory for non-referenced items in the referential context. Items that temporarily matched the unfolding referring expression were better remembered than those that did not. The longer the period of ambiguity, the stronger the memory boost, particularly for items activated early in the expression. In sum, the fact that spoken language unfolds over time creates momentary ambiguity about the speaker's intention; this ambiguity, in turn, allows the listener to later remember not only what the speaker did say, but also what they could have, but did not.

A Rational Model of Incremental Argument Interpretation: The Comprehension of Swedish Transitive Clauses.

A central component of sentence understanding is verb-argument interpretation, determining how the referents in the sentence are related to the events or states expressed by the verb. Previous work has found that comprehenders change their argument interpretations incrementally as the sentence unfolds, based on morphosyntactic (e.g., case, agreement), lexico-semantic (e.g., animacy, verb-argument fit), and discourse cues (e.g., givenness). However, it is still unknown whether these cues have a privileged role in language processing, or whether their effects on argument interpretation originate in implicit expectations based on the joint distribution of these cues with argument assignments experienced in previous language input. We compare the former, linguistic account against the latter, expectation-based account, using data from production and comprehension of transitive clauses in Swedish. Based on a large corpus of Swedish, we develop a rational (Bayesian) model of incremental argument interpretation. This model predicts the processing difficulty experienced at different points in the sentence as a function of the Bayesian surprise associated with changes in expectations over possible argument interpretations. We then test the model against reading times from a self-paced reading experiment on Swedish. We find Bayesian surprise to be a significant predictor of reading times, complementing effects of word surprisal. Bayesian surprise also captures the qualitative effects of morpho-syntactic and lexico-semantic cues. Additional model comparisons find that it—with a single degree of freedom—captures much, if not all, of the effects associated with these cues. This suggests that the effects of form- and meaning-based cues to argument interpretation are mediated through expectation-based processing.

Ladies First: Gender Stereotypes Drive Anticipatory Eye-Movements During Incremental Sentence Interpretation.

Immediate contextual information and world knowledge allow comprehenders to anticipate incoming language in real time. The cognitive mechanisms that underlie such behavior are, however, still only partially understood. We examined the novel idea that gender attitudes may influence how people make predictions during sentence processing. To this end, we conducted an eye-tracking experiment where participants listened to passive-voice sentences expressing gender-stereotypical actions (e.g., “The wood is being painted by the florist”) while observing displays containing both female and male characters representing gender-stereotypical professions (e.g., florists, soldiers). In addition, we assessed participants’ explicit gender-related attitudes to explore whether they might predict potential effects of gender-stereotypical information on anticipatory eye movements. The observed gaze pattern reflected that participants used gendered information to predict who was agent of the action. These effects were larger for female- vs. male-stereotypical contextual information but were not related to participants’ gender-related attitudes. Our results showed that predictive language processing can be moderated by gender stereotypes, and that anticipation is stronger for female (vs. male) depicted characters. Further research should test the direct relation between gender-stereotypical sentence processing and implicit gender attitudes. These findings contribute to both social psychology and psycholinguistics research, as they extend our understanding of stereotype processing in multimodal contexts and regarding the role of attitudes (on top of world knowledge) in language prediction.

Neural dynamics of semantic composition

Human speech comprehension is remarkable for its immediacy and rapidity. The listener interprets an incrementally delivered auditory input, millisecond by millisecond as it is heard, in terms of complex multilevel representations of relevant linguistic and nonlinguistic knowledge. Central to this process are the neural computations involved in semantic combination, whereby the meanings of words are combined into more complex representations, as in the combination of a verb and its following direct object (DO) noun (e.g., "eat the apple"). These combinatorial processes form the backbone for incremental interpretation, enabling listeners to integrate the meaning of each word as it is heard into their dynamic interpretation of the current utterance. Focusing on the verb-DO noun relationship in simple spoken sentences, we applied multivariate pattern analysis and computational semantic modeling to source-localized electro/magnetoencephalographic data to map out the specific representational constraints that are constructed as each word is heard, and to determine how these constraints guide the interpretation of subsequent words in the utterance. Comparing context-independent semantic models of the DO noun with contextually constrained noun models reflecting the semantic properties of the preceding verb, we found that only the contextually constrained model showed a significant fit to the brain data. Pattern-based measures of directed connectivity across the left hemisphere language network revealed a continuous information flow among temporal, inferior frontal, and inferior parietal regions, underpinning the verb's modification of the DO noun's activated semantics. These results provide a plausible neural substrate for seamless real-time incremental interpretation on the observed millisecond time scales.

Incorporating Word Significance into Aspect-Level Sentiment Analysis

Aspect-level sentiment analysis has drawn growing attention in recent years, with higher performance achieved through the attention mechanism. Despite this, previous research does not consider some human psychological evidence relating to language interpretation. This results in attention being paid to less significant words especially when the aspect word is far from the relevant context word or when an important context word is found at the end of a long sentence. We design a novel model using word significance to direct attention towards the most significant words, with novelty decay and incremental interpretation factors working together as an alternative for position based models. The interpretation factor represents the maximization of the degree each new encountered word contributes to the sentiment polarity and a counter balancing stretched exponential novelty decay factor represents decaying human reaction as a sentence gets longer. Our findings support the hypothesis that the attention mechanism needs to be applied to the most significant words for sentiment interpretation and that novelty decay is applicable in aspect-level sentiment analysis with a decay factor β = 0.7 .

The temporal dynamics of structure and content in sentence comprehension: Evidence from fMRI-constrained MEG.

Humans have a striking capacity to combine words into sentences that express new meanings. Previous research has identified key brain regions involved in this capacity, but little is known about the time course of activity in these regions, as hemodynamic methods such as fMRI provide little insight into temporal dynamics of neural activation. We performed an MEG experiment to elucidate the temporal dynamics of structure and content processing within four brain regions implicated by fMRI data from the same experiment: the temporo-parietal junction (TPJ), the posterior temporal lobe (PTL), the anterior temporal lobe (ATL), and the anterior inferior frontal gyrus (IFG). The TPJ showed increased activity for both structure and content near the end of the sentence, consistent with a role in incremental interpretation of event semantics. The PTL, a region not often associated with core aspects of syntax, showed a strong early effect of structure, consistent with predictive parsing models, and both structural and semantic context effects on function words. These results provide converging evidence that the PTL plays an important role in lexicalized syntactic processing. The ATL and IFG, regions traditionally associated with syntax, showed minimal effects of sentence structure. The ATL, PTL and IFG all showed effects of semantic content: increased activation for real words relative to nonwords. Our fMRI-guided MEG investigation therefore helps identify syntactic and semantic aspects of sentence comprehension in the brain in both spatial and temporal dimensions.

The scope of children’s scope: Representation, parsing and learning

This paper reviews some developmental psycholinguistic literature on quantifier scope. I demonstrate how scope has been used as a valuable probe into children’s grammatical representations, the nature of children’s on-line understanding mechanisms, and the role that experience plays in language acquisition. First, children’s interpretations of certain scopally ambiguous sentences reveals that their syntactic representations are hierarchical, with the c-command relation playing a fundamental role in explaining interpretive biases. Second, children’s scope errors are explained by incremental parsing and interpretation mechanisms, paired with difficulty revising initial interpretations. Third, a priming manipulation reveals that children’s clauses, like those of adults, are represented with predicate-internal subjects. Finally, data on scope variation in Korean reveals that in the absence of disambiguating evidence, parameter setting is an essentially random process. Together, these discoveries reveal how the developmental psycholinguistics of scope has proved a valuable tool for probing issues of grammar, parsing and learning.

Who's who? The incremental processing of presuppositions (and beyond)

Who's who? The incremental processing of presuppositions (and beyond)

Pragmatic effects of `more than' and `at least' in incremental interpretation

We investigate the incremental interpretation of comparative and superlative numeral modifiers by manipulating the speaker’s epistemic state in an eye-tracking reading experiment. The results reveal a different processing profile for two types of numeral modifiers. We take this difference to point to a difference in the source and nature of the attested effects (e.g., Quantity- vs. Manner-based pragmatic reasoning). Our findings inform the existing theoretical landscape, invalidating a number of accounts of speaker ignorance effects with numeral modifiers and giving support to Quantity-based accounts of such effects with superlative modifiers.

Incremental generation of answers during the comprehension of questions with quantifiers

The paper presents a study on the online interpretation of quantified questions involving complex domain restriction, for instance, are all triangles blue that are in the circle. Two probe reaction time (RT) task experiments were conducted to study the incremental nature of answer generation while manipulating visual contexts and response hand overlap between tasks. We manipulated the contexts in such a way that the incremental answer to the question changed from ‘yes’ to ‘no’ or remained the same before and after encountering the extraposed relative clause. The findings of both experiments provide evidence for incremental answer preparation but only if the context did not involve the risk of answer revision. Our results show that preliminary output from incremental semantic interpretation results in response priming that facilitates congruent responses in the probe RT task.

Referential numerosity in quantification expressions. An ERP study on Italian.

Referential numerosity in quantification expressions. An ERP study on Italian.

On predicting form and meaning in a second language.

We used event-related potentials (ERP) to investigate whether Spanish-English bilinguals preactivate form and meaning of predictable words. Participants read high-cloze sentence contexts (e.g., "The student is going to the library to borrow a . . ."), followed by the predictable word (book), a word that was form-related (hook) or semantically related (page) to the predictable word, or an unrelated word (sofa). Word stimulus onset synchrony (SOA) was 500 ms (Experiment 1) or 700 ms (Experiment 2). In both experiments, all nonpredictable words elicited classic N400 effects. Form-related and unrelated words elicited similar N400 effects. Semantically related words elicited smaller N400s than unrelated words, which however, did not depend on cloze value of the predictable word. Thus, we found no N400 evidence for preactivation of form or meaning at either SOA, unlike native-speaker results (Ito, Corley et al., 2016). However, non-native speakers did show the post-N400 posterior positivity (LPC effect) for form-related words like native speakers, but only at the slower SOA. This LPC effect increased gradually with cloze value of the predictable word. We do not interpret this effect as necessarily demonstrating prediction, but rather as evincing combined effects of top-down activation (contextual meaning) and bottom-up activation (form similarity) that result in activation of unseen words that fit the context well, thereby leading to an interpretation conflict reflected in the LPC. Although there was no evidence that non-native speakers preactivate form or meaning, non-native speakers nonetheless appear to use bottom-up and top-down information to constrain incremental interpretation much like native speakers do. (PsycINFO Database Record

INCREMENTAL REFINEMENT OF FAÇADE MODELS WITH ATTRIBUTE GRAMMAR FROM 3D POINT CLOUDS

Data acquisition using unmanned aerial vehicles (UAVs) has gotten more and more attention over the last years. Especially in the field of building reconstruction the incremental interpretation of such data is a demanding task. In this context formal grammars play an important role for the top-down identification and reconstruction of building objects. Up to now, the available approaches expect offline data in order to parse an a-priori known grammar. For mapping on demand an on the fly reconstruction based on UAV data is required. An incremental interpretation of the data stream is inevitable. This paper presents an incremental parser of grammar rules for an automatic 3D building reconstruction. The parser enables a model refinement based on new observations with respect to a weighted attribute context-free grammar (WACFG). The falsification or rejection of hypotheses is supported as well. The parser can deal with and adapt available parse trees acquired from previous interpretations or predictions. Parse trees derived so far are updated in an iterative way using transformation rules. A diagnostic step searches for mismatches between current and new nodes. Prior knowledge on fac¸ades is incorporated. It is given by probability densities as well as architectural patterns. Since we cannot always assume normal distributions, the derivation of location and shape parameters of building objects is based on a kernel density estimation (KDE). While the level of detail is continuously improved, the geometrical, semantic and topological consistency is ensured.