Sentence Matching Task Research Articles

Semantically Enriched Cross-Lingual Sentence Embeddings for Crisis-related Social Media Texts

Tasks such as semantic search and clustering on crisis-related social media texts enhance our comprehension of crisis discourse, aiding decision-making and targeted interventions. Pre-trained language models have advanced performance in crisis informatics, but their contextual embeddings lack semantic meaningfulness. Although the CrisisTransformers family includes a sentence encoder to address the semanticity issue, it remains monolingual, processing only English texts. Furthermore, employing separate models for different languages leads to embeddings in distinct vector spaces, introducing challenges when comparing semantic similarities between multi-lingual texts. Therefore, we propose multi-lingual sentence encoders (CT-XLMR-SE and CT-mBERT-SE) that embed crisis-related social media texts for over 50 languages, such that texts with similar meanings are in close proximity within the same vector space, irrespective of language diversity. Results in sentence encoding and sentence matching tasks are promising, suggesting these models could serve as robust baselines when embedding multi-lingual crisis-related social media texts. The models are publicly available at: https://huggingface.co/crisistransformers.

Structural and Extralinguistic Aspects of Code-Switching: Evidence From Papiamentu-Dutch Auditory Sentence Matching

Despite a wealth of studies on effects of switch locations in code-switching (CS), we know relatively little about how structural factors such as switch location and extralinguistic factors such as directionality preferences may jointly modulate CS (cf., Stell and Yapko, 2015). Previous findings in the nominal domain suggest that within-constituent switching (within the noun phrase) may be easier to process than between-constituent switching (a structural effect), and that there may also be directionality effects with switches preferred only in one language direction (an extra-linguistic effect). In this study we examine a different domain, namely how VP-external (preverbal) vs. VP-internal (postverbal) switch location and switch directionality affects the processing of Papiamentu–Dutch mixed subject-verb-object (SVO) sentences. We manipulated switch location (preverbal/postverbal), and directionality of switch (PD/DP) and tested 50 Papiamentu–Dutch bilinguals on an auditory sentence matching task. The results from the mixed conditions showed no effect of switch location. Instead, we found only an effect of directionality and in an unexpected direction for this population, with switches from Dutch to Papiamentu being processed faster than switches from Papiamentu to Dutch regardless of switch location. The results highlight the importance of taking extralinguistic factors into account, but also the challenges of studying CS, particularly in lesser studied speech communities, and the need for a data-driven, cross-disciplinary approach to the study of CS.

Differential Patterns of Children's Knowledge of Quantifier Meaning Revealed Under Different Tasks

This study examines children’s comprehension of quantifiers in Hebrew, using several tasks. We focused on a linguistic ambiguity related to universal quantifiers that express a distinction between collectivity and distributivity: all can be assigned with both a collective reading and a distributive reading (“a flower for all fairies” can be interpreted as an event with one flower or an event with multiple flowers), whereas each has a distributive reading only (“a flower for each fairy” is an event with multiple flowers). Unlike English, Hebrew has a single universal quantifier and thus, it expresses the collectivity/distributivity distinction using two morphosyntactic forms: one form (kol+ definite plural noun) is equivalent to all and has the two readings, and the other form (kol+ indefinite singular noun) is equivalent to each and has only one reading. We examined how Hebrew-speaking preschoolers (4-6 years) understand sentences in the two forms, and how they resolve the ambiguity of the ambiguous form, while focusing on the type and presence of contrast in three preference tasks: Experiment 1 used a conventional picture-matching task where the collective and distributive meanings were contrasted using two pictures (meaning contrast); Experiment 2 used a sentence-matching task where the two morphosyntactic forms were contrasted using two sentences (linguistic contrast); and Experiment 3 used a novel drawing task including instructions in one form (no contrast). In all tasks, adults showed a consistent response pattern, matching the ambiguous form (equivalent to all) to the collective reading and the distributive form (equivalent to each) to the distributive reading. Children, on the other hand, were extremely affected by the task, showing adult-like performance pattern in the picture-matching task, but not in the other tasks. This suggests that children can distinguish between the two morphosyntactic forms, but they do not fully attain adults’ preference pattern. The differences between the tasks can be attributed to the salience of the contrast, task experience, or working memory. The results highlight the need for a careful selection of language tasks both in basic research and in clinical assessment.

Sentence Similarity Computation in Question Answering Robot

Computing semantic similarity between sentences or texts is vital in many natural language processing (NLP) tasks such as search, query suggestion, and question answering (QA). Many methods have been developed, based on lexical matching, distributional semantics, etc. However, lexical features, like string matching, fail to capture semantic similarity. In this research, our focus lies on the implementation of distributional representations and how to tune parameters when obtaining representations of words with commonly used word embedding techniques, e.g., Word2Vec and GloVe. We conduct experiments in the setting of Chinese semantic sentence matching tasks on the finance-domain. We examine the goodness of word embedding by both the cosine similarity of semantically similar sentence pairs and semantically dissimilar pairs. Based on our experiments, Word2Vec performs better than GloVe in the sense that Chinese character embedding from Word2Vec yield larger disparity of cosine distances between similar sentence pairs and dissimilar pairs. Also we report the optimal parameters for Word2Vec continuous bag-of-word (CBOW) through our trials, with window size being 6 and embedding dimension being 400, which can be good initial values for other projects.

Children’s understanding of yesterday and tomorrow

A picture–sentence matching task was used to investigate children’s understanding of yesterday and tomorrow. In Experiment 1, 3- to 5-year-olds viewed two pictures of an object with a visible change of state (e.g., a carved pumpkin and an intact pumpkin) while listening to sentences referring to past or future actions (“I carved the pumpkin yesterday” or “I’m gonna carve the pumpkin tomorrow”) and selected the matching picture. Children performed better with past tense sentences than with future tense sentences, and including tomorrow in future tense sentences increased accuracy. In the next two experiments, 4- and 5-year-olds (Experiment 2) and adults (Experiment 3) completed the same task but with sentences containing conflicting temporal information (“I carved the pumpkin tomorrow”). Children tended to select pictures depicting the outcome of actions regardless of tense or temporal adverb, whereas adults’ judgments were based on temporal adverbs. In Experiment 4, 3- to 5-year-olds completed tasks requiring either forward or backward temporal reasoning about sentences referring to before, after, yesterday, today, and tomorrow. Across sentence types, forward temporal reasoning was easier for children than backward temporal reasoning. Altogether, results indicated that children understand yesterday better than tomorrow due to the increased cognitive demands involved in reasoning about future events.

Wh-Constraints In Interlanguage Grammar Of Persian EFL Learners and its Implication for Teaching English as a Foreign Language

In second language (L2) studies, one direction of research is to investigate whether adult L2 learners still have access to Universal Grammar (UG) . In the related vein, this study is an indirect assessment of availability of Wh-constraints in interlanguage grammar of Persian EFL learners through an on-line sentence-matching task. To do so, 60 university students, both male and female, majoring in EFL at Khorasgan university in Iran were chosen through an Oxford Placement Test (OPT) and divided into three proficiency groups(i.e. 20 Low Intermediate, 20 High Intermediate, and 20 Advanced groups). Moreover, 10 native English speakers were chosen as the control group in this study. Both English native speakers and Persian EFL learners performed an on-line sentence-matching task. The on-line test contained 40 pairs of English sentences, 20 matching grammatical pairs which observed the UG Wh- constraint and 20 matching ungrammatical pairs which violated this Principle. In sentence-matching task students responded to two sentences on a computer screen indicating whether the two sentences were identical or not. The reaction time of EFL learners to both grammatical and ungrammatical pairs was measured and compared to that of English native speakers. The results revealed that native speakers of English can do the task faster than non-natives. Thus, Wh- constraint is not accessible to second language learners to the same extent that it is to first language learners. This study furthers our understanding of the Persian EFL learners’ performance in the area of L2 language acquisition. Moreover, the more proficient groups were faster than the less ones in sentence-matching task. As the proficiency increased, the results got much closer to the results of native speakers. Therefore, proficiency can be considered as a strong factor for UG activation.

Evaluating vector space models with canonical correlation analysis

AbstractVector space models are used in language processing applications for calculating semantic similarities of words or documents. The vector spaces are generated with feature extraction methods for text data. However, evaluation of the feature extraction methods may be difficult. Indirect evaluation in an application is often time-consuming and the results may not generalize to other applications, whereas direct evaluations that measure the amount of captured semantic information usually require human evaluators or annotated data sets. We propose a novel direct evaluation method based on canonical correlation analysis (CCA), the classical method for finding linear relationship between two data sets. In our setting, the two sets are parallel text documents in two languages. A good feature extraction method should provide representations that reflect the semantic content of the documents. Assuming that the underlying semantic content is independent of the language, we can study feature extraction methods that capture the content best by measuring dependence between the representations of a document and its translation. In the case of CCA, the applied measure of dependence is correlation. The evaluation method is based on unsupervised learning, it is language- and domain-independent, and it does not require additional resources besides a parallel corpus. In this paper, we demonstrate the evaluation method on a sentence-aligned parallel corpus. The method is validated by showing that the obtained results with bag-of-words representations are intuitive and agree well with the previous findings. Moreover, we examine the performance of the proposed evaluation method with indirect evaluation methods in simple sentence matching tasks, and a quantitative manual evaluation of word translations. The results of the proposed method correlate well with the results of the indirect and manual evaluations.

Transfer effects in the interpretation of definite articles by Spanish heritage speakers

This study investigates the role of transfer from the stronger language by focusing on the interpretation of definite articles in Spanish and English by Spanish heritage speakers (i.e., minority language-speaking bilinguals) residing in the U.S., where English is the majority language. Spanish plural NPs with definite articles can express generic reference (Los elefantes tienen colmillos de marfil), or specific reference (Los elefantes de este zoológico son marrones). English plurals with definite articles can only have specific reference (The elephants in this zoo are brown), while generic reference is expressed with bare plural NPs (Elephants have ivory tusks). Furthermore, the Spanish definite article is preferred in inalienable possession constructions (Pedro levantó la mano“Peter raised the hand”), whereas in English the use of a definite article typically means that the body part belongs to somebody else (alienable possession). Twenty-three adult Spanish heritage speakers completed three tasks in Spanish (acceptability judgment, truth-value judgment, and picture–sentence matching tasks) and the same three tasks in English. Results show that the Spanish heritage speakers exhibited transfer from English into Spanish with the interpretation of definite articles in generic but not in inalienable possession contexts. Implications of this finding for the field of heritage language research and for theories of article semantics are discussed.

Agrammatic comprehension caused by a glioma in the left frontal cortex

It has been known that lesions in the left inferior frontal gyrus (L. IFG) do not always cause Broca’s aphasia, casting doubt upon the specificity of this region. We have previously devised a picture–sentence matching task for a functional magnetic resonance imaging (fMRI) study, and observed that both pars triangularis (L. F3t) of L. IFG (extending to pars opercularis (L. F3op)) and the left lateral premotor cortex (L. LPMC) are selectively involved in syntactic processing. The present study with lesion-symptoms mapping was conducted to examine whether the function of these regions is indeed critical for syntactic comprehension. Using the same picture–sentence matching task, we examined 21 patients with a glioma in the left frontal cortex but with no apparent disability in verbal/written communication or intelligence quotient. This task included three main conditions of sentence types: canonical/subject-initial active sentences, non-canonical/subject-initial passive sentences, and non-canonical/object-initial scrambled sentences. The patients preoperatively underwent a high-resolution 3D-MRI, and voxel-based lesion-symptom mapping was employed for the error rates data. We found that the patients with a lesion in L. F3op/F3t or L. LPMC showed differential patterns of condition-selective deficits in the comprehension of sentences. More specifically, the L. F3op/F3t-damaged patients had more profound deficits in the comprehension of non-canonical sentences, whereas the L. LPMC-damaged patients had more profound deficits in the comprehension of object-initial scrambled sentences. These results establish that a lesion in L. F3op/F3t or L. LPMC is sufficient to cause agrammatic comprehension.

L2 knowledge of verb placement by Japanese learners of English

This paper reports on an experiment which examined the knowledge of verb placement by Japanese learners of English (JLEs). The results of two grammaticality judgement tasks indicated that JLEs acquire the unavailability of an NP-shift operation relatively early, but their judgements of sentences involving V-raising (i.e., illicit *SV-Adv-O word order) was indecisive up to the Intermediate level, and this seemed to improve at the Advanced level. On the other hand, the results of a sentence matching task showed that no JLE group clearly differentiated the +/-V-raising word orders in terms of their matching speed for both the present and the past tense contexts. The results are interpreted to indicate the possibility that JLEs can learn the overt word order facts, but never end up with the same grammatical representation of V-raising parameter as native speakers. This supports the hypotheses arguing for impairment of grammatical features in second language acquisition.

Monitoring in language perception: Evidence from ERPs in a picture–sentence matching task

P600 effects have been observed after syntactic ambiguous sentences, after several types of syntactic and semantic anomalies and after orthographic anomalies. On the basis of these findings, several investigators propose the P600 effect to reflect syntactic repair or syntactic restructuring. According to our Monitoring Theory the P600 effect reflects more general sentence reanalysis, to check whether the input sentence has been perceived appropriately. When the brain encounters a highly unexpected linguistic event, a conflict arises between the expected representation and the representation derived from the input. This conflict is proposed to trigger a process of reanalysis. In the present study, expectancy was manipulated by varying the truth-value of a sentence in relation to a picture. ERPs were recorded from 27 electrodes while we presented participants (N=30) pictures of spatial arrays followed by a sentence giving a correct or incorrect description of the picture. The mismatches were predicted to create a conflict between the conceptual representation on the basis of the picture and the actual sentence and should therefore lead to a P600. A P600 effect was indeed observed after both intra-dimensional □ △—‘the triangle stands in front of the square.’, and extra-dimensional □ △—‘the triangle stands below the square.’ mismatches. The present results support our Monitoring Theory; that is, the function of reprocessing reflected by the P600 effect is not purely syntactic repair or restructuring but is more general in nature, to check for possible processing errors.

The Nature and Time Course of Pragmatic Plausibility Effects

The experiments reported in this article used a delayed same/different sentence matching task with concurrent measurement of eye movements to investigate the nature of the plausibility effect. The results clearly show that plausibility effects are not due to low level lexical associative processes, but arise as a consequence of the processing of the earliest or most basic form of sentential meaning. In fact, when sentential implausibility and lexical association are varied simultaneously, it is only sentential implausibility that exerts an effect. Effects of implausibility occur rapidly--sometimes parafoveally--and are localised in the regions of the sentence where the implausibility occurs, suggesting an incremental interpretive process progressing on a roughly word-by-word basis. It is suggested that plausibility effects result from the operation of a heuristically-driven process of sentential interpretation. This appears to behave in a 'modular' fashion, despite being influenced by real world knowledge and probabilities.

Separating phonological and semantic processing in auditory sentence processing: a high-resolution event-related brain potential study.

Phonological and semantic processing was studied using high-resolution event-related brain potentials (ERPs) during a sentence-matching task to investigate the spatial distribution of the phonological mismatch negativity (PMN) and the N400 response. It was hypothesized that the two components were spatially separable and that the activity matched prior localization knowledge. Participants examined visual-auditory sentence pairs that related within a semantic hierarchy (e.g., visual: "The man is teaching in the classroom"; Auditory: "The man is in the em leader school/barn"). Semantic congruency was varied for the final words of the spoken sentences. Incongruent words mismatched expectation in terms of both the initial phonological features (unexpected sound) and semantic features (unexpected meaning). In addition, the category-exemplar probability of the final words was either high or low, with low probability words being more difficult to anticipate. Low probability words were predicted to selectively affect PMN activity. We found that incongruent words elicited a PMN (287 msec) and a N400 (424 msec), for both the high and low probability words. As expected, low probability congruent words elicited a small PMN but no N400. In contrast, high probability congruent words elicited neither a detectible PMN nor a N400. The primary PMN sources were in left inferior frontal and inferior parietal lobes. The primary N400 source activation occurred along the left perisylvian cortex, consistent with prior N400 source localization work. From these results, it was concluded that the PMN and N400 were localized to separate cortical language (and memory) regions and had different source activation patterns.

Passives in agrammatic sentence comprehension: A German study

Background: A large number of studies examining agrammatic comprehension of canonical and non‐canonical sentences in Broca's aphasia have focused on passives and results have been interpreted in theoretical frameworks such as the trace deletion hypothesis (TDH: Grodzinsky, 1995a). However, there are a number of unresolved issues associated with passives. The linguistic analysis of passive structures in different languages has remained controversial as well as the empirical neurolinguistic basis of agrammatic passive comprehension. In addition, a variety of morphological and semantic questions have been raised with respect to the implicit argument in short passives and the ordering of thematic roles reflected by different positions of the by‐phrase in long passives.Aims: The major aims of the present study were to re‐examine the analyses of passives with and without traces, the role of an implicit argument in short passives, and the influence of the position of the by‐phrase on agrammatic sentence comprehension.Methods & Procedures: A binary picture–sentence matching task was administered to six non‐fluent German agrammatic speakers. Various types of passives including long, short, and topicalised passives were tested. Additionally, comprehension of active SVO sentences was assessed in a separate but similar session. Only those patients whose comprehension on active sentences was above chance were included.Outcomes & Results: As a group, the six subjects performed above chance over all passive types. If only long canonical passives are considered, as is done in most studies, five subjects showed a pattern compatible with the TDH. However, the picture was modified if other passive constructions were taken into account, in which case only three of the six subjects showed TDH conformity.Conclusions: There is no unique pattern of agrammatic passive comprehension and only half of the agrammatic subjects conformed to the trace deletion hypothesis. Given the results on long canonical and topicalised passives, our data support linguistic analyses that assume a trace‐based derivation of passives. Furthermore, the results are in line with linguistic analyses adopting an implicit argument in short passives. Since comprehension of topicalised passives with a canonical order of theta‐roles was not better than that of long passives without a canonical order, the agrammatic problem with passives does not seem to hinge on semantics.

Sentence matching: a re-examination

This article examines sentence matching, a methodology frequently used in the second language (L2) literature to determine notions of grammaticality of nonnative speakers (NNS). Native speakers (NS) of French and L2 learners of French performed a sentence-matching task focusing on three areas of French grammar: adverb placement, subject-verb agreement and clitic-pronoun placement. In sentence-matching tasks participants respond to two sentences on a computer screen indicating whether the two sentences are identical or not. In general, grammatical sentences are responded to faster than ungrammatical sentences and have been used in the L2 literature as a way of determining grammatical knowledge. The results from the NSs of French show that when there is a high degree of difficulty in interpretation of ungrammatical sentences, sentence matching is a useful tool for determining grammaticality. For NNSs there is little evidence that sentence matching predicts grammaticality. A traditional acceptability judgement task was administered to NNSs. Sentence-matching did not correlate with NNS’s individual notions of grammaticality. Issues of proficiency level and the nature of ungrammatical sentences are important determinants when considering the validity of sentence-matching as a research tool.

Assessing L2 knowledge of Spanish clitic placement: converging methodologies

In this paper, we report on an experiment investigating adult second language (L2) acquisition of Spanish object clitic placement by native speakers of English (which lacks clitics) and French (where clitics contrast in certain respects with Spanish). Two different experimental methodologies are compared: an on-line sentence matching (SM) task and an off-line grammaticality judgement (GJ) task. Subjects were advanced and intermediate level English-speaking and French-speaking learners of Spanish, together with a native-speaker control group. A variety of constructions involving Spanish clitic placement were tested. The results from the two tasks complement each other: all groups show significant effects for grammaticality on the SM task and considerable accuracy on the GJ task, suggesting that L2 clitic placement can successfully be acquired even when the first language (L1) lacks clitics. However, both tasks reveal that L2 learners have difficulties in restructuring and causative contexts, which we attribute to problems with clitic climbing.

EARLY, MANDATORY, PRAGMATIC PROCESSING

The experiment reported in this paper used a delayed same/different sentence matching task with concurrent measurement of eye movements to investigate three questions: whether pragmatic plausibility effects are restricted to certain phrasal environments; how rapidly such effects are shown in on-line sentence processing; and whether they are a product of optional, high-level, inferential processes. The results clearly show that plausibility effects are not restricted to low–level phrasal units and that they appear to arise as a necessary consequence of the process responsible for deriving basic sentence meaning. The rapid and highly localized nature of the effects supports a view of sentence processing involving incremental interpretation of the earliest available syntactic representations. We argue that the apparently mandatory nature of plausibility effects, coupled with their insensitivity to repetition context, presents difficulties for both modular and interactive views of sentence processing.

Sentence matching and processing in L2 development

The processing strategies described in Clahsen (1984) to explain the develop ment of German word order make predictions that can be tested ex perimentally. Clahsen's Initialization/Finalization Strategy (IFS) in particular predicts that uninverted, ADV-SVO sentences will exact less cost in terms of processing than inverted, ADV-VSO sentences, even though inverted sent ences are grammatical in the target language and uninverted sentences are ungrammatical. The experimental means employed to test this prediction is the Sentence Matching (SM) procedure described originally in Freedman and Forster (1985). In the SM procedure, response times are elicited for particular types of sentences by measuring the time (in msec.) it takes for subjects to determine whether two sentences presented by computer are identical or different. The results of one of the experiments reported here show that inverted sentences result in significantly shorter response times than uninverted sentences for non-native speakers. This finding directly contradicts the IFS-derived prediction. However, further experimental work reported here indicates that native speakers do not respond at all to the inverted-uninverted contrast. The rest of the article thus seeks to explain this somewhat surprising finding. The proposed explanation also suggests that natives and non-natives may process sentences in the SM task in rather different ways.

The psychological status of overgenerated sentences

A sentence matching task is used to assess the processing effects of the ungrammaticality produced by violations of constraints on movement rules. Surprisingly, no effects of ungrammaticality were observed either for violations of the Specified Subject Constraint or the Subjacency constraint. These ‘overgenerated’ sentences could apparently be processed with the same fluency as fully grammatical controls, despite the fact that other types of ungrammaticality produced marked increases in matching times. It is proposed that the matching task utilizes a level of mental representation at which overgenerated sentences are indistinguishable from fully grammatical sentences. This implies a close correspondence between formal derivational mechanisms and features of the operation of the language processor.

Prefix versus suffix information processing in the comprehension of tense and aspect

ABSTRACTThis research evaluated the Polish child's capacity to comprehend aspect and tense distinctions realized by either prefixation or suffixation. Ten 2; 6 and ten 3; 6 children were given a picture–sentence matching task. Aspect picture sets contrasted completed with incompleted situations and sentences contrasted perfective with imperfective verbs differing by a prefix or a suffix. Tense picture sets portrayed either ongoing vs. anticipated action or results of action vs. anticipated action contrasts. Again sentences differed in verb forms either by prefixes or suffixes. The results showed that young children can pay attention to the beginnings of words as easily as to the ends of words and can understand a wide range of aspect and tense distinctions.