Lexical Model Research Articles

Robust neural tracking of linguistic speech representations using a convolutional neural network

Objective. When listening to continuous speech, populations of neurons in the brain track different features of the signal. Neural tracking can be measured by relating the electroencephalography (EEG) and the speech signal. Recent studies have shown a significant contribution of linguistic features over acoustic neural tracking using linear models. However, linear models cannot model the nonlinear dynamics of the brain. To overcome this, we use a convolutional neural network (CNN) that relates EEG to linguistic features using phoneme or word onsets as a control and has the capacity to model non-linear relations. Approach. We integrate phoneme- and word-based linguistic features (phoneme surprisal, cohort entropy (CE), word surprisal (WS) and word frequency (WF)) in our nonlinear CNN model and investigate if they carry additional information on top of lexical features (phoneme and word onsets). We then compare the performance of our nonlinear CNN with that of a linear encoder and a linearized CNN. Main results. For the non-linear CNN, we found a significant contribution of CE over phoneme onsets and of WS and WF over word onsets. Moreover, the non-linear CNN outperformed the linear baselines. Significance. Measuring coding of linguistic features in the brain is important for auditory neuroscience research and applications that involve objectively measuring speech understanding. With linear models, this is measurable, but the effects are very small. The proposed non-linear CNN model yields larger differences between linguistic and lexical models and, therefore, could show effects that would otherwise be unmeasurable and may, in the future, lead to improved within-subject measures and shorter recordings.

The lexical constructional model meets syntax: guidelines of the formalized lexical-constructional model (FL_CxG )

This paper offers the basic guidelines of a formalized version of the Lexical Constructional Model (LCM; Ruiz de Mendoza & Mairal Usón, 2008, 2011; Ruiz de Mendoza & Galera, 2014), the Formalized Lexical-Constructional Grammar (FL_CxG), which will pave the way for future computational developments, such as parsers or lexical databases. The FL_CxG deploys (i) the typologically oriented syntactic apparatus of Role and Reference Grammar (Van Valin, 2005; Van Valin & LaPolla, 1997), (ii) the catalogue of constructional units arranged in a 4-layer typology, as proposed by the LCM, and (iii) some insights for semantic representations from the Generative Lexicon Theory (Pustejovsky, 1995; Pustejovsky & Batiukova, 2019), and Minimal English (Goddard, 2018). All the components of the FL_CxG (lexical units and construct(ion)s) are formally encoded as Typed Feature Structures in the format of Attribute Value Matrixes. These units are to be understood as constraints operating in the unification processes which underlie the generation/decoding of a given fragment of language.

Simplification of Arabic text: A hybrid approach integrating machine translation and transformer-based lexical model

The process of text simplification (TS) is crucial for enhancing the comprehension of written material, especially for people with low literacy levels and those who struggle to understand written content. In this study, we introduce the first automated approach to TS that combines word-level and sentence-level simplification techniques for Arabic text. We employ three models: a neural machine translation model, an Arabic-BERT-based lexical model, and a hybrid model that combines both methods to simplify the text. To evaluate the models, we created and utilized two Arabic datasets, namely EW-SEW and WikiLarge, comprising 82,585 and 249 sentence pairs, respectively. As resources were scarce, we made these datasets available to other researchers. The EW-SEW dataset is a commonly used English TS corpus that aligns each sentence in the original English Wikipedia (EW) with a simpler reference sentence from Simple English Wikipedia (SEW). In contrast, the WikiLarge dataset has eight simplified reference sentences for each of the 249 test sentences. The hybrid model outperformed the other models, achieving a BLEU score of 55.68, a SARI score of 37.15, and an FBERT score of 86.7% on the WikiLarge dataset, demonstrating the effectiveness of the combined approach.

Effective matching of patients to clinical trials using entity extraction and neural re-ranking

Introduction:Clinical trials (CTs) often fail due to inadequate patient recruitment. Finding eligible patients involves comparing the patient’s information with the CT eligibility criteria. Automated patient matching offers the promise of improving the process, yet the main difficulties of CT retrieval lie in the semantic complexity of matching unstructured patient descriptions with semi-structured, multi-field CT documents and in capturing the meaning of negation coming from the eligibility criteria. Objectives:This paper tackles the challenges of CT retrieval by presenting an approach that addresses the patient-to-trials paradigm. Our approach involves two key components in a pipeline-based model: (i) a data enrichment technique for enhancing both queries and documents during the first retrieval stage, and (ii) a novel re-ranking schema that uses a Transformer network in a setup adapted to this task by leveraging the structure of the CT documents. Methods:We use named entity recognition and negation detection in both patient description and the eligibility section of CTs. We further classify patient descriptions and CT eligibility criteria into current, past, and family medical conditions. This extracted information is used to boost the importance of disease and drug mentions in both query and index for lexical retrieval. Furthermore, we propose a two-step training schema for the Transformer network used to re-rank the results from the lexical retrieval. The first step focuses on matching patient information with the descriptive sections of trials, while the second step aims to determine eligibility by matching patient information with the criteria section. ResultsOur findings indicate that the inclusion criteria section of the CT has a great influence on the relevance score in lexical models, and that the enrichment techniques for queries and documents improve the retrieval of relevant trials. The re-ranking strategy, based on our training schema, consistently enhances CT retrieval and shows improved performance by 15% in terms of precision at retrieving eligible trials. ConclusionThe results of our experiments suggest the benefit of making use of extracted entities. Moreover, our proposed re-ranking schema shows promising effectiveness compared to larger neural models, even with limited training data. These findings offer valuable insights for improving methods for retrieval of clinical documents.

Lexical modeling for the development of Amharic automatic speech recognition systems

Amharic is the second most spoken Semitic language after Arabic. It has its own syllabary writing system, each character representing a consonant and a vowel. Automatic Speech Recognition (ASR) researches for Amharic have been conducted on the basis of grapheme-based pronunciation lexicon, taking advantage of the nature of its writing system. However, the epenthetic vowel and the glottal stop consonant represented in the writing system may not be pronounced in all of their occurrences. Moreover, the writing system does not differentiate geminated and non-geminated forms of consonants. Therefore, the grapheme-based pronunciation lexicon used so far has limitations with regard to these language features. To handle these limitations, we have prepared word- and morpheme-based pronunciation lexicons using data-driven and knowledge-driven experts’ transcription. The data-driven transcription has been used for the preparation of training pronunciation lexicon while the knowledge-driven has been used to prepare morpheme- and word-based pronunciation lexicons for decoding. When morpheme-based knowledge-driven lexicons are used, better ASR performance (compared with the baseline ASR system that used grapheme-based lexicon) has been achieved although the number of phones is much more (60) than the number of phones used in the grapheme-based lexicon (37).

Auditory pseudoword rhyming effects in bilingual children reflect second language proficiency: An ERP study

This study investigated second language (L2-English) phonological processing in 31 Spanish-English bilingual, 6- to 8-year-old schoolchildren in an event-related potential (ERP) auditory pseudoword rhyming paradigm. In addition, associations between ERP effects and L2 proficiency as measured by standardized tests of receptive language and receptive vocabulary were explored. We found a classic posterior ERP rhyming effect that was more widely distributed in children with higher L2 proficiency in group analyses and was larger for children with better L2 proficiency in correlation analyses. In contrast, the amplitude of an early (75–125 ms) auditory positivity was larger in children with lower L2 proficiency. This pattern suggests differential use of early and late auditory/phonological processing resources in the pseudoword rhyme task associated with L2 proficiency, which is consistent with the predictions of the lexical restructuring model in a bilingual context.

Synergetic Properties of Lexical Structures in Chinese and English

ABSTRACT The synergetic lexical model provides a unique framework for exploration of the interrelationships between the lexical properties of languages. Previous studies concerning several properties of this lexical model have yielded many successful fittings results, but very few studies have investigated synonymy, a major property of this model. The present study uses 825 Chinese and 848 English tokens retrieved from Chinese and English corpora, dictionaries, and thesaurus to conduct a contrastive study on the interrelations between four major properties of this lexical model: word length, word frequency, polysemy, and synonymy. The successful fittings of both languages demonstrate the cross-linguistic validity of the synergetic lexical model, though English belongs to the Germanic language family, while Chinese, a highly analytical language, is of the Sino-Tibetan language family. Moreover, our analysis of the parameters of the fitting results shows that, compared to English, Chinese possesses a greater resistance to shortening word length and a quicker response to semantic change.

‘Pregnancy no bi disease’: Contextual beliefs in antenatal classes in selected Nigerian hospitals

This paper examines how language is used to show shared beliefs between caregivers and pregnant women in antenatal classes in the Nigerian context, which hitherto suffers neglect. The data consist of tape recordings and observational notes taken during antenatal classes in selected hospitals in Lokoja, Nigeria. The study adopted a top-down analytic approach using Odebunmi’s model of contextual beliefs, supported by van Dijk’s epistemic context model, and Fetzer and Fischer’s lexical markers model. The analysis shows three categories of shared knowledge: a. state of pregnancy and postnatal care b. sexual relationships and spice lexemes c. traditional child care and superstitions. Shared knowledge of the state of pregnancy and postnatal care manifests in pregnancy misconceptions, nutrition during pregnancy and postnatal care orientations highlighted by lexical markers such as recoverable knowledge connector and joint attention builder. Shared knowledge of sexual relations and spice lexemes reflect sexual communication and native spices using metaphorization and proximal deictic lexical markers. Grandmothers’ involvement and insistence on using traditional methods and superstitions frame shared knowledge of traditional child care and superstitions through belief affirmative and self-reformulation markers. Analysis of the shared beliefs shows how caregivers represent pregnant women’s mental knowledge in interactions as well as how pregnant women interject beliefs for clarification and validity. This way, both participants demonstrate how contextual beliefs facilitate communicative processes and meaningful understanding in their interactions.

Cross-script L1–L2 and L2–L1 masked translation priming and phonological priming: Evidence from unbalanced Korean–English bilinguals

Aims: Despite a history of research in translation-equivalent priming (either cognate or noncognate and phonological priming with cross-script languages), research with Korean–English bilinguals is very scarce. In this study, we report two masked-priming lexical decision tasks in both directions with Korean–English unbalanced bilinguals, investigating whether cross-language activation occurs depending on different types of cognate, noncognate, and homophone prime–target pairs, the degree of phonological similarity between prime and target, and how L2 proficiency influences each type of priming. Methodology: Two experiments were conducted using a masked translation priming paradigm with unbalanced Korean–English bilinguals: Experiment 1 in the L2–L1 direction and Experiment 2 in the L1–L2 direction. Each experiment used a total of 96 prime–target pairs of cognates, noncognates, and homophones, and 60 subjects for each experiment were asked to judge whether the items were real words or not in a lexical decision task. Data and analysis: Linear mixed-effects models were used to investigate the effects of priming in each condition, the effect of phonological similarity on cognate translation priming, and L2 proficiency. Findings/conclusions: Our results showed that both cognate translation priming and noncognate translation priming arose in the L1–L2 direction, but the phonological priming was not found. In the L2–L1 direction, on the contrary, cognate translation priming and phonological priming effects were observed, but not the noncognate translation priming. Also, the degree of phonological similarity influenced the magnitude of cognate translation priming effects in both directions with a different pattern, supporting for the phonological overlap account between different-script languages. The current study is compatible with the language nonselective activation view and suggests the shared phonological representations even between different-script languages. The results are discussed within several current bilingual lexical processing models.

Metaphors in Italian and Croatian compounds

In this paper, the metaphorical NN and NA/AN compounds in the Italian and Croatian language are described using the contrastive method. A narrow semantic corpus analysis is performed using the generative lexical models, and the metaphorical meanings are shown in the framework of cognitive linguistics.

Fostering Engineering Students’ Competences Development Through Lexical Aspect Acquisition Model

This study aims to investigate the effect of enhanced lexical aspect acquisition model on engineering students’ competences development, namely such 21st century skills as communication (written or oral), self-organization, systematic and critical thinking, project development and implementation, team work, application of fundamental knowledge, work with information. The researchers have developed a novel approach to fostering lexical aspect acquisition through multistage model. The participants of the research were first-year engineering students of the mineral specializations in Saint-Petersburg Mining University. Suggested model-based learning process and conventional-based teaching were used in experimental and control groups accordingly during the spring term 2022. The SPSS software was used to interpret the obtained data. The statistical analysis was carried out with the output of Cronbach’s Alpha reliability test and Pearson correlation test with identification of p-value. The findings showed a significant difference between the level of lexical acquisition in the control and experimental groups proving reliability and validity of the current research.

Ontology-based semantic retrieval of documents using Word2vec model

Semantic retrieval of engineering knowledge is crucial in various engineering activities, such as process development and product model planning. To make a solution for this issue, previously, a few word-based semantic enabling existing approaches such as Lexical Retrieval Model with Semantic Residual Embedding’s (LR-SRE), Document Retrieval Model through Semantic Linking (DR-MTS), Semantic Term Matching in Axiomatic Approaches to Information Retrieval (STM-IR) and Hybrid Ontology for Semantic Information Retrieval Model using Keyword Matching Indexing System (HOS-IR) were utilized. But, they all have shown less query-based accuracy results than the required value. In our proposed Information Retrieval (IR) design, the semantic knowledge-based retrieval scheme has been implemented. For query, entered by a user and processed for finding the dominated word. Word is then compared for its similarity equations, and similarity values are then computed to give output. Highly similar values are obtained as the class value. From class, the respective clusters are selected. Then, documents in that cluster are retrieved and ranked according to the relevance of the user. To support the accuracy level performance of this IR system, a word2vec model has been employed with the benefits of Horse Herd Optimization (HHO), which helps to extract vectors as features for classification. These results are stored as a .csv file for further retrieval. By implementing the proposed IR-word2vec model, the results showed that it outperforms other existing techniques by improved similarity index and accuracy for query results in an execution time of 1.7 s.

Automated MeSH term suggestion for effective query formulation in systematic reviews literature search

• The introduction of the new task of suggesting MeSH terms for systematic review literature search, modelled within the context of an information specialist looking for MeSH terms to add to a Boolean query. • The formulation of MeSH term suggestion methods, based on pre-trained language models, to help information specialists and researchers to construct effective Boolean queries for systematic review creation. • An empirical evaluation of the effectiveness of different MeSH terms suggestion methods. • An understanding of how the MeSH term suggested by the proposed automatic methods differ from those MeSH terms originally selected by information specialists when formulating queries. High-quality medical systematic reviews require comprehensive literature searches to ensure the recommendations and outcomes are sufficiently reliable. Indeed, searching for relevant medical literature is a key phase in constructing systematic reviews and often involves domain (medical researchers) and search (information specialists) experts in developing the search queries. Queries in this context are highly complex, based on Boolean logic, include free-text terms and index terms from standardised terminologies (e.g., the Medical Subject Headings (MeSH) thesaurus), and are difficult and time-consuming to build. The use of MeSH terms, in particular, has been shown to improve the quality of the search results. However, identifying the correct MeSH terms to include in a query is difficult: information experts are often unfamiliar with the MeSH database and unsure about the appropriateness of MeSH terms for a query. Naturally, the full value of the MeSH terminology is often not fully exploited. This article investigates methods to suggest MeSH terms based on an initial Boolean query that includes only free-text terms. In this context, we devise lexical and pre-trained language models based methods. These methods promise to automatically identify highly effective MeSH terms for inclusion in a systematic review query. Our study contributes an empirical evaluation of several MeSH term suggestion methods. We further contribute an extensive analysis of MeSH term suggestions for each method and how these suggestions impact the effectiveness of Boolean queries.

Effects of Native Translation Frequency and L2 Proficiency on L2 word Recognition: Evidence from Korean Speakers of English as a Foreign Language.

The goal of this study was to examine how the lexical characteristics of L1 translated words affect L2 word recognition by Korean speakers of English as a foreign language (EFL) with different English proficiency levels. To this end, we conducted a lexical decision task, in which participants decided whether English strings were words or nonwords. The experiment had three critical conditions: (1) English words that only had a loanword translation in Korean (e.g., card), (2) English words whose native word translation had a lower frequency than their loanword translation (e.g., coat), and (3) English words whose native word translation had a higher frequency than their loanword translation (e.g., cash). Results showed that English words whose native word translations in Korean were of high frequency were recognized faster than those with low frequency native word translations. More interestingly, the L1 native word frequency effect was stronger for speakers with lower English proficiency than for more proficient EFL speakers. These findings are discussed with respect to L2 lexical processing models.

Split Lexical Insertion in Parasitic Gap Constructions*

AbstractWe propose to extend the feature movement theory (Chomsky 1995; Ochi 1999; Lasnik 1999, 2001; Agbayani 1998, 2000, 2006; Agbayani & Ochi 2006) by claiming that the separation of formal features (FF) from the rest of a lexical item occurs not only in syntactic movement but also in the course of lexical insertion. Applying this hypothesis to Parasitic Gap constructions, we argue that the FF and the rest of the lexical item are merged into the Parasitic Gap site and the real gap site, respectively. The split lexical insertion model straightforwardly accounts for the major properties of these constructions without appealing to sideward movement (Nunes 2001, 2004) or to additional assumptions about the featural content of the split portions of the lexical item under fission (Assmann 2012/Himmelreich 2017). An interesting result of this proposal is that it places parasitic gap constructions under a general theory of movement and lexical insertion.

Grammatical Constructions of Time and Date Nominations in the Russian and Chinese Languages

The work is the comparative study of lexical and grammatical means of expressing the idea of time in the Russian and Chinese languages. The main research method is the reverse reconstruction method, in which the studied grammatical model of the Russian language is first translated into Chinese, then the Chinese language model is sequentially reconstructed word for word in Russian while preserving the grammatical features of the Chinese language. The result of the work is an opportunity for the researcher (including the teacher of the Russian language who works in the Chinese audience and does not speak the students’ native language) to understand how the Russian lexical and grammatical model is seen by the native speakers of the Chinese language. With the help of this method the prepositionalcase constructions with the meaning of time (89 units) were investigated, the similarities and differences in the grammatical structure of these models were established. The textbooks and the teaching aids on Russian as a foreign language became the source of definition of these models. All studied lexical and grammatical models are combined into a parallel catalog created by reverse reconstruction and showing the zones of grammatical discrepancies when expressing the idea of time in two languages.

Hierarchy, Not Lexical Regularity, Modulates Low-Frequency Neural Synchrony During Language Comprehension.

Neural responses appear to synchronize with sentence structure. However, researchers have debated whether this response in the delta band (0.5-3 Hz) really reflects hierarchical information or simply lexical regularities. Computational simulations in which sentences are represented simply as sequences of high-dimensional numeric vectors that encode lexical information seem to give rise to power spectra similar to those observed for sentence synchronization, suggesting that sentence-level cortical tracking findings may reflect sequential lexical or part-of-speech information, and not necessarily hierarchical syntactic information. Using electroencephalography (EEG) data and the frequency-tagging paradigm, we develop a novel experimental condition to tease apart the predictions of the lexical and the hierarchical accounts of the attested low-frequency synchronization. Under a lexical model, synchronization should be observed even when words are reversed within their phrases (e.g., "sheep white grass eat" instead of "white sheep eat grass"), because the same lexical items are preserved at the same regular intervals. Critically, such stimuli are not syntactically well-formed; thus a hierarchical model does not predict synchronization of phrase- and sentence-level structure in the reversed phrase condition. Computational simulations confirm these diverging predictions. EEG data from N = 31 native speakers of Mandarin show robust delta synchronization to syntactically well-formed isochronous speech. Importantly, no such pattern is observed for reversed phrases, consistent with the hierarchical, but not the lexical, accounts.

Assessing receptive vocabulary using state‑of‑the‑art natural language processing techniques

Abstract Semantic embedding approaches commonly used in natural language processing such as transformer models have rarely been used to examine L2 lexical knowledge. Importantly, their performance has not been contrasted with more traditional annotation approaches to lexical knowledge. This study used NLP techniques related to lexical annotations and semantic embedding approaches to model the receptive vocabulary of L2 learners based on their lexical production during a writing task. The goal of the study is to examine the strengths and weaknesses of both approaches in understanding L2 lexical knowledge. Findings indicate that transformer approaches based on semantic embeddings outperform linguistic annotations and Word2vec models in predicting L2 learners’ vocabulary scores. The findings help to support the strength and accuracy of semantic-embedding approaches as well as their generalizability across tasks when compared to linguistic feature models. Limitations to semantic-embedding approaches, especially interpretability, are discussed.

CONTINUUM OF PARAPHRASING – SYNTAX SYNONYMY IN THE DICHOTOMY LANGUAGE IS SPEECH (on the material of modern French artistic prose)

This paper is dedicated to the studies of syntactical synonymy as one of kinds of periphrasing in the dichotomy language/ discourse and theory of mental kinetism. It is establishedthat in the process of transformation in language are formed the one basic (with one terminal chain) and two basic (with two or more terminal chains)synonymic periphrases of pivotal (primary) syntagm or proposition, that are actualized in discourse in the form of co(n)textually preferential options. On the bases of syntactical and semantic valence of elements of synonymic constructionsare defined the key lexemes of each option which form with peripheral lexical components certain models of synonymic derivational trees (nominal, verbal, adjectival etc.). As the result of studies the virtual terminal chains of discourse innovations-synonymic structures are reconstructed and the hierarchical and horizontal semantic and syntactical relations between them are established in the functional-semantic field of syntactical synonymy.

Identifying the Perceived Severity of Patient-Generated Telemedical Queries Regarding COVID: Developing and Evaluating a Transfer Learning-Based Solution.

BackgroundTriage of textual telemedical queries is a safety-critical task for medical service providers with limited remote health resources. The prioritization of patient queries containing medically severe text is necessary to optimize resource usage and provide care to those with time-sensitive needs.ObjectiveWe aim to evaluate the effectiveness of transfer learning solutions on the task of telemedical triage and provide a thorough error analysis, identifying telemedical queries that challenge state-of-the-art natural language processing (NLP) systems. Additionally, we aim to provide a publicly available telemedical query data set with labels for severity classification for telemedical triage of respiratory issues.MethodsWe annotated 573 medical queries from 3 online health platforms: HealthTap, HealthcareMagic, and iCliniq. We then evaluated 6 transfer learning solutions utilizing various text-embedding strategies. Specifically, we first established a baseline using a lexical classification model with term frequency–inverse document frequency (TF-IDF) features. Next, we investigated the effectiveness of global vectors for text representation (GloVe), a pretrained word-embedding method. We evaluated the performance of GloVe embeddings in the context of support vector machines (SVMs), bidirectional long short-term memory (bi-LSTM) networks, and hierarchical attention networks (HANs). Finally, we evaluated the performance of contextual text embeddings using transformer-based architectures. Specifically, we evaluated bidirectional encoder representation from transformers (BERT), Bio+Clinical-BERT, and Sentence-BERT (SBERT) on the telemedical triage task.ResultsWe found that a simple lexical model achieved a mean F1 score of 0.865 (SD 0.048) on the telemedical triage task. GloVe-based models using SVMs, HANs, and bi-LSTMs achieved a 0.8-, 1.5-, and 2.1-point increase in the F1 score, respectively. Transformer-based models, such as BERT, Bio+Clinical-BERT, and SBERT, achieved a mean F1 score of 0.914 (SD 0.034), 0.904 (SD 0.041), and 0.917 (SD 0.037), respectively. The highest-performing model, SBERT, provided a statistically significant improvement compared to all GloVe-based and lexical baselines. However, no statistical significance was found when comparing transformer-based models. Furthermore, our error analysis revealed highly challenging query types, including those with complex negations, temporal relationships, and patient intents.ConclusionsWe showed that state-of-the-art transfer learning techniques work well on the telemedical triage task, providing significant performance increase over lexical models. Additionally, we released a public telemedical triage data set using labeled questions from online medical question-and-answer (Q&A) platforms. Our analysis highlights various avenues for future works that explicitly model such query challenges.