Annotated Corpus Research Articles

CFERE: Multi-type Chinese financial event relation extraction

Extracting various types of event relations in financial texts can benefit many downstream applications supporting financial analysis. This paper addresses the multi-type event relation extraction problem in the finance domain focusing on handling several issues in existing studies, including (1) limited event relation types involved, (2) insufficient feasibility when handling non-annotated data, (3) ineffectiveness in recognizing multi-type event relation, and (4) the asynchronous event and event relation extraction process. To tackle these limitations, we carefully define six types of event relations based on the characteristics of financial texts (e.g., abundant numerical words) and further devise an integral framework for Chinese financial event relation extraction. The framework is capable of handling unsupervised event extraction and event relation recognition jointly. Specifically, according to linguistic characteristics, a Core Verb Chain is employed for the event identification. Then, by constructing a Syntactic Semantic Dependency Parsing graph, scattered events are combined into pairs, and event ellipsis elements can be completed to prevent event information loss. Also, to capture more sentence semantics, we formulate an Event Restore module that converts the structured event pairs into event restore sentence pairs and pour the pairs into the BERT model for relation type identification. Finally, to enhance the embeddings for event core elements, an Event Core Embeddings layer is augmented in BERT, and we fine-tune the model on our annotated financial corpus. Extensive experiments are conducted to validate the effectiveness of our method, and the results confirm its superiority over the state-of-the-art baselines.

Automatic recognition and classification of future work sentences from academic articles in a specific domain

Future work sentences (FWS) are the particular sentences in academic papers that contain the author's description of their proposed follow-up research direction. This paper presents methods to automatically extract FWS from academic papers and classify them according to the different future directions embodied in the paper's content. FWS recognition methods will enable subsequent researchers to locate future work sentences more accurately and quickly and reduce the time and cost of acquiring the corpus. At the same time, changes in the content of future work will be illuminated, and a foundation will be laid for a more in-depth semantic analysis of future work sentences. The current work on automatic identification of future work sentences is relatively small, and the existing research cannot accurately identify FWS from academic papers, and thus cannot conduct data mining on a large scale. Furthermore, there are many aspects to the content of future work, and the subdivision of the content is conducive to the analysis of specific development directions. In this paper, Nature Language Processing (NLP) is used as a case study, and FWS are extracted from academic papers and classified into different types. We manually build an annotated corpus with six different types of FWS. Then, automatic recognition and classification of FWS are implemented using machine learning models, and the performance of these models is compared based on the evaluation metrics. The results show that the Bernoulli Bayesian model has the best performance in the automatic recognition task, with the Macro F1 reaching 90.73%, and the SCIBERT model has the best performance in the automatic classification task, with the weighted average F1 reaching 72.63%. Finally, we extract keywords from FWS and gain a deep understanding of the key content described in FWS, and we also demonstrate that content determination in FWS will be reflected in the subsequent research work by measuring the similarity between future work sentences and the abstracts.

Unsilencing colonial archives via automated entity recognition

PurposeThis paper aims to expand the scope and mitigate the biases of extant archival indexes.Design/methodology/approachThe authors use automatic entity recognition on the archives of the Dutch East India Company to extract mentions of underrepresented people.FindingsThe authors release an annotated corpus and baselines for a shared task and show that the proposed goal is feasible.Originality/valueColonial archives are increasingly a focus of attention for historians and the public, broadening access to them is a pressing need for archives.

Building an annotated corpus for automatic metadata extraction from multilingual journal article references.

Bibliographic references containing citation information of academic literature play an important role as a medium connecting earlier and recent studies. As references contain machine-readable metadata such as author name, title, or publication year, they have been widely used in the field of citation information services including search services for scholarly information and research trend analysis. Many institutions around the world manually extract and continuously accumulate reference metadata to provide various scholarly services. However, manually collection of reference metadata every year continues to be a burden because of the associated cost and time consumption. With the accumulation of a large volume of academic literature, several tools, including GROBID and CERMINE, that automatically extract reference metadata have been released. However, these tools have some limitations. For example, they are only applicable to references written in English, the types of extractable metadata are limited for each tool, and the performance of the tools is insufficient to replace the manual extraction of reference metadata. Therefore, in this study, we focused on constructing a high-quality corpus to automatically extract metadata from multilingual journal article references. Using our constructed corpus, we trained and evaluated a BERT-based transfer-learning model. Furthermore, we compared the performance of the BERT-based model with that of the existing model, GROBID. Currently, our corpus contains 3,815,987 multilingual references, mainly in English and Korean, with labels for 13 different metadata types. According to our experiment, the BERT-based model trained using our corpus showed excellent performance in extracting metadata not only from journal references written in English but also in other languages, particularly Korean. This corpus is available at http://doi.org/10.23057/47.

A probability distribution of dependencies in interlanguage

AbstractThe diversity of syntactic units in second language has attracted much scholarly attention. Most existing studies on syntactic diversity have focused on only a small number of syntactic structures, and it is difficult to find studies that consider the full range of syntactic dependencies present in the dataset. Based on a syntactic annotated interlanguage corpus that we constructed, this paper is a quantitative study of dependencies in English-speaking Chinese learners’ interlanguage across proficiency levels. We fit the frequency distributions of dependency type, word class (both as dependent and governor), verb as a governor, and noun as a dependent with a modified right-truncated Zipf-Alekseev distribution and Zipf’s law. Our findings show that: (1) from the mathematical model, interlanguage followed distributional regularities like natural languages in terms of the syntactic structure distribution; (2) most of the determination coefficients’R2were high, indicating that the investigated distributions in interlanguage fit the distributional law finding in natural languages. This also demonstrated that both interlanguages and natural languages consistently conform to the law of linguistic diversity and uniformity; (3) the dependency relation distribution parametersaandbmanifest the developmental trend of L2 learners’ proficiency levels, demonstrating that the parameters had universal applicability in reflecting interlanguage proficiency.

O corpus AMR-PT e a anotação semântica de sentenças desafiadoras de textos jornalísticos e opinativos

ABSTRACT One of the most popular semantic representation languages in Natural Language Processing (NLP) is Abstract Meaning Representation (AMR). This formalism encodes the meaning of single sentences in directed rooted graphs. For English, there is a large annotated corpus that provides qualitative and reusable data for building or improving existing NLP methods and applications. For building AMR corpora for non-English languages, including Brazilian Portuguese, automatic and manual strategies have been conducted. The automatic annotation methods are essentially based on the cross-linguistic alignment of parallel corpora and the inheritance of the AMR annotation. The manual strategies focus on adapting the AMR English guidelines to a target language. Both annotation strategies have to deal with some phenomena that are challenging. This paper explores in detail some characteristics of Portuguese for which the AMR model had to be adapted and introduces two annotated corpora: AMRNews, a corpus of 870 annotated sentences from journalistic texts, and OpiSums-PT-AMR, comprising 404 opinionated sentences in AMR.

Editors’ Introduction

Editors’ Introduction

Learning about Schools in the British Isles Through a Video Corpus: Reflections on an Online Project for Digital Literacy and Multimodal Corpus Construction

Through first-hand experiences undertaken with students using the OpenMWS platform (Taibi 2020), the paper reports on the progress made by second-year language degree students in English linguistics at the University of Messina as regards their use of online corpus construction, annotation and search tools when exploring video genres. The paper describes the ways in which participation in the Messina OVP (Online Video Project) on schools in the British Isles during and after the Covid crisis has proved beneficial for the students in question, both in terms of acquiring textual competences and as regards creating new interactive communities (Vasta and Baldry 2020; Baldry et al 2022).

Anomaly Detection in Social Media Texts Using Optimal Convolutional Neural Network

Social Networking Sites (SNSs) are nowadays utilized by the whole world to share ideas, images, and valuable contents by means of a post to reach a group of users. The use of SNS often inflicts the physical and the mental health of the people. Nowadays, researchers often focus on identifying the illegal behaviors in the SNS to reduce its negative influence. The state-of-art Natural Language processing techniques for anomaly detection have utilized a wide annotated corpus to identify the anomalies and they are often time-consuming as well as certainly do not guarantee maximum accuracy. To overcome these issues, the proposed methodology utilizes a Modified Convolutional Neural Network (MCNN) using stochastic pooling and a Leaky Rectified Linear Unit (LReLU). Here, each word in the social media text is analyzed based on its meaning. The stochastic pooling accurately detects the anomalous social media posts and reduces the chance of overfitting. The LReLU overcomes the high computational cost and gradient vanishing problem associated with other activation functions. It also doesn’t stop the learning process when the values are negative. The MCNN computes a specified score value using a novel integrated anomaly detection technique. Based on the score value, the anomalies are identified. A Teaching Learning based Optimization (TLBO) algorithm has been used to optimize the feature extraction phase of the modified CNN and fast convergence is offered. In this way, the performance of the model is enhanced in terms of classification accuracy. The efficiency of the proposed technique is compared with the state-of-art techniques in terms of accuracy, sensitivity, specificity, recall, and precision. The proposed MCNN-TLBO technique has provided an overall architecture of 97.85%, 95.45%, and 97.55% for the three social media datasets namely Facebook, Twitter, and Reddit respectively.

Word Sense Disambiguation Corpus Development for Romanian Language

Research in the area of ​​the interconnection of lexical resources represents a real challenge, because it addresses the difficult problem of semantic understanding and, more precisely, the disambiguation of the meaning of the words - Word Sense Disambiguation (WSD). In the current and prospective context of the information society, the existence of the digital format of the fundamental works of a national culture is strictly necessary. It is a topical issue throughout the world of creating a representative corpus of a language accessible through the Internet, the corpus being a concrete, clear picture of the use of that language. In this study we will describe the development of a Romanian language GOLD corpus, related to the multiple meanings existing for various words. We propose a corpus annotation standard, based on three lexical resources as follows: the Thesaurus Dictionary of the Romanian Language in electronic format (eDTLR), from which we extracted a list of words with multiple meanings; from the Reference Corpus for Contemporary Romanian Language (CoRoLa) we extracted contexts in which these words were founded and from the the Romanian WordNet (RoWN) resource, we took into account the sense meaning of the word from the corpus context.

Annotation of person marking constructions in the Corpus of Hungarian Lyrical Poetry

This paper presents the annotation scheme for the manual annotation of the Corpus of Hungarian Lyrical Poetry. The corpus will consist of 400-600 annotated texts, grouped into four sub-corpora: 20th century lyrical texts from the canon of Hungarian public education, contemporary lyrical texts, slam poetry texts, and song lyrics. The manual annotation is based on automatically generated and manually checked annotations of lemmas, parts of speech and morphosyntactic features. The manual annotation of syntactic properties proposed in the annotation scheme follows a dependency analysis approach and allows us to obtain quantitative data on person marking constructions in Hungarian lyrical texts. Besides the annotation of verb-dependent relations, the paper also presents the annotation of specific phenomena such as auxiliary verbs, vocatives, elliptical structures, and nominal predicates. The annotation scheme was tested using a test corpus of 16 texts. We also provide some examples of the types of quantitative data that can be extracted from the annotated corpus.

Corpus compilation for digital humanities in lower– resourced languages: A practical look at compiling thematic digital media corpora in Serbian, Croatian and Slovenian

The digital era has unlocked unprecedented possibilities of compiling corpora of social discourse, which has brought corpus linguistic methods into closer interaction with other methods of discourse analysis and the humanities. Even when not using any specific techniques of corpus linguistics, drawing on some sort of corpus is increasingly resorted to for empirically–grounded social–scientific analysis (sometimes dubbed ‘corpus–assisted discourse analysis’ or ‘corpus–based critical discourse analysis’, cf. Hardt–Mautner 1995; Baker 2016). In the post–Yugoslav space, recent corpus developments have brought table–turning advantages in many areas of discourse research, along with an ongoing proliferation of corpora and tools. Still, for linguists and discourse analysts who embark on collecting specialized corpora for their own research purposes, many questions persist – partly due to the fast–changing background of these issues, but also due to the fact that there is still a gap in the corpus method, and in guidelines for corpus compilation, when applied beyond the anglophone contexts. In this paper we aim to discuss some possible solutions to these difficulties, by presenting one step–by–step account of a corpus building procedure specifically for Croatian, Serbian and Slovenian, through an example of compiling a thematic corpus from digital media sources (news articles and reader comments). Following an overview of corpus types, uses and advantages in social sciences and digital humanities, we present the corpus compilation possibilities in the South Slavic language contexts, including data scraping options, permissions and ethical issues, the factors that facilitate or complicate automated collection, and corpus annotation and processing possibilities. The study shows expanding possibilities for work with the given languages, but also some persistently grey areas where researchers need to make decisions based on research expectations. Overall, the paper aims to recapitulate our own corpus compilation experience in the wider context of South–Slavic corpus linguistics and corpus linguistic approaches in the humanities more generally

Digital surveillance in Latin American diseases outbreaks: information extraction from a novel Spanish corpus

BackgroundIn order to detect threats to public health and to be well-prepared for endemic and pandemic illness outbreaks, countries usually rely on event-based surveillance (EBS) and indicator-based surveillance systems. Event-based surveillance systems are key components of early warning systems and focus on fast capturing of data to detect threat signals through channels other than traditional surveillance. In this study, we develop Natural Language Processing tools that can be used within EBS systems. In particular, we focus on information extraction techniques that enable digital surveillance to monitor Internet data and social media.ResultsWe created an annotated Spanish corpus from ProMED-mail health reports regarding disease outbreaks in Latin America. The corpus has been used to train algorithms for two information extraction tasks: named entity recognition and relation extraction. The algorithms, based on deep learning and rules, have been applied to recognize diseases, hosts, and geographical locations where a disease is occurring, among other entities and relations. In addition, an in-depth analysis of micro-average F1 metrics shows the suitability of our approaches for both tasks.ConclusionsThe annotated corpus and algorithms presented could leverage the development of automated tools for extracting information from news and health reports written in Spanish. Moreover, this framework could be useful within EBS systems to support the early detection of Latin American disease outbreaks.

Aspectuo-Temporal Underspecification in Anindilyakwa: Descriptive, Theoretical, Typological and Quantitative Issues

Many so-called ‘zero tense’-marked (which we define as morphologically reduced and underspecified inflections) or untensed verb forms found in tenseless languages, have been characterized as context dependent for their temporal and aspectual interpretation, with the verb’s aspectual content (either as event structure or viewpoint properties) being given more or less prominent roles in their temporal anchoring. In this paper, we focus on a morpho-phonologically reduced inflectional verbal paradigm in Anindilyakwa (Groote Eylandt archipelago, NT, Australia), which is both temporally and aspectually underspecified, and constitutes an instance of zero tense as defined above. On the basis of a quantitative study of an annotated corpus of zero-inflected utterances, we establish that in the absence of independent overt or covert temporal information, the temporal anchoring of this ‘zero tense’ exhibits complex patterns of sensitivity to event structural parameters. Notably we establish that while dynamicity/stativity and telicity/atelicity are to some extent valuable predictors for the temporal interpretation of zero tense in Anindilyakwa, only atomicity (i.e., event punctuality) and boundedness categorically impose a past temporal anchoring—this confirms insights found in previous works, both on Anindilyakwa and on other languages, while also differing from other generalisations contained in these works. Our analysis also shows that unlike several zero tenses identified in various languages (especially in Pidgins and Creoles), Anindilyakwa zero tense-marked dynamic utterances do not correlate with a past temporal reading. Rather, we show that Anindilyakwa seems to come closest to languages possessing zero tensed-verbs (or tenseless verbs) where boundedness monotonically enforces a past temporal anchoring, such as Navajo and Mandarin Chinese. We also show that aspect-independent temporal information appears to determine the temporal anchoring of all zero tense-marked unbounded atelic utterances (both stative and dynamic) in Anindilyakwa—a fact at once conflicting with some claims made in previous works on zero tenses, while confirming results from past studies of Indigenous languages of the Americas (especially Yucatec Maya), concerning the role of temporal anaphora in the temporal interpretation of ‘tenseless’ verb forms.

Development of a baseline model for MAX/MXene synthesis recipes extraction via pre-trained model with domain knowledge

Due to their unique combination of metallic- and ceramic-like properties, MAX phases have attracted a lot of attentions. By selectively etching A-site atoms, MXenes with unique two-dimensional structures can be potentially generated. Due to their extraordinary properties, MXenes have currently made their way to the forefront of various research areas including electronics, photonics and catalysis. Therefore, the development of novel synthesis strategies for MAX/MXene is a key issue for the further development of MAX/MXene. Distilling insights from scientific literatures could accelerate the exploration of novel synthesis recipes; however, manually extracting scattered information from thousands of journal articles is laborious. In this study, we present an annotated corpus incorporating domain knowledge about MAX/MXene synthesis processes, deriving from experimental sections within 110 papers on MAX/MXene research; and based on that, a baseline model (including named entity recognition (NER) and relation extraction (RE) parts) is proposed for distilling information about MAX/MXene synthesis conditions from literatures using pre-trained natural language processing (NLP) models. We also demonstrate the efficacy of the proposed pipeline owning to the joint effort of domain knowledge (about MAX/MXene) and machine learning; where the entity recognition model possessing optimized setting could detect the entities with F1 score of 0.8452, and for relation extraction model with F1 score of 0.8476. It is hoped that the current work would provide an auxiliary for the future research and development of novel MAX/MXenes. In addition, the developed model could serve as a pre-trained model of MAX/MXenes synthesis routes extraction for future data augment.

Distinguishing selection pressures in an evolving communication system: Evidence from color-naming in “cross signing”

Cross-signing—the emergence of an interlanguage between users of different sign languages—offers a rare chance to examine the evolution of a natural communication system in real time. To provide an insight into this process, we analyse an annotated video corpus of 340 minutes of interaction between signers of different language backgrounds on their first meeting and after living with each other for several weeks. We focus on the evolution of shared color terms and examine the role of different selectional pressures, including frequency, content, coordination and interactional context. We show that attentional factors in interaction play a crucial role. This suggests that understanding meta-communication is critical for explaining the cultural evolution of linguistic systems.

CafeteriaSA corpus: scientific abstracts annotated across different food semantic resources

In the last decades, a great amount of work has been done in predictive modeling of issues related to human and environmental health. Resolution of issues related to healthcare is made possible by the existence of several biomedical vocabularies and standards, which play a crucial role in understanding the health information, together with a large amount of health-related data. However, despite a large number of available resources and work done in the health and environmental domains, there is a lack of semantic resources that can be utilized in the food and nutrition domain, as well as their interconnections. For this purpose, in a European Food Safety Authority-funded project CAFETERIA, we have developed the first annotated corpus of 500 scientific abstracts that consists of 6407 annotated food entities with regard to Hansard taxonomy, 4299 for FoodOn and 3623 for SNOMED-CT. The CafeteriaSA corpus will enable the further development of natural language processing methods for food information extraction from textual data that will allow extracting food information from scientific textual data. Database URL: https://zenodo.org/record/6683798#.Y49wIezMJJF.

Voiceprint and machine learning models for early detection of bulbar dysfunction in ALS

Background and Objective: Bulbar dysfunction is a term used in amyotrophic lateral sclerosis (ALS). It refers to motor neuron disability in the corticobulbar area of the brainstem which leads to a dysfunction of speech and swallowing. One of the earliest symptoms of bulbar dysfunction is voice deterioration characterized by grossly defective articulation, extremely slow laborious speech, marked hypernasality and severe harshness. Recently, research efforts have focused on voice analysis to capture this dysfunction. The main aim of this paper is to provide a new methodology to diagnose this dysfunction automatically at early stages of the disease, earlier than clinicians can do.Methods: The study focused on the creation of a voiceprint consisting of a pattern generated from the quasi-periodic components of a steady portion of the five Spanish vowels and the computation of the five principal and independent components of this pattern. Then, a set of statistically significant features was obtained using multivariate analysis of variance and the outcomes of the most common supervised classification models were obtained.Results: The best model (random forest) obtained an accuracy, sensitivity and specificity of 88.3%, 85.0% and 95.0% respectively when classifying bulbar vs. control participants but the results worsened when classifying bulbar vs. no-bulbar patients (accuracy, sensitivity and specificity of 78.7%, 80.0% and 77.5% respectively for support vector machines). Due to the great uncertainty found in the annotated corpus of the ALS patients without bulbar involvement, we used a safe semi-supervised support vector machine to relabel the ALS participants diagnosed without bulbar involvement as bulbar and no-bulbar. The performance of the results obtained increased, especially when classifying bulbar and no-bulbar patients obtaining an accuracy, sensitivity and specificity of 91.0%, 83.3% and 100.0% respectively for support vector machines. This demonstrates that our model can improve the diagnosis of bulbar dysfunction compared not only with clinicians, but also the methods published to date.Conclusions: The results obtained demonstrate the efficiency and applicability of the methodology presented in this paper. It may lead to the development of a cheap and easy-to-use tool to identify this dysfunction in early stages of the disease and monitor progress.

Semi-supervised Learning Models for Sentiment Analysis on Marketplace Dataset

Sentiment analysis aims to categorize opinions using an annotated corpus to train the model. However, building a high-quality, fully annotated corpus takes a lot of effort, time, and expense. The semi-supervised learning technique efficiently adds training data automatically from unlabeled data. The labeling process, which requires human expertise and requires time, can be helped by an SSL approach. This study aims to develop an SSL-Model for sentiment analysis and to compare the learning capabilities of Naive Bayes (NB) and Random Forest (RF) in the SSL. Our model attempts to annotate opinion documents in Indonesian. We use an ensemble multi-classifier that works on unigrams, bigrams, and trigrams vectors. Our model test uses a marketplace dataset containing rating comments scrapping from Shopee for smartphone products in the Indonesian Language. The research started with data preparation, vectorization using TF-IDF, feature extraction, modeling using Random Forest (RF) and Naïve Bayes (NB), and evaluation using Accuracy and F1-score. The performance of the NB model outperformed previous research, increasing by 5,5%. The conclusion is that SSL performance highly depends on the number of training data and the compatibility of the features or patterns in the document with machine learning. On our marketplace dataset, better to use Random Forest.

Developing a deep learning natural language processing algorithm for automated reporting of adverse drug reactions

The detection of adverse drug reactions (ADRs) is critical to our understanding of the safety and risk-benefit profile of medications. With an incidence that has not changed over the last 30 years, ADRs are a significant source of patient morbidity, responsible for 5%–10% of acute care hospital admissions worldwide. Spontaneous reporting of ADRs has long been the standard method of reporting, however this approach is known to have high rates of under-reporting, a problem that limits pharmacovigilance efforts. Automated ADR reporting presents an alternative pathway to increase reporting rates, although this may be limited by over-reporting of other drug-related adverse events.We developed a deep learning natural language processing algorithm to identify ADRs in discharge summaries at a single academic hospital centre. Our model was developed in two stages: first, a pre-trained model (DeBERTa) was further pre-trained on 1.1 million unlabelled clinical documents; secondly, this model was fine-tuned to detect ADR mentions in a corpus of 861 annotated discharge summaries. This model was compared to a version without the pre-training step, and a previously published RoBERTa model pretrained on MIMIC III, which has demonstrated strong performance on other pharmacovigilance tasks. To ensure that our algorithm could differentiate ADRs from other drug-related adverse events, the annotated corpus was enriched for both validated ADR reports and confounding drug-related adverse events using. The final model demonstrated good performance with a ROC–AUC of 0.955 (95% CI 0.933 - 0.978) for the task of identifying discharge summaries containing ADR mentions, significantly outperforming the two comparator models.