Large-scale Text Research Articles

Framing smart working in the Covid-19 era: a data driven approach

In the last years, smart working has been introduced as an agile and dynamic way of working which provides high levels of performance, productivity, and job satisfaction in a “triple-win” configuration for customers, employees, and organizations. Although the advantages and disadvantages related to these new working practices have been recognized by the literature, how these new work practices, widely adopted during the Covid-19 pandemic, have been generally perceived on a massive scale is still unclear. Based on these considerations, this work aims to carry out a big data analysis in order to frame the collective perception about smart working. A large-scale text analytics study has been conducted on Twitter from January to June 2021. The data-driven approach identified the most frequently used macro-concepts about smart working by Twitter’s users: five perceived in a positive way (smartness, work-life balance, flexible approach, productivity, innovative working environment) and five perceived negatively (tech paralysis, technostress, technology hiccups, demotivation, social isolation). In this sense, the study offers several insights by generating useful implications for researchers and professionals both in the organizational design and psychological and Human Resources Management field.

Deep learning-based approach for Arabic open domain question answering

Open-domain question answering (OpenQA) is one of the most challenging yet widely investigated problems in natural language processing. It aims at building a system that can answer any given question from large-scale unstructured text or structured knowledge-base. To solve this problem, researchers traditionally use information retrieval methods to retrieve the most relevant documents and then use answer extractions techniques to extract the answer or passage from the candidate documents. In recent years, deep learning techniques have shown great success in OpenQA by using dense representation for document retrieval and reading comprehension for answer extraction. However, despite the advancement in the English language OpenQA, other languages such as Arabic have received less attention and are often addressed using traditional methods. In this paper, we use deep learning methods for Arabic OpenQA. The model consists of document retrieval to retrieve passages relevant to a question from large-scale free text resources such as Wikipedia and an answer reader to extract the precise answer to the given question. The model implements dense passage retriever for the passage retrieval task and the AraELECTRA for the reading comprehension task. The result was compared to traditional Arabic OpenQA approaches and deep learning methods in the English OpenQA. The results show that the dense passage retriever outperforms the traditional Term Frequency-Inverse Document Frequency (TF-IDF) information retriever in terms of the top-20 passage retrieval accuracy and improves our end-to-end question answering system in two Arabic question-answering benchmark datasets.

Joint Audio-Text Model for Expressive Speech-Driven 3D Facial Animation

Speech-driven 3D facial animation with accurate lip synchronization has been widely studied. However, synthesizing realistic motions for the entire face during speech has rarely been explored. In this work, we present a joint audio-text model to capture the contextual information for expressive speech-driven 3D facial animation. The existing datasets are collected to cover as many different phonemes as possible instead of sentences, thus limiting the capability of the audio-based model to learn more diverse contexts. To address this, we propose to leverage the contextual text embeddings extracted from the powerful pre-trained language model that has learned rich contextual representations from large-scale text data. Our hypothesis is that the text features can disambiguate the variations in upper face expressions, which are not strongly correlated with the audio. In contrast to prior approaches which learn phoneme-level features from the text, we investigate the high-level contextual text features for speech-driven 3D facial animation. We show that the combined acoustic and textual modalities can synthesize realistic facial expressions while maintaining audio-lip synchronization. We conduct the quantitative and qualitative evaluations as well as the perceptual user study. The results demonstrate the superior performance of our model against existing state-of-the-art approaches.

Six dimensions describe action understanding: The ACT-FASTaxonomy.

Humans engage in a wide variety of different actions and activities. These range from simple motor actions like reaching for an object, to complex activities like governing a nation. Navigating everyday life requires people to make sense of this diversity of actions. We suggest that the mind simplifies this complex domain by attending primarily to the most essential features of actions. Using a parsimonious set of action dimensions, the mind can organize action knowledge in a low-dimensional representational space. In seven studies, we derive and validate such an action taxonomy. Study 1 uses large-scale text analyses to generate and test potential action dimensions. Study 2 validates interpretable labels for these dimensions. Studies 3-5 demonstrate that these dimensions can explain human judgments about actions. We perform model selection on data from these studies to arrive at the optimal set of six psychological dimensions, together forming the Abstraction, Creation, Tradition, Food, Animacy, Spiritualism Taxonomy (ACT-FAST). Study 6 demonstrates that ACT-FAST can predict socially relevant qualities of actions, including how, when, where, why, and by whom they are performed. Finally, Study 7 shows that ACT-FAST can explain action-related patterns of brain activity using naturalistic functional MRI (MRI). Together, these studies reveal the dimensional structure the mind applies to organize action concepts. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Integrating topic modeling and word embedding to characterize violent deaths

SignificanceWe introduce an approach to identify latent topics in large-scale text data. Our approach integrates two prominent methods of computational text analysis: topic modeling and word embedding. We apply our approach to written narratives of violent death (e.g., suicides and homicides) in the National Violent Death Reporting System (NVDRS). Many of our topics reveal aspects of violent death not captured in existing classification schemes. We also extract gender bias in the topics themselves (e.g., a topic about long guns is particularly masculine). Our findings suggest new lines of research that could contribute to reducing suicides or homicides. Our methods are broadly applicable to text data and can unlock similar information in other administrative databases.

Exploring the themes of the territory: a topic modelling approach to 40 years of publications in International Journal of Lifelong Education (1982–2021)

ABSTRACT The question of what the research of lifelong education is all about needs to be revisited from time to time. Not only is this line of research approached from a multitude of academic disciplines – such as sociology, psychology or philosophy – the very concepts that are used to denote the field also undergo important changes over time, e.g. from lifelong education to lifelong learning. In this contribution, we will explore this rather elusive research territory using a large-scale text analysis of a specific scientific journal, International Journal of Lifelong Education, based on metadata (abstracts, keywords and titles) from 1,185 articles published between 1982 to 2021. Based on topic modelling techniques, we identify the main themes that have been prevalent within the journal, and how the journal’s content has changed character over time. We end the paper with a more critical examination of what kind of political and scientific currents might help explain what has led research practices to be more descriptive, micro-oriented and work-related over time.

A Domain Specific Multi-Document Reading Comprehension Method for Artificial Intelligence Application

With the development of artificial intelligence, information retrieval and information extraction and knowledge services from large-scale texts are currently one of the most urgent needs of people. Machine reading comprehension technology is one of the key technologies that can be applied to knowledge mining. At present, multi-document reading comprehension has received a lot of attention, and its application scenarios are also very extensive. The main goal of this article is to find the answer to the question from a large number of smartphone manuals based on the questions raised by the user about the operation of the smartphone. This paper designs a pipeline structure with three modules: retrieval, extraction, and sorting. At the same time, it designs auxiliary tasks for the extraction model to improve the extraction ability, and uses a new answer scoring method to select answers. The final experiment proves that our method can effectively improve the answer's quality.

Government Responsive Selectivity and Public Limited Mediation Role in Air Pollution Governance: Evidence from Large Scale Text Data Content Mining

Government Responsive Selectivity and Public Limited Mediation Role in Air Pollution Governance: Evidence from Large Scale Text Data Content Mining

Text Embedding Augmentation Based on Retraining With Pseudo-Labeled Adversarial Embedding

Pre-trained language models (LMs) have been shown to achieve outstanding performance in various natural language processing tasks; however, these models have a significantly large number of parameters to handle large-scale text corpora during the pre-training process, and thus, they entail the risk of overfitting when fine-tuning for small task-oriented datasets is conducted. In this paper, we propose a text embedding augmentation method to prevent such overfitting. The proposed method applies augmentation to a text embedding by generating an adversarial embedding, which is not identical to original input embedding but maintaining the characteristics of the original input embedding, using PGD-based adversarial training for input text data. A pseudo-label that is identical to the label of the input text is then assigned to adversarial embedding to conduct retraining by using adversarial embedding and pseudo-label as input embedding and label pair for a separate LM. Experimental results on several text classification benchmark datasets demonstrated that the proposed method effectively prevented overfitting, which commonly occurs when adjusting a large-scale pre-trained LM to a specific task.

The (real) need for a human touch: testing a human–machine hybrid topic classification workflow on a New York Times corpus

The classification of the items of ever-increasing textual databases has become an important goal for a number of research groups active in the field of computational social science. Due to the increased amount of text data there is a growing number of use-cases where the initial effort of human classifiers was successfully augmented using supervised machine learning (SML). In this paper, we investigate such a hybrid workflow solution classifying the lead paragraphs of New York Times front-page articles from 1996 to 2006 according to policy topic categories (such as education or defense) of the Comparative Agendas Project (CAP). The SML classification is conducted in multiple rounds and, within each round, we run the SML algorithm on n samples and n times if the given algorithm is non-deterministic (e.g., SVM). If all the SML predictions point towards a single label for a document, then it is classified as such (this approach is also called a “voting ensemble"). In the second step, we explore several scenarios, ranging from using the SML ensemble without human validation to incorporating active learning. Using these scenarios, we can quantify the gains from the various workflow versions. We find that using human coding and validation combined with an ensemble SML hybrid approach can reduce the need for human coding while maintaining very high precision rates and offering a modest to a good level of recall. The modularity of this hybrid workflow allows for various setups to address the idiosyncratic resource bottlenecks that a large-scale text classification project might face.

What users seek and share in online diabetes communities: examining similarities and differences in expressions and themes

PurposeThis paper aims to investigate user health information seeking and sharing patterns and content in an online diabetes community and explore the similarities and differences in the ways and themes they expressed.Design/methodology/approachMultiple methods are applied to analyze the expressions and themes that users seek and share based on large-scale text data in an online diabetes community. First, a text classifier using deep learning method is performed based on the expression category this study developed. Second, statistical and social network analyses are used to measure the popularity and compare differences between expressions. Third, topic modeling, manual coding and similarity analysis are used to mining topics and thematic similarity between seeking and sharing threads.FindingsThere are four different ways users seek and share in online health communities (OHCs) including informational seeking, situational seeking, objective information sharing and experiential information sharing. The results indicate that threads with self-disclosure could receive more replies and attract more users to contribute. This study also examines the 10 topics that were discussed for information seeking and 14 topics for information sharing. They shared three discussion themes: self-management, medication and symptoms. Information about symptoms can be largely matched between seeking and sharing threads while there is less overlap in self-management and medication categories.Originality/valueBeing different from previous studies that mainly describe one type of health information behavior, this paper analyzes user health information seeking and sharing behaviors in OHCs and investigates whether there is a correspondence or discrepancy between expressions and information users spontaneously seek and share in OHCs.

Chinese Medical Entity Recognition Model Based on Character and Word Vector Fusion

The medical information carried in electronic medical records has high clinical research value, and medical named entity recognition is the key to extracting valuable information from large-scale medical texts. At present, most of the studies on Chinese medical named entity recognition are based on character vector model or word vector model. Owing to the complexity and specificity of Chinese text, the existing methods may fail to achieve good performance. In this study, we propose a Chinese medical named entity recognition method that fuses character and word vectors. The method expresses Chinese texts as character vectors and word vectors separately and fuses them in the model for features. The proposed model can effectively avoid the problems of missing character vector information and inaccurate word vector partitioning. On the CCKS 2019 dataset for the named entity recognition task of Chinese electronic medical records, the proposed model achieves good performance and can effectively improve the accuracy of Chinese medical named entity recognition compared with other baseline models.

EGFI: drug-drug interaction extraction and generation with fusion of enriched entity and sentence information.

The rapid growth in literature accumulates diverse and yet comprehensive biomedical knowledge hidden to be mined such as drug interactions. However, it is difficult to extract the heterogeneous knowledge to retrieve or even discover the latest and novel knowledge in an efficient manner. To address such a problem, we propose EGFI for extracting and consolidating drug interactions from large-scale medical literature text data. Specifically, EGFI consists of two parts: classification and generation. In the classification part, EGFI encompasses the language model BioBERT which has been comprehensively pretrained on biomedical corpus. In particular, we propose the multihead self-attention mechanism and packed BiGRU to fuse multiple semantic information for rigorous context modeling. In the generation part, EGFI utilizes another pretrained language model BioGPT-2 where the generation sentences are selected based on filtering rules. We evaluated the classification part on 'DDIs 2013' dataset and 'DTIs' dataset, achieving the F1 scores of 0.842 and 0.720 respectively. Moreover, we applied the classification part to distinguish high-quality generated sentences and verified with the existing growth truth to confirm the filtered sentences. The generated sentences that are not recorded in DrugBank and DDIs 2013 dataset demonstrated the potential of EGFI to identify novel drug relationships. Source code are publicly available at https://github.com/Layne-Huang/EGFI.

Should doctors use or avoid medical terms? The influence of medical terms on service quality of E-health

With epidemics and pandemics like COVID-19, many offline healthcare services have been suspended and shifted to online, where patients and doctors typically communicate by typing texts. The limited communication poses a threat to the service quality of E-health, and also raises higher demand on the language skills of doctors, in which medical terms are a common concern. Traditional studies of offline healthcare mostly hold a negative attitude towards the use of medical terms by doctors. However, should we still advise doctors to avoid using medical terms in E-health? To answer this question, this paper conducts a study combining technical and empirical analyses based on real data. In this paper, a novel unsupervised text-mining method is proposed to automatically identify medical terms with crowd wisdom from large-scale doctor-patient communication texts. Then, a TREC-type experiment is carried out to validate the proposed method in terms of Precision, Recall, and F_1-measure, demonstrating that it can identify accurate and comprehensive medical terms. Based on the identified medical terms, an empirical analysis is conducted to verify the influence of medical terms used by doctors on the service quality of E-health. The analysis results show that for patients with low health literacy, the use of medical terms by doctors would decrease their service quality. However, for patients with high health literacy, the use of medical terms by doctors can significantly increase their service quality, revealing that doctors could improve their service quality in E-health by adjusting their medical term usage according to the health literacy of patients.

A disease-specific language representation model for cerebrovascular disease research

BackgroundEffectively utilizing disease-relevant text information from unstructured clinical notes for medical research presents many challenges. BERT (Bidirectional Encoder Representation from Transformers) related models such as BioBERT and ClinicalBERT, pre-trained on biomedical corpora and general clinical information, have shown promising performance in various biomedical language processing tasks. ObjectivesThis study aims to explore whether a BERT-based model pre-trained on disease-related clinical information can be more effective for cerebrovascular disease-relevant research. MethodsThis study proposed the StrokeBERT which was initialized from BioBERT and pre-trained on large-scale cerebrovascular disease related clinical text information. The pre-trained corpora contained 113,590 discharge notes, 105,743 radiology reports, and 38,199 neurological reports. Two real-world empirical clinical tasks were conducted to validate StrokeBERT's performance. The first task identified extracranial and intracranial artery stenosis from two independent sets of radiology angiography reports. The second task predicted the risk of recurrent ischemic stroke based on patients’ first discharge information. ResultsIn stenosis detection, StrokeBERT showed improved performance on targeted carotid arteries, with an average AUC compared to that of ClinicalBERT of 0.968 ± 0.021 and 0.956 ± 0.018, respectively. In recurrent ischemic stroke prediction, after 10-fold cross-validation on 1,700 discharge information, StrokeBERT presented better prediction ability (AUC±SD = 0.838 ± 0.017) than ClinicalBERT (AUC±SD = 0.808 ± 0.045). The attention scores of StrokeBERT showed better ability to detect and associate cerebrovascular disease related terms than current BERT based models. ConclusionsThis study shows that a disease-specific BERT model improved the performance and accuracy of various disease-specific language processing tasks and can readily be fine-tuned to advance cerebrovascular disease research and further developed for clinical applications.

LitCovid-AGAC: cellular and molecular level annotation data set based on COVID-19.

Currently, coronavirus disease 2019 (COVID-19) literature has been increasing dramatically, and the increased text amount make it possible to perform large scale text mining and knowledge discovery. Therefore, curation of these texts becomes a crucial issue for Bio-medical Natural Language Processing (BioNLP) community, so as to retrieve the important information about the mechanism of COVID-19. PubAnnotation is an aligned annotation system which provides an efficient platform for biological curators to upload their annotations or merge other external annotations. Inspired by the integration among multiple useful COVID-19 annotations, we merged three annotations resources to LitCovid data set, and constructed a cross-annotated corpus, LitCovid-AGAC. This corpus consists of 12 labels including Mutation, Species, Gene, Disease from PubTator, GO, CHEBI from OGER, Var, MPA, CPA, NegReg, PosReg, Reg from AGAC, upon 50,018 COVID-19 abstracts in LitCovid. Contain sufficient abundant information being possible to unveil the hidden knowledge in the pathological mechanism of COVID-19.

Efficient Multi-Stage Conjugate Gradient for Trust Region Step

The trust region step problem, by solving a sphere constrained quadratic programming, plays a critical role in the trust region Newton method. In this paper, we propose an efficient Multi-Stage Conjugate Gradient (MSCG) algorithm to compute the trust region step in a multi-stage manner. Specifically, when the iterative solution is in the interior of the sphere, we perform the conjugate gradient procedure. Otherwise, we perform a gradient descent procedure which points to the inner of the sphere and can make the next iterative solution be a interior point. Subsequently, we proceed with the conjugate gradient procedure again. We repeat the above procedures until convergence. We also present a theoretical analysis which shows that the MSCG algorithm converges. Moreover, the proposed MSCG algorithm can generate a solution in any prescribed precision controlled by a tolerance parameter which is the only parameter we need. Experimental results on large-scale text data sets demonstrate our proposed MSCG algorithm has a faster convergence speed compared with the state-of-the-art algorithms.

Short-Form video classification based on Gate shift module and Semantic embedding

Most of the existing video classification methods are based on large-scale data training, which can better realize the recognition and classification of known categories. However, data labelling is cumbersome and most things are unknown. Therefore, the existing video classification methods fall into a data bottleneck. This paper proposes a short video classification method based on GSM and semantic embedding. It uses super large-scale text information to assist the recognition process of the video classification model. This is an important development in the classification effect of knowledge categories. Specifically, this article expands the video classification method, adds category semantic embedding in the video feature extraction process, and trains to continuously fit the word vector of the corresponding category, and then uses semantic similarity to realize the classification of unknown categories. Multi-angle comparative experiments verify the effectiveness of this model, which can achieve good classification of unknown video categories.

중국어 동등 비교문의 비교 의미역 식별

The act of comparing is a comprehensive human cognitive process based on judgments about similarities and differences in value and degree. Inheriting its complex nature, Chinese comparative structure shows various aspects worth researching in the study of Artificial Intelligence and Linguistics. In response to this, this paper uncovers the structural, distributional features of the main Chinese equative constructions (等比句) and its semantic roles, namely Comparative Elements (CEs), based on which implemented automatic CEs extraction.<BR> First, based on comparative prepositional phrase, the equative constructions are classified into four main categories: A. [和······相同]; B. [和/像······一样]; C. [和/像······一般]; D. [有/像······这麽(那麽)]. Next, structural and semantic differences, collocational features, and usage patterns for each type were revealed. For example, type A functions as a central predicate, while type D is mainly used as an adverbial phrase. ‘一般’ in type C generally has stronger metaphorical implication than ‘一样’ in type D, and tends to co-occur with preposition ‘像’. Finally, we established a rule-based CEs extraction model employing structural and lexical features, and measured its accuracy. As a result, we found a blind spot in which the model could not correctly recognize the comparative elements of SUB and DIM that are likely to appear remote from comparative prepositional phrases. This suggests that probabilistic approach is required to identify CEs that deviates from the marked construction.<BR> With these findings, this paper established the basis for automatic Comparative Elements extraction by suggesting the main types and structural features of Chinese equative constructions through quantitative analysis of the actual corpus. Besides, the extraction model presented in this paper can be used to retrieve and summarize comparative information in large-scale texts such as newspapers, emails, product reviews, and social network data.

Text-mining for Lawyers: How Machine Learning Techniques Can Advance our Understanding of Legal Discourse

Many questions facing legal scholars and practitioners can be answered only by analysing and interrogating large collections of legal documents: statutes, treaties, judicial decisions and law review articles. I survey a range of novel techniques in machine learning and natural language processing – including topic modelling, word embeddings and transfer learning – that can be applied to the large-scale investigation of legal texts