Topic Task Research Articles

Is metadata of articles about COVID-19 enough for multilabel topic classification task?

The ever-increasing volume of COVID-19-related articles presents a significant challenge for the manual curation and multilabel topic classification of LitCovid. For this purpose, a novel multilabel topic classification framework is developed in this study, which considers both the correlation and imbalance of topic labels, while empowering the pretrained model. With the help of this framework, this study devotes to answering the following question: Do full texts, MeSH (Medical Subject Heading), and biological entities of articles about COVID-19 encode more discriminative information than metadata (title, abstract, keyword, and journal name)? From extensive experiments on our enriched version of the BC7-LitCovid corpus and Hallmarks of Cancer corpus, the following conclusions can be drawn. Our framework demonstrates superior performance and robustness. The metadata of scientific publications about COVID-19 carries valuable information for multilabel topic classification. Compared to biological entities, full texts and MeSH can further enhance the performance of our framework for multilabel topic classification, but the improved performance is very limited. Database URL: https://github.com/pzczxs/Enriched-BC7-LitCovid.

Tagging Items with Emerging Tags: A Neural Topic Model Based Few-Shot Learning Approach

The tagging system has become a primary tool to organize information resources on the Internet, which benefits both users and the platforms. To build a successful tagging system, automatic tagging methods are desired. With the development of society, new tags keep emerging. The problem of tagging items with emerging tags is an open challenge for an automatic tagging system, and it has not been well studied in the literature. We define this problem as a tag-centered cold-start problem in this study and propose a novel neural topic model based few-shot learning method named NTFSL to solve the problem. In our proposed method, we innovatively fuse the topic modeling task with the few-shot learning task, endowing the model with the capability to infer effective topics to solve the tag-centered cold-start problem with the property of interpretability. Meanwhile, we propose a novel neural topic model for the topic modeling task to improve the quality of inferred topics, which helps enhance the tagging performance. Furthermore, we develop a novel inference method based on the variational auto-encoding framework for model inference. We conducted extensive experiments on two real-world datasets, and the results demonstrate the superior performance of our proposed model compared with state-of-the-art machine learning methods. Case studies also show the interpretability of the model.

Unsupervised question-retrieval approach based on topic keywords filtering and multi-task learning

Currently, the majority of retrieval-based question-answering systems depend on supervised training using question pairs. However, there is still a significant need for further exploration of how to employ unsupervised methods to improve the accuracy of retrieval-based question-answering systems. From the perspective of question topic keywords, this paper presents TFCSG, an unsupervised question-retrieval approach based on topic keyword filtering and multi-task learning. Firstly, we design the topic keyword filtering algorithm, which, unlike the topic model, can sequentially filter out the keywords of the question and can provide a training corpus for subsequent unsupervised learning. Then, three tasks are designed in this paper to complete the training of the question-retrieval model. The first task is a question contrastive learning task based on topic keywords repetition strategy, the second is questions and its corresponding sequential topic keywords similarity distribution task, and the third is a sequential topic keywords generation task using questions. These three tasks are trained in parallel in order to obtain quality question representations and thus improve the accuracy of question-retrieval task. Finally, our experimental results on the four publicly available datasets demonstrate the effectiveness of the TFCSG, with an average improvement of 7.1%, 4.4%, and 3.5% in the P@1, MAP, and MRR metrics when using the BERT model compared to the baseline model. The corresponding metrics improved by 5.7%, 3.5% and 3.0% on average when using the RoBERTa model. The accuracy of unsupervised similar question-retrieval task is effectively improved. In particular, the values of P@1, P@5, and P@10 are close, the retrieved similar questions are ranked more advance.

Survey of transformers and towards ensemble learning using transformers for natural language processing

The transformer model is a famous natural language processing model proposed by Google in 2017. Now, with the extensive development of deep learning, many natural language processing tasks can be solved by deep learning methods. After the BERT model was proposed, many pre-trained models such as the XLNet model, the RoBERTa model, and the ALBERT model were also proposed in the research community. These models perform very well in various natural language processing tasks. In this paper, we describe and compare these well-known models. In addition, we also apply several types of existing and well-known models which are the BERT model, the XLNet model, the RoBERTa model, the GPT2 model, and the ALBERT model to different existing and well-known natural language processing tasks, and analyze each model based on their performance. There are a few papers that comprehensively compare various transformer models. In our paper, we use six types of well-known tasks, such as sentiment analysis, question answering, text generation, text summarization, name entity recognition, and topic modeling tasks to compare the performance of various transformer models. In addition, using the existing models, we also propose ensemble learning models for the different natural language processing tasks. The results show that our ensemble learning models perform better than a single classifier on specific tasks.Graphical

The risks of symbiotic relationship between terrorism and the media in media coverage of russia’s military aggression against Ukraine

Introduction. The symbiotic relationship between terrorism and the media may have destructive media effects on society in peacetime, but under wartime conditions, these effects may intensify, pose new threats, and weaken the state’s position in fighting against the enemy. Relevance of the study. Identifying and analyzing these risks is a critical and relevant topic and scientific task during Russia's military aggression against Ukraine, which has significantly scaled after Russia invaded Ukraine in 2022. Therefore, the purpose of the article is to analyze the risks of the symbiotic relationship between terrorism and the media in the coverage of Russia's aggression by Ukrainian journalists. Methodology. The negative consequences of the symbiotic relationship between terrorism and the media are identified through the analysis of scientific and media criticism articles. The risks of such ties during Russia’s military aggression against Ukraine were determined by induction, reasoning by analogy, ascent from the abstract to the concrete, synthesis, and generalization. Results. The disproportionate to the overall war threat, overly dramatic and sensitive, sensationalized, detailed coverage of Russia’s terrorist practices in the Ukrainian media can lead to destructive media influence desired by the enemy: spreading fear, panic and paralyzing pessimism among Ukrainians, which turn them into a dysfunctional, disorganized society incapable of mobilizing resources under existential threats; such a society usually makes more significant emotional, irrational political pressure on the authorities making concede to terrorists; polarization of public opinion and internal political destabilization; Ukrainians who are latent supporters of Russia may perceive massive terrorist attacks as a manifestation of Russian tremendous military capability and a call to action (risks of collaboration and sabotage); distraction of public attention from other problems, such as losses on the battlefield; obtaining information from the media about the psychological state of society, countermeasures of the security forces (OSINT intelligence), etc. Conclusions. When covering Russia’s military aggression against Ukraine, Ukrainian journalists need to weaken the symbiotic relationship between terrorism and the media. They should develop a media frame for covering the Russian Federation’s terror that provides objective and comprehensive information for citizens and does not cause long-term destructive media effects on Ukrainian society, as well as indirect assistance to the enemy. Ukrainian media professionals can use the article’s results and journalistic examples to formulate or improve their editorial policy on media coverage of Russia’s terror in Ukraine.

STID-Prompt: Prompt learning for sentiment-topic-importance detection in financial news

With the development of the Internet and the financial industry, the analysis and judgement of financial news has become increasingly important. Common tasks in this area include sentiment analysis, topic classification, and importance judgement. Existing approaches use fine-tuning paradigms to address these tasks. However, the lack of labeled data in the field of financial news poses a small sample learning problem. The new paradigm represented by prompt learning provides a new way and means to improve the performance of small-sample classification. In addition, existing methods do not focus on research on sentiment, topic, and importance simultaneously, which is also an important factor in evaluating financial news. In practical applications, joint judgment of sentiment, topic, and importance is more effective. Finally, these methods require different tuning for each specific task. They do not consider relationships between individual tasks and cannot utilize information across tasks. This limits their application in complex situations. In this work, we propose a prompt-based financial news classification model (STID-Prompt) to address these issues. For the first two problems, we solve the sentiment analysis task, the topic classification task, the importance judgment task, and the 〈sentiment, topic, importance〉 classification task by designing complex prompt templates. For the third problem, we propose a unique prompt-based joint multi-task learning approach. It learns knowledge from multiple tasks and integrates it into the target task, and the multi-task learning approach further improves the performance of the model. Experimental results show the effectiveness of our approach even with less training data.

Comparing neural sentence encoders for topic segmentation across domains: not your typical text similarity task

Neural sentence encoders (NSE) are effective in many NLP tasks, including topic segmentation. However, no systematic comparison of their performance in topic segmentation has been performed. Here, we present such a comparison, using supervised and unsupervised segmentation models based on NSEs. We first compare results with baselines, showing that the use of NSEs does often provide improvements, except for specific domains such as news shows. We then compare over three different datasets a range of existing NSEs and a new NSE based on ad hoc pre-training strategy. We show that existing literature documenting general performance gains of NSEs does not always conform to the results obtained by the same NSEs in topic segmentation. If Transformers-based encoders do improve over previous approaches, fine-tuning in sentence similarity tasks or even on the same topic segmentation task we aim to solve does not always equate to better performance, as results vary across method being used and domains of application. We aim to explain this phenomenon and the relative poor performance of NSEs in news shows by considering how well different NSEs encode the underlying lexical cohesion of same-topic segments; to do so, we introduce a new metric, ARP. The results from this study suggest that good topic segmentation results do not always rely on good cohesion modelling on behalf of the segmenter and that is dependent upon what kind of text we are trying to segment. Also, it appears evident that traditional sentence encoders fail to create topically cohesive clusters of segments when used on conversational data. Overall, this work advances our understanding of the use of NSEs in topic segmentation and of the general factors determining the success (or failure) of a topic segmentation system. The new proposed metric can quantify the lexical cohesion of a multi-topic document under different sentence encoders and, as such, might have many different uses in future research, some of which we suggest in our conclusions.

다중목표 대화형 추천시스템을 위한 사전 학습된 언어모델들에 대한 성능 평가

In this paper, we examine pre-trained language models used in Multi-Goal Conversational Recommender Systems (MG-CRS), comparing and analyzing their performances of various pre-trained language models. Specifically, we investigate the impact of the sizes of language models on the performance of MG-CRS. The three types of language models of BERT, GPT2, and BART, and compare their accuracy in two tasks of 'type prediction and topic prediction on the MG-CRS dataset, DuRecDial 2.0. Experimental results show that all models demonstrate excellent performance in the type prediction task, but provide significant in performance depending on models or their sizes in the topic prediction task. Based on these findings, the study provides directions for improving the performance of MG-CRS.

Education topic in All-Russian English language Olympiad

The article considers the tasks of All-Russian English Language School Olympiad (ARELSO or Olympiad) dedicated to the topic of education. The Year of the Teacher and Mentor stimulates a keen interest in educative value of school subjects and their career guidance opportunities, and also of various contests and Olympiads for gifted and talented schoolchildren. The author emphasizes the importance of ARELSO as a means to identify schoolchildrens future vocational orientation. Outlining researchers contribution to the field of developing gifted childrens potential, and preparing them for intellectual contests and Olympiads, we have to admit the absence of full-scale research into subject matter of Olympiad tasks. The relevance of the study is determined by a thorough analysis of the 2009/20102022/2023 Olympiad tasks, and, based on the authors experience as an Olympiad tasks designer, an attempt to offer examples of municipal level tasks dedicated to the topic of education. Such tasks can be found in all parts of the Olympiad (Listening, Reading, Use of English, Writing). The author emphasizes the thematic importance of the Olympiad and the necessity of regularly designing education topic tasks with the aim of schoolchildrens vocational orientation.

Effect of Story-Continuation Writing Task on High School Students’ Writing

The present study investigates the effects of Story-Continuation Writing Task on high students’ English writing performance and attitude. A total number of 74 senior one students were randomly assigned to a Story-Continuation Class and a Topic-Writing Class with receiving 12 weeks’ instructional treatment of SCWT and Topic-Writing Task respectively and using material specially developed for the study. A pre and post English writing test were utilized to measure the effect of learners’ English writing performance. Findings demonstrated that the Story-Continuation Writing Task was superior to Topic Writing Task as significant differences existed in the posttest with the Story-Continuation Class outperforming the Topic-Writing Class, indicating that the Story-Continuation Writing Task (SCWT) had a better facilitating impact on promoting senior students’ English writing competence. The application of Story-Continuation Writing Task as an integrated writing task for high school English writing pedagogy is considered.

Robustness Analysis of Traffic Sign Recognization based on ResNet

Autonomous driving has always been an important research topic and application task of artificial intelligence, which has attracted the attention of a large number of researchers. As an important component of the environmental perception module in autonomous driving tasks, traffic sign recognition can help drivers understand road information in a timely manner and avoid potentially dangerous driving operations. To this end, accurate recognition of traffic signs is crucial from both a strategic and a practical point of view. The early recognition technology of traffic signs is mainly based on the detection of color and shape, whose recognition accuracy is limited due to the fading and deformation of traffic signs. Numerous researchers have successfully used these deep learning-based object identification algorithms for traffic sign detection and recognition thanks to the development of the Faster R-CNN and YOLO series algorithms. However, we argue that few efforts focus on the recognition performance of the model in different scenarios, which is especially important in the process of autonomous driving. Based on this observation, this paper first constructs the ResNet model. Part of the image recognition accuracy predicted by the model reached 99%. Afterward, the robustness of the model is explored by simulating complex scenes by changing illumination and noise, and it is proved that the model has good generalization ability and practical application ability.

Comment-Guided Semantics-Aware Image Aesthetics Assessment

Existing image aesthetics assessment methods mainly rely on the visual features of images but ignore their rich semantics. Nowadays, with the widespread application of social media, the comments corresponding to images in the form of texts can be easily accessed and provide rich semantic information, which can be utilized to effectively complement image features. This paper proposes a comment-guided semantics-aware image aesthetics assessment method, which is built upon a multi-task learning framework for image aesthetics prediction and comment-guided semantics classification. To assist image aesthetics assessment, we first model the semantics of an image as the topic features of its corresponding comments using Latent Dirichlet Allocation. We then propose a two-stream multitask learning framework for both topic feature prediction and aesthetic score distribution prediction. Topic feature prediction task enables to infer the semantics from images, since the comments are usually unavailable during inference and comment-guided semantics can only serve as supervision during training. We further propose to deeply fuse aesthetics and semantic features using a layerwise feature fusion method. Experimental results demonstrate that the proposed method outperforms state-of-the-art image aesthetics assessment methods.

Using keywords in the automatic classification of language of gender violence

This paper employs lexical analysis tools, quantitative processing methods, and natural language processing procedures to analyze language samples and identify lexical items that support automatic topic detection in natural language processing. This paper discusses how keyword extraction, a technique from corpus linguistics, can be employed in obtaining features that improve automatic classification; in particular, this research is concerned with extracting keywords from a corpus obtained from social networks. The corpus consists of 1,841,385 words and is subdivided into three sub-corpora that have been categorized according to the topic of the comments in each one of them. These three topics are violence against women, violence against the LGBT community, and violence in general. The corpus has been obtained by scraping comments from YouTube videos that address issues such as street harassment, femicide, feminist movements, drug trafficking, forced disappearances, equal marriage, among others. The topic detection tasks performed with the corpus extracted from the social media showed that the keywords rendered a 98% accuracy when classifying the collection of comments from 51 videos, as one of the three categories mentioned above, and 92% when classifying almost 7,500 comments individually. When keywords were removed from the classification task and all words were used to perform the classification task, accuracy dropped by an average of 17%. These results support the argument for keyword relevance in automatic topic detection.

Izindaba-Tindzaba: Machine learning news categorisation for Long and Short Text for isiZulu and Siswati

Local/Native South African languages are classified as low-resource languages. As such, it is essential to build the resources for these languages so that they can benefit from advances in the field of natural language processing. In this work, the focus was to create annotated news datasets for the isiZulu and Siswati native languages based on news topic classification tasks and present the findings from these baseline classification models. Due to the shortage of data for these native South African languages, the datasets that were created were augmented and oversampled to increase data size and overcome class classification imbalance. In total, four different classification models were used namely Logistic regression, Naive bayes, XGBoost and LSTM. These models were trained on three different word embeddings namely Bag-Of-Words, TFIDF and Word2vec. The results of this study showed that XGBoost, Logistic Regression and LSTM, trained from Word2vec performed better than the other combinations.

Topic-Oriented Dialogue Summarization

A multi-turn dialogue often contains multiple discussion topics. In several scenarios (e.g., customer service dispute, public opinion monitoring), people are only interested in the gist of a specific topic in the dialogue. Therefore, we propose a novel summarization task, i.e., Topic-Oriented Dialogue Summarization (TODS). Given a dialogue with a topic label, TODS aims to produce a summary covering the main content of the given topic in the dialogue. To model the relationship between dialogues and topics, three key abilities are needed for TODS: (1) Learning the semantic information of different topics. (2) Locating the topic-related content in the dialogue. (3) Distinguishing summaries for different topics in the same dialogue. Thus, we propose three topic-related auxiliary tasks to make the summarization model learn the three abilities above. First, the topic identification task aims at generating all the topics in the dialogue. Second, the topic attention restriction task tries to constrain the attention distribution on topic-related utterances. Third, the topic summary distinguishing task focuses on increasing the difference of summaries for different topics in the same dialogue. Experimental results on two public TODS datasets show that all auxiliary tasks are critical for TODS and help generate high-quality summaries. We also point out the expansions and challenges in TODS for future research.

ТЕМАТИЧЕСКОЕ МОДЕЛИРОВАНИЕ И СУММАРИЗАЦИЯ ТЕКСТОВ В ОБЛАСТИ КИБЕРБЕЗОПАСНОСТИ

Purpose: improving the quality of text document analysis through the use of machine learning and intellectual analysis models in summarizing and topic modeling tasks, which will reduce the load on an expert who analyzes and generalizes significant volumes of semi-structured text data on information security topics from various sources. Methods: machine learning methods were used for the operational processing and analysis of large volumes of heterogeneous ill-structured information in natural language (NL). Methods of thematic modeling and summarization of texts based on deep neural networks, including neural network language models based on the architecture of transformers, are applied. Practical relevance: the main stages of the machine procedure of thematic modeling and summarization of professional texts in the field of information security are highlighted. The results of a comparative evaluation of the effectiveness of using clustering models, latent semantic analysis, Fast Text, Text Rank language models and BERT transformers for these purposes are presented. Recommendations are given regarding the prospects for the practical application of these models as a means of intellectual support for the professional activities of cybersecurity specialists. Scientific novelty: a complex of machine learning models for thematic modeling and summarization of professional texts is proposed, based on neural network models of attachments and transformer models, characterized by an algorithm for preparing a corpus of texts for training models and the use of a learning transfer algorithm, which will increase the efficiency of analysis and generalization of domain-specific corpora of texts.

Image Captioning With Novel Topics Guidance and Retrieval-Based Topics Re-Weighting

Topic modelling (TM) has shown significant progress in boosting the effectiveness of image captioning in the last few years. Although important improvements have been shown in previous topic-guided image captioning models, some challenges remain unsolved, such as the independence of the topic predictors and the sentence generators, resulting in ineffective exploitation of semantic information. Also, all the predicted topics or the top-one topic are used throughout the whole captioning task without considering the current time step's linguistic context, which deviates the captioning network to focus on inaccurate image objects. To tackle these challenges, we propose a novel image captioning method consisting of four modules: enhanced topic predictor (ETP), retrieval-based topics re-weighting module (RTR), subsequent topic predictor (STP), and caption generation module. The prediction and generation modules are trained in an end-to-end manner to promote the efficient use of topics by predicting suitable topics at each time step. ETP predicts the topics using the image features, and is enhanced with topic embedding (TE). The RTR is only applied in the testing stage for re-weighting the topics predicted by ETP. In each time step, the STP automatically predicts concise topics subsets to alleviate the diversity of the image topics. Compared with the existing topic-based models, our model can automatically generate more accurate and diverse captions, boosting the explainability of how the topics influence the generated word in each time step. Extensive experiments on the MS-COCO and Flickr30K benchmark datasets show that our method enhances the overall image captioning's performance and the topic prediction task, and outperforms many recent image captioning approaches in terms of the evaluation metrics.

Exploring science-technology linkages: A deep learning-empowered solution

In-depth exploration of the knowledge linkages between science and technology (S&T) is an essential prerequisite for accurately understanding the S&T innovation laws, promoting the transformation of scientific outcomes, and optimizing S&T innovation policies. A novel deep learning-based methodology is proposed to investigate S&T linkages, where papers and patents are applied to represent science and technology. In order to accurately and comprehensively reveal the linkages between science and technology topics, the proposed framework combines the information of knowledge structure with textual semantics. Furthermore, the exploration analysis is also conducted from the perspective of realizing the optimal matching between science and technology topics, which can realize combinatorial optimization of the S&T knowledge systems. Specifically, science and technology networks are constructed based on Node2Vec and BERT. Then, science and technology topics are identified based on the Fast Unfolding algorithm and Z-Score index. Finally, a science-technology bipartite graph is constructed, the S&T topic linkages identification task is successfully transferred into a bipartite matching problem, and the maximum-weight matching is identified using a Kuhn-Munkres bipartite algorithm. An experiment on natural language processing demonstrates the feasibility and reliability of the proposed methodology.

An Integrated Multi-Task Model for Fake News Detection

Fake news detection attracts many researchers’ attention due to the negative impacts on the society. Most existing fake news detection approaches mainly focus on semantic analysis of news’ contents. However, the detection performance will dramatically decrease when the content of news is short. In this paper, we propose a novel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fake news detection multi-task learning (FDML)</i> model based on the following observations: 1) some certain topics have higher percentages of fake news; and 2) some certain news authors have higher intentions to publish fake news. FDML model investigates the impact of topic labels for the fake news and introduce contextual information of news at the same time to boost the detection performance on the short fake news. Specifically, the FDML model consists of representation learning and multi-task learning parts to train the fake news detection task and the news topic classification task, simultaneously. As far as we know, this is the first fake news detection work that integrates the above two tasks. The experiment results show that the FDML model outperforms state-of-the-art methods on real-world fake news dataset.

Congressional Committees, Electoral Connections, and Legislative Speech

This article examines the effects of committee specialization and district characteristics on speech participation by topic and congressional forum. It argues that committee specialization should increase speech participation during legislative debates, while district characteristics should affect the likelihood of speech participation in non-lawmaking forums. To examine these expectations, we analyze over 100,000 speeches delivered in the Chilean Chamber of Deputies between 1990 and 2018. To carry out our topic classification task, we utilize the recently developed state-of-the-art multilingual Transformer model XLM-RoBERTa. Consistent with informational theories, we find that committee specialization is a significant predictor of speech participation in legislative debates. In addition, consistent with theories purporting that legislative speech serves as a vehicle for the electoral connection, we find that district characteristics have a significant effect on speech participation in non-lawmaking forums.