Document Classification Model Research Articles

Deep learning models can predict violence and threats against healthcare providers using clinical notes

Violence, verbal abuse, threats, and sexual harassment of healthcare providers by patients is a major challenge for healthcare organizations around the world, contributing to staff turnover, distress, absenteeism, reduced job satisfaction, and worsening mental and physical health. To enable interventions prior to possible violent episodes, we trained two deep learning models to predict violence against healthcare workers 3 days prior to violent events for case and control patients. The first model is a document classification model using clinical notes, and the second is a baseline regression model using largely structured data. Our document classification model achieved an F1 score of 0.75 while our model using structured data achieved an F1 of 0.72, both exceeding the predictive performance of a psychiatry team who reviewed the same documents (0.5 F1). To aid in the explainability and understanding of risk factors for violent events, we additionally trained a named entity recognition classifier on annotations of the same corpus, which achieved an overall F1 of 0.7. This study demonstrates the first deep learning model capable of predicting violent events within healthcare settings using clinical notes, surpassing the first published baseline of human experts. We anticipate our methods can be generalized and extended to enable intervention at other hospital systems.

Development of an Internet-based Product-related Child Injury Textual Data Platform (IPCITDP) in China.

Internet-based media stories provide valuable information for emerging risks of product-related child injury prevention and control, but critical methodological challenges and high costs of data acquisition and processing restrict practical use by stakeholders. We constructed a data platform through literature reviews and multi-round research group discussions. Developed components included standard search strategies, filtering criteria, textual document classification, information extraction standards and a keyword dictionary. We used ten thousand manually labelled media stories to validate the textual document classification model, which was established using the Bidirectional Encoder Representation from Transformers (BERT). Multiple information extraction methods based on natural language processing algorithms were adopted to extract data for 29 structured variables from media stories. They were evaluated through manual validation of 1000 media stories about product-related child injury. We mapped the geographic distribution of media sources and media-reported product-related child injury events. We developed an internet-based product-related child injury textual data platform, IPCITDP, consisting of four layers - automatic data search and acquisition, data processing, data storage, and data application - concerning product-related child injury online media stories in China. Each layer occurred daily. External validation demonstrated high performance for the BERT classification model we established (accuracy = 0.9703) and the combined information extraction strategies (accuracy >0.70 for 25 variables). As of 31 December 2023, IPCITDP collected 35 275 eligible product-related child injury reports from 13 261 news media websites or social media platform accounts which were geographically located across all 31 mainland Chinese provinces and covered over 97% of prefecture-level cities. The injury cases in IPCITDP were typically reported several months or years earlier than official announcements about the product-related child injury risks. Our data platform added data concerning 15 supplementary variables that the national product-related injury surveillance system lacks. Two examples demonstrate the value of IPCITDP in supplementing existing data and providing early epidemiological detection of emerging signals concerning product-related child injury: magnetic beads and electric self-balancing scooters. Our data platform provides injury data that can support early detection of new product-related child injury characteristics and supplement existing data sources to reduce the burden of product-related injury among Chinese children.

Feature Extraction of Nearest Neighbors based on Large-scale Language Model for Document Classification

Feature Extraction of Nearest Neighbors based on Large-scale Language Model for Document Classification

Machine learning for document classification in an archive of the National Afrikaans Literary Museum and Research Centre


 
 
 
 
 
 
 
 https://dx.doi.org/10.4314/jsasa.v56i1.10
 ISSN: 1012-2796
 ©SASA 2023
 
 
 
 
 
 
 
 
 
 Most archives were established before the digital age, where hardcopies of much smaller volumes were archived. In the information age, archives struggle to accommodate the large volumes of material produced. In addition, many archives, including in South Africa, had to contend with budget cuts that reduced the number of staff available. If digital material is not archived now, it creates the risk of gaps in the historical record in the future. In addition, with digital humanities gaining wider acceptance, large corpuses of digital material are needed, which archives could provide. This study’s aim was to investigate whether document classification using machine learning classifiers is feasible in a South African archive context, with a focus on the National Afrikaans Literary Museum and Research Centre (NALN). The researchers created and trained a document classification model and tested it for accuracy against human classifiers. It followed a basic linguistic approach to prepare specific text documents for text classification, in terms of Galloway and Roux’s (2019) six categories, namely articles, media reports, books, interviews, reviews, and dissertations and theses. The classification was done using two annotators, after which the annotated corpus was employed as training data for machine learning models. Following Rolan et al. (2018), Suominen (2019), and Connelly et al. (2020), Python libraries were used for document classifications. The researchers show that machine learning classifiers can accurately categorise documents into different types. If implemented, this means that archives can improve their collection efforts without spending more on salaries. One way of coping with the information explosion is to develop metadata generation tools, like machine learning and artificial intelligence. If metadata could be automatically generated, it would reduce the pressure on archival personnel by providing a way to handle larger volumes.

A Fuzzy-Based Emotion Detection Method to Classify the Relevance of Pleasant/Unpleasant Emotions Posted by Users in Reviews of Service Facilities

Many sentiment analysis methods have been proposed recently to evaluate, through the Web, the perceptions of users and their satisfaction with the use of products and services; these approaches have been applied in various fields in which it is necessary to evaluate, for example, the degree of appreciation of a product or a service or political orientations or emotional states following an event or the occurrence of a phenomenon. On the other hand, these methods are based on natural language processing models needed to capture information hidden in comments, which generally require a high computational cost which can affect their performance; for this reason, review-collecting providers prefer to synthetically evaluate user satisfaction by considering a score on a numerical scale entered by users. To overcome this criticality, we propose an emotion detection method based on a light fuzzy-based document classification model to capture the relevance of pleasant and unpleasant emotions expressed by users in their reviews of service facilities. This method is implemented in a geo-computational framework and tested to evaluate the satisfaction of customers of theater venues located in the municipality of Naples (Italy). A fuzzy-based approach is used to classify user satisfaction according to the relevance of the emotional categories of pleasant and unpleasant. We show that our emotion detection method refines service feature pleasure assessments expressed on scales by users in their reviews.

Analyzing the potential of active learning for document image classification

Deep learning has been extensively researched in the field of document analysis and has shown excellent performance across a wide range of document-related tasks. As a result, a great deal of emphasis is now being placed on its practical deployment and integration into modern industrial document processing pipelines. It is well known, however, that deep learning models are data-hungry and often require huge volumes of annotated data in order to achieve competitive performances. And since data annotation is a costly and labor-intensive process, it remains one of the major hurdles to their practical deployment. This study investigates the possibility of using active learning to reduce the costs of data annotation in the context of document image classification, which is one of the core components of modern document processing pipelines. The results of this study demonstrate that by utilizing active learning (AL), deep document classification models can achieve competitive performances to the models trained on fully annotated datasets and, in some cases, even surpass them by annotating only 15–40% of the total training dataset. Furthermore, this study demonstrates that modern AL strategies significantly outperform random querying, and in many cases achieve comparable performance to the models trained on fully annotated datasets even in the presence of practical deployment issues such as data imbalance, and annotation noise, and thus, offer tremendous benefits in real-world deployment of deep document classification models. The code to reproduce our experiments is publicly available at https://github.com/saifullah3396/doc_al.

Machine Learning Based Representative Spatio-Temporal Event Documents Classification

As the scale of online news and social media expands, attempts to analyze the latest social issues and consumer trends are increasing. Research on detecting spatio-temporal event sentences in text data is being actively conducted. However, a document contains important spatio-temporal events necessary for event analysis, as well as non-critical events for event analysis. It is important to increase the accuracy of event analysis by extracting only the key events necessary for event analysis from among a large number of events. In this study, we define important 'representative spatio-temporal event documents' for the core subject of documents and propose a BiLSTM-based document classification model to classify representative spatio-temporal event documents. We build 10,000 gold-standard training datasets to train the proposed BiLSTM model. The experimental results show that our BiLSTM model improves the F1 score by 2.6% and the accuracy by 4.5% compared to the baseline CNN model.

Exploring Final Project Trends Utilizing Nuclear Knowledge Taxonomy

The National Nuclear Energy Agency of Indonesia (BATAN) taxonomy is a nuclear competence field organized into six categories. The Polytechnic Institute of Nuclear Technology, as an institution of nuclear education, faces a challenge in organizing student publications according to the fields in the BATAN taxonomy, especially in the library. The goal of this research is to determine the most efficient automatic document classification model using text mining to categorize student final project documents in Indonesian and monitor the development of the nuclear field in each category. The kNN algorithm is used to classify documents and identify the best model by comparing Cosine Similarity, Correlation Similarity, and Dice Similarity, along with vector creation binary term occurrence and TF-IDF. A total of 99 documents labeled as reference data were obtained from the BATAN repository, and 536 unlabeled final project documents were prepared for prediction. In this study, several text mining approaches such as stem, stop words filter, n-grams, and filter by length were utilized. The number of k is 4, with Cosine-binary being the best model with an accuracy value of 97 percent, and kNN works optimally when working with binary term occurrence in Indonesian language documents when compared to TF-IDF. Engineering of Nuclear Devices and Facilities is the most popular field among students, while Management is the least preferred. However, Isotopes and Radiation are the most prominent fields in Nuclear Technochemistry. Text mining can assist librarians in grouping documents based on specific criteria. There is also the possibility of observing the evolution of each existing category based on the increase of documents and the application of similar methods in various circumstances. Because of the curriculum and courses given, the growth of each discipline of nuclear science in the study program is different and varied.

On the use of Pretrained Language Models for Legal Italian Document Classification

Document classification is helpful for law professionals to improve content browsing and retrieval. Pretrained Language Models, such as BERT, have become established for legal document classification. However, legal content is quite diversified. For example, documents vary in length from very short maxims to relatively long judgements; certain document types are rich of domain-specific expressions and can be annotated with multiple labels from domain-specific taxonomies. This paper studies to what extent existing pretrained models are suited to the legal domain. Specifically, we examine a real business case focused on Italian legal document classification. On a proprietary dataset with thousands of diversified categories (e.g., legal judgements, maxims, and legal news) we explore the use of Pretrained Language Models adapted to handle various content types. We collect both quantitative and qualitative results, highlighting best and worst cases, anomalous categories, and limitations of currently available models.

On Optimality of Long Document Classification using Deep Learning

Document classification is effective with elegant models of word numerical distributions. The word embeddings are one of the categories of numerical distributions of words from the WordNet. The modern machine learning algorithms yearn on classifying documents based on the categorical data. The context of interest on the categorical data is posed with weights and the sense and quality of the sentences is estimated for sensible classification of documents. The focus of the current work is on legal and criminal documents extracted from the popular news channels, particularly on classification of long length legal and criminal documents. Optimization is the essential instrument to bring the quality inputs to the document classification model. The existing models are studied and a feasible model for the efficient document classification is proposed. The experiments are carried out with meticulous filtering and extraction of legal and criminal records from the popular news web sites and preprocessed with WordNet and Text Processing contingencies for efficient inward for the learning framework.

CLASSIFICATION OF LARGE DOCUMENTS USING MACHINE LEARNING TECHNIQUES

Document classification is a growing interest in the research of text mining. Correctly identifying the documents into particular category is still presenting challenge because of large and vast amount of features in the dataset. In regards to the existing classifying approaches, Naive Bayes is potentially good at serving as a document classification model due to its simplicity. The aim of this paper is to highlight the performance of employing Naive Bayes in document classification. Results show that Naive Bayes is the best classifiers against several common classifiers (such as decision tree, neural network, and support vector machines) in term of accuracy and computational efficiency.

APPCorp: a corpus for Android privacy policy document structure analysis

With the increasing popularity of mobile devices and the wide adoption of mobile Apps, an increasing concern of privacy issues is raised. Privacy policy is identified as a proper medium to indicate the legal terms, such as GDPR, and to bind legal agreement between service providers and users. However, privacy policies are usually long and vague for end users to read and understand. It is thus important to be able to automatically analyze the document structures of privacy policies to assist user understanding. In this work we create a manually labelled corpus containing $167$ privacy policies (of more than $447$K words and $5,276$ annotated paragraphs). We report the annotation process and details of the annotated corpus. We also benchmark our data corpus with $4$ document classification models, thoroughly analyze the results and discuss challenges and opportunities for the research committee to use the corpus. We release our labelled corpus as well as the classification models for public access.

Highlighted Document Image Classification

There are many existing document image classification researches, but most of them are not designed for use in constrained computer resources, like printers, or focused on documents with highlighter pen marks. To enable printers to better discriminate highlighted documents, we designed a set of features in CIE Lch(a* b*) space to use along with the support vector machine. The features include two gamut-based features and six low-level color features. By first identifying the highlight pixels, and then computing the distance from the highlight pixels to the boundary of the printer gamut, the gamut-based features can be obtained. The low-level color features are built upon the color distribution information of the image blocks. The best feature subset of the existing and new features is constructed by sequential forward floating selection (SFFS) feature selection. Leave-one-out cross-validation is performed on a dataset with 400 document images to evaluate the effectiveness of the classification model. The cross-validation results indicate significant improvements over the baseline highlighted document classification model.

A Deep-Learning Based Visual Sensing Concept for a Robust Classification of Document Images under Real-World Hard Conditions.

This paper’s core objective is to develop and validate a new neurocomputing model to classify document images in particularly demanding hard conditions such as image distortions, image size variance and scale, a huge number of classes, etc. Document classification is a special machine vision task in which document images are categorized according to their likelihood. Document classification is by itself an important topic for the digital office and it has several usages. Additionally, different methods for solving this problem have been presented in various studies; their respectively reached performance is however not yet good enough. This task is very tough and challenging. Thus, a novel, more accurate and precise model is needed. Although the related works do reach acceptable accuracy values for less hard conditions, they generally fully fail in the face of those above-mentioned hard, real-world conditions, including, amongst others, distortions such as noise, blur, low contrast, and shadows. In this paper, a novel deep CNN model is developed, validated and benchmarked with a selection of the most relevant recent document classification models. Additionally, the model’s sensitivity was significantly improved by injecting different artifacts during the training process. In the benchmarking, it does clearly outperform all others by at least 4%, thus reaching more than 96% accuracy.

Design and Implementation of an Attention-Based Interpretable Model for Document Classification

With the rapid development of artificial intelligence and machine learning, deep learning technology has been successfully applied to various tasks in the field of natural language processing. However, the erroneous decisions produced by the deep learning model and the characteristics of its black box cause users to doubt its decisions. We urgently need the model to have the ability to make rationalized explanations of its decisions, that is, interpretability. In this paper, we propose an attention-based interpretable model for document classification, which aims to have a better model performance and at the same time have better interpretability. Experiments show that our classification model is close to the performance of other mainstream models. Also, the attribution score explanation provided by the interpretation method are effective.

Word-Embedding-Based Traffic Document Classification Model for Detecting Emerging Risks Using Sentiment Similarity Weight

With the increase in traffic accident rates, traffic risk detection is becoming increasingly important. Moreover, it is necessary to provide appropriate traffic information considering user locations and routes and design an analysis method accordingly. This paper proposes a word-embedding-based traffic document classification model for detecting emerging risks using a quantity termed sentiment similarity weight (SSW). The proposed method detects emerging risks by considering and classifying the importance and polarity of keywords in traffic document. Conventional sentiment analysis methods fail to utilize semantically significant keywords unless they are included in a sentiment dictionary. In this study, through word imputation using an established similarity dictionary and by widening the limited utilization range, the proposed method overcomes the disadvantage of sentiment dictionaries. The proposed method is evaluated through three tests. In the first, the similarity between keywords is measured, and thus model accuracy is evaluated. In the second test, three classifiers for emerging risk classification are compared. In the last test, emerging risk detection is assessed according to whether the proposed SSW is applied, and its effectiveness is therefore verified. The evaluation results demonstrate that the proposed traffic-related document classification model using the SSW has an f-measure of 0.907, indicating satisfactory performance. Therefore, the proposed SSW can be effectively used as a parameter in traffic-related document classification and enables the detection of emerging risks.

A Hybrid Document Features Extraction with Clustering based Classification Framework on Large Document Sets

As the size of the document collections are increasing day-by-day, finding an essential document clusters for classification problem is one of the major problem due to high inter and intra document variations. Also, most of the conventional classification models such as SVM, neural network and Bayesian models have high true negative rate and error rate for document classification process. In order to improve the computational efficacy of the traditional document classification models, a hybrid feature extraction-based document cluster approach and classification approaches are developed on the large document sets. In the proposed work, a hybrid glove feature selection model is proposed to improve the contextual similarity of the keywords in the large document corpus. In this work, a hybrid document clustering similarity index is optimized to find the essential key document clusters based on the contextual keywords. Finally, a hybrid document classification model is used to classify the clustered documents on large corpus. Experimental results are conducted on different datasets, it is noted that the proposed document clustering-based classification model has high true positive rate, accuracy and low error rate than the conventional models.

Seed-Guided Topic Model for Document Filtering and Classification

One important necessity is to filter out the irrelevant information and organize the relevant information into meaningful categories. However, developing text classifiers often requires a large number of labeled documents as training examples. Manually labeling documents is costly and time-consuming. More importantly, it becomes unrealistic to know all the categories covered by the documents beforehand. Recently, a few methods have been proposed to label documents by using a small set of relevant keywords for each category, known as dataless text classification . In this article, we propose a seed-guided topic model for the dataless text filtering and classification (named DFC). Given a collection of unlabeled documents, and for each specified category a small set of seed words that are relevant to the semantic meaning of the category, DFC filters out the irrelevant documents and classifies the relevant documents into the corresponding categories through topic influence. DFC models two kinds of topics: category-topics and general-topics . Also, there are two kinds of category-topics: relevant-topics and irrelevant-topics. Each relevant-topic is associated with one specific category, representing its semantic meaning. The irrelevant-topics represent the semantics of the unknown categories covered by the document collection. And the general-topics capture the global semantic information. DFC assumes that each document is associated with a single category-topic and a mixture of general-topics. A novelty of the model is that DFC learns the topics by exploiting the explicit word co-occurrence patterns between the seed words and regular words (i.e., non-seed words) in the document collection. A document is then filtered, or classified, based on its posterior category-topic assignment. Experiments on two widely used datasets show that DFC consistently outperforms the state-of-the-art dataless text classifiers for both classification with filtering and classification without filtering. In many tasks, DFC can also achieve comparable or even better classification accuracy than the state-of-the-art supervised learning solutions. Our experimental results further show that DFC is insensitive to the tuning parameters. Moreover, we conduct a thorough study about the impact of seed words for existing dataless text classification techniques. The results reveal that it is not using more seed words but the document coverage of the seed words for the corresponding category that affects the dataless classification performance.

An novel cluster based feature selection and document classification model on high dimension trec data

TREC text documents are complex to analyze the features its relevant similar documents using the traditional document similarity measures. As the size of the TREC repository is increasing, finding relevant clustered documents from a large collection of unstructured documents is a challenging task. Traditional document similarity and classification models are implemented on homogeneous TREC data to find essential features for document entities that are similar to the TREC documents. Also, most of the traditional models are applicable to limited text document sets for text analysis. The main issues in the traditional text mining models in TREC repository include :1) Each document is represented in vector form with many sparsity values 2) Failed to find the document semantic similarity between the intra and inter clusters 3) High mean squared error rate. In this paper, novel feature selection based clustered and classification model is proposed on large number of different TREC repositories. Traditional latent Semantic Indexing and document clustering models are failed to find the topic relevance on large number of TREC clinical text document sets due to computational memory and time. Proposed document feature selection and clustered based classification model is applied on TREC clinical benchmark datasets. From the experimental results, it is proved that the proposed model is efficient than the existing models in terms of computational memory, accuracy and error rate are concerned.

Simultaneous Learning of Sentence Clustering and Class Prediction for Improved Document Classification

In document classification it is common to represent a document as the so called bag-of-words form, which is essentially a global term distribution indicating how often certain terms appear in a text. Ignoring the spatial statistics (i.e., where in a text they appear) can potentially lead to a suboptimal solution. The key motivation or assumption in this paper is that there may exist underlying segmentation of sentences in a document, and perhaps this partitioning might be intuitively appealing (e.g., each group corresponds to a particular sentiment or gist of arguments). If the segmentation is known somehow, terms belonging to the same/different groups can potentially be treated in an equal/different manner for classification. Based on the idea, we build a novel document classification model comprised of two parts: a sentence tagger that predicts the group labels of sentences, and a classifier that forms the input features as a weighted term frequency vector that is aggregated from all sentences but weighed differently cluster-wise according to the prediction in the first model. We suggest an efficient learning strategy for this model. For several benchmark document classification problems, we demonstrate that the proposed approach yields significantly improved classification performance over several existing algorithms.