Noisy Text Research Articles

Deceptively simple: An outsider's perspective on natural language processing

AbstractThis article highlights a collection of ideas with an underlying deceptive simplicity that addresses several practical challenges in computational social science and generative AI safety. These ideas lead to (1) an interpretable and quantifiable framework for political polarization; (2) a language identifier robust to noisy social media text settings; (3) a cross‐lingual semantic sampler that harnesses code‐switching; and (4) a bias audit framework that uncovers shocking racism, antisemitism, misogyny, and other biases in a wide suite of large language models.

Directional Texture Editing for 3D Models

AbstractTexture editing is a crucial task in 3D modelling that allows users to automatically manipulate the surface materials of 3D models. However, the inherent complexity of 3D models and the ambiguous text description lead to the challenge of this task. To tackle this challenge, we propose ITEM3D, a Texture Editing Model designed for automatic 3D object editing according to the text Instructions. Leveraging the diffusion models and the differentiable rendering, ITEM3D takes the rendered images as the bridge between text and 3D representation and further optimizes the disentangled texture and environment map. Previous methods adopted the absolute editing direction, namely score distillation sampling (SDS) as the optimization objective, which unfortunately results in noisy appearances and text inconsistencies. To solve the problem caused by the ambiguous text, we introduce a relative editing direction, an optimization objective defined by the noise difference between the source and target texts, to release the semantic ambiguity between the texts and images. Additionally, we gradually adjust the direction during optimization to further address the unexpected deviation in the texture domain. Qualitative and quantitative experiments show that our ITEM3D outperforms the state‐of‐the‐art methods on various 3D objects. We also perform text‐guided relighting to show explicit control over lighting. Our project page: https://shengqiliu1.github.io/ITEM3D/.

Multimodal semantic enhanced representation network for micro-video event detection

Currently, micro-videos have gained widespread acceptance as a prominent form of user-generated content across various social media platforms. Accurate event analysis of micro-videos can greatly enhance the many diverse social media platforms applications. Although some studies have shown promising results from a multimodal perspective, there is still a challenge in extracting informative cues from inaccurate modalities, particularly for text modality that is prone to inaccuracies and noise. In this paper, we propose a multimodal semantically enhanced representation network (MSERN) for micro-video event detection. To better address inaccurate and noisy text sentences, we first extract visual concepts in the form of adjective-noun pairs (ANPs), through a fine-grained common representation module, to complement the textual descriptions. To maximize the acquisition of modality-specific cues from both visual and textual modalities, we then implement a coarse-grained private representation module to ensure that private representations encompass unique facets of the modalities beyond the common perspective. Finally, because two modules will collaborate, the fine-grained common and coarse-grained private representations are integrated to ensure a reinforced micro-video representation. We evaluate our proposed method on a micro-video event detection dataset and the experimental results show a superior performance compared to the state-of-the-art methods.

Robust Blind Text Image Deblurring via Maximum Consensus Framework

The blind text image deblurring problem presents a formidable challenge, requiring the recovery of a clean and sharp text image from a blurry version with an unknown blur kernel. Sparsity-based strategies have demonstrated their efficacy by emphasizing the sparse priors of the latent image and kernel. However, these existing strategies have largely neglected the influence of additional noise, imposing limitations on their performance. To overcome this limitation, we propose a novel framework designed to effectively mitigate the impact of extensive noise prevalent in blurred images. Our approach centers around a robust Maximum Consensus Framework, wherein we optimize the quantity of interest from the noisy blurry image based on the maximum consensus criterion. Furthermore, we propose the integration of the Alternating Direction Method of Multipliers (ADMM) and the Half-Quadratic Splitting (HQS) method to address the computationally intractable L0 norm problem. This innovative strategy enables improvements in the deblurring performance of blurry text images with the additional synthetic noise. Experimental evaluations conducted on various noisy blurry text images demonstrate the superiority of the proposed approach over existing methods.

Unlocking maintenance insights in industrial text through semantic search

Maintenance records in Computerized Maintenance Management Systems (CMMS) contain valuable human knowledge on maintenance activities. These records primarily consist of noisy and unstructured texts written by maintenance experts. The technical nature of the text, combined with a concise writing style and frequent use of abbreviations, makes it difficult to be processed through classical Natural Language Processing (NLP) pipelines. Due to these complexities, this text must be normalized before feeding to classical machine learning models. Developing these custom normalization pipelines requires manual labor and domain expertise and is a time-consuming process that demands constant updates. This leads to the under-utilization of this valuable source of information to generate insights to help with maintenance decision support. This study proposes a Technical Language Processing (TLP) pipeline for semantic search in industrial text using BERT (Bidirectional Encoder Representations), a transformer-based Large Language Model (LLM). The proposed pipeline can automatically process complex unstructured industrial text and does not require custom preprocessing. To adapt the BERT model for the target domain, three unsupervised domain fine-tuning techniques are compared to identify the best strategy for leveraging available tacit knowledge in industrial text. The proposed approach is validated on two industrial maintenance records from the mining and aviation domains. Semantic search results are analyzed from a quantitative and qualitative perspective. Analysis shows that TSDAE, a state-of-the-art unsupervised domain fine-tuning technique, can efficiently identify intricate patterns in the industrial text regardless of associated complexities. BERT model fine-tuned with TSDAE on industrial text achieved a precision of 0.94 and 0.97 for mining excavators and aviation maintenance records, respectively.

Large scale annotated dataset for code-mix abusive short noisy text

Large scale annotated dataset for code-mix abusive short noisy text

Normalization of Malay Noisy Text in Social Media using Levenshtein Distance and Rule-Based Techniques

The rise of digital communication via hand phone and Internet has led to the widespread use of short-form words and abbreviations in text messaging. This trend poses challenges for data mining activities involving text processing and analysis, particularly in social media platforms where users employ a wide variety of abbreviations, slang, misspellings, and grammatical errors. To address this challenge, this study aimed to develop an algorithm for normalizing Malay noisy text using Levenshtein Distance (LD) and rule-based techniques. The LD is used to transform Malay spelling error words into their standard form, while rule-based techniques enhanced the conversion success rate for three categories of noisy term, namely slang, common Malay noisy text, and mixed language. The project was implemented using Python programming language, which demonstrated the effectiveness of the LD and rule-based techniques in normalizing noisy text in social media. The approach successfully normalized 80% of Malay noisy text into their standard text, which provides strong foundation for further study. Furthermore, this work open opportunities for introducing new approaches and rules to improve the normalization success rate, which can facilitate the analysis of text data in social media platforms. It is recommended that future studies focus on expanding the dataset and applying statistical validation methods to ensure the robustness and accuracy of the normalization model.

Implementation of Dynamic Topic Modeling to Discover Topic Evolution on Customer Reviews

Annotation and analysis of online customer reviews were identified as significant problems in various domains, including business intelligence, marketing, and e-governance. In the last decade, various approaches based on topic modeling have been developed to solve this problem. The known solutions, however, often only work well on content with static topics. As a result, it is challenging to analyze customer reviews that include dynamic and constantly expanding collections of short and noisy texts. A method was proposed to handle such dynamic content. The proposed system applied a dynamic topic model using BERTopic to monitor topics and word evolution over time. It would help decide when the topic model needs to be retrained to capture emerging topics. Several experiments were conducted to test the practicality and effectiveness of the proposed framework. It demonstrated how a dynamic topic model could handle the emergence of new and over-time-correlated topics in customer review data. As a result, improved performance was achieved compared to the baseline static topic model, with 25% of new segmented texts discovered using the dynamic topic model. Experimental results have, therefore, convincingly demonstrated that the proposed framework can be used in practice to develop automatic review annotation tools.

Hyperbolic graph convolutional neural network with contrastive learning for automated ICD coding

The International Classification of Diseases (ICD) is a widely used criterion for disease classification, health monitoring, and medical data analysis. Deep learning-based automated ICD coding has gained attention due to the time-consuming and costly nature of manual coding. The main challenges of automated ICD coding include imbalanced label distribution, code hierarchy and noisy texts. Recent works have considered using code hierarchy or description for better label representation to solve the problem of imbalanced label distribution. However, these methods are still ineffective and redundant since they only interact with a constant label representation. In this work, we introduce a novel Hyperbolic Graph Convolutional Network with Contrastive Learning (HGCN-CL) to solve the above problems and the shortcomings of the previous methods. We adopt a Hyperbolic graph convolutional network on ICD coding to capture the hierarchical structure of codes, which can solve the problem of large distortions when embedding hierarchical structure with graph convolutional network. Besides, we introduce contrastive learning for automatic ICD coding by injecting code features into text encoder to generate hierarchical-aware positive samples to solve the problem of interacting with constant code features. We conduct experiments on the public MIMIC-III and MIMIC-II datasets. The results on MIMIC III show that HGCN-CL outperforms previous state-of-art methods for automatic ICD coding, which achieves a 2.7% and 3.6% improvement respectively compared to previous best results (Hypercore). We also provide ablation experiments and hierarchy visualization to verify the effectiveness of components in our model.

Term-BLAST-like alignment tool for concept recognition in noisy clinical texts.

Methods for concept recognition (CR) in clinical texts have largely been tested on abstracts or articles from the medical literature. However, texts from electronic health records (EHRs) frequently contain spelling errors, abbreviations, and other nonstandard ways of representing clinical concepts. Here, we present a method inspired by the BLAST algorithm for biosequence alignment that screens texts for potential matches on the basis of matching k-mer counts and scores candidates based on conformance to typical patterns of spelling errors derived from 2.9 million clinical notes. Our method, the Term-BLAST-like alignment tool (TBLAT) leverages a gold standard corpus for typographical errors to implement a sequence alignment-inspired method for efficient entity linkage. We present a comprehensive experimental comparison of TBLAT with five widely used tools. Experimental results show an increase of 10% in recall on scientific publications and 20% increase in recall on EHR records (when compared against the next best method), hence supporting a significant enhancement of the entity linking task. The method can be used stand-alone or as a complement to existing approaches. Fenominal is a Java library that implements TBLAT for named CR of Human Phenotype Ontology terms and is available at https://github.com/monarch-initiative/fenominal under the GNU General Public License v3.0.

Argumentation Mining System for Corpus-based Discourse Analysis based on Structured Arguments

An Argumentation mining system can analyze a large volume of text data through a variety of sources. Nowadays it is highly useful in the areas of business, economics, and finance with digital marketing being the most promising field along with social media. It is the study of corpus-based discourse analysis that involves the automatic identification of argumentative structure in text. Initially, AM talks about extracting structured arguments from natural text, often unstructured or noisy text. Theoretical approaches of AM and pragmatic schemes that satisfy the needs of social media generated data, recognizing the need for adapting more flexible and expandable schemes, capable of adjusting to argumentation conditions that exist in social media. In this scenario it is a very challenging argumentation scheme able to identify the distinct sub-task and capture the needs of social media text, revealing the need for adopting a more flexible and extensible framework. Corpus-based Machine Learning of linguistic annotations has enabled researchers to identify repetitive linguistic patterns of language use and to uncover hidden meaning in all areas of Natural Language Processing.

Correction of whitespace and word segmentation in noisy Pashto text using CRF

Word segmentation is the process of splitting up the text into words. In English and most European languages, word boundaries are identified by whitespace, while in Pashto, there is no explicit word delimiter. Pashto uses whitespace for word separation but not consistently, and it cannot be considered a reliable word-boundary identifier. This inconsistency makes the Pashto word segmentation unique and challenging. Moreover, Pashto is a low-resource, non-standardized language with no established rules for the correct usage of whitespace that leads to two typical spelling errors, space-omission, and space-insertion. These errors significantly affect the performance of the word segmenter. This study aims to develop a state-of-the-art word segmenter for Pashto, with a proofing tool to identify and correct the position of space in a noisy text. The CRF algorithm is incorporated to train two machine learning models for these tasks. For models' training, we have developed a text corpus of nearly 3.5 million words, annotated for the correct positions of spaces and explicit word boundary information using a lexicon-based technique, and then manually checked for errors. The experimental results of the model are very satisfactory, where the F1-scores are 99.2% and 96.7% for the proofing model and word segmenter, respectively.

Local-to-Global Causal Reasoning for Cross-Document Relation Extraction

Cross-document relation extraction (RE), as an extension of information extraction, requires integrating information from multiple documents retrieved from open domains with a large number of irrelevant or confusing noisy texts. Previous studies focus on the attention mechanism to construct the connection between different text features through semantic similarity. However, similarity-based methods cannot distinguish valid information from highly similar retrieved documents well. How to design an effective algorithm to implement aggregated reasoning in confusing information with similar features still remains an open issue. To address this problem, we design a novel local-to-global causal reasoning (LGCR) network for cross-document RE, which enables efficient distinguishing, filtering and global reasoning on complex information from a causal perspective. Specifically, we propose a local causal estimation algorithm to estimate the causal effect, which is the first trial to use the causal reasoning independent of feature similarity to distinguish between confusing and valid information in cross-document RE. Furthermore, based on the causal effect, we propose a causality guided global reasoning algorithm to filter the confusing information and achieve global reasoning. Experimental results under the closed and the open settings of the large-scale dataset CodRED demonstrate our LGCR network significantly outperforms the state-of-the-art methods and validate the effectiveness of causal reasoning in confusing information processing.

Improving End-to-End Speech Translation by Leveraging Auxiliary Speech and Text Data

We present a method for introducing a text encoder into pre-trained end-to-end speech translation systems. It enhances the ability of adapting one modality (i.e., source-language speech) to another (i.e., source-language text). Thus, the speech translation model can learn from both unlabeled and labeled data, especially when the source-language text data is abundant. Beyond this, we present a denoising method to build a robust text encoder that can deal with both normal and noisy text data. Our system sets new state-of-the-arts on the MuST-C En-De, En-Fr, and LibriSpeech En-Fr tasks.

Transferable Post-hoc Calibration on Pretrained Transformers in Noisy Text Classification

Transferable Post-hoc Calibration on Pretrained Transformers in Noisy Text Classification

Context-aware Answer Selection in Community Question Answering Exploiting Spatial Temporal Bidirectional Long Short-Term Memory

Community Question Answering (CQA) sites provide knowledge sharing facility as the users can post questions and other users can share their answers. The selection of top-quality answers from the set of answers in a thread is a significant and challenging task in Natural Language Processing (NLP). To address this issue, we propose a deep learning based spatial temporal Bidirectional Long Short-Term Memory (Bi-LSTM) algorithm. The existing studies mainly focus only computing semantic similarity between questions and answers using votes given by the users. The proposed hybrid approach, based on both forward and backward, consider question to answer and answer to answer similarity. The forward LSTM captures the spatial impact of the answer to estimate the relevancy, whereas the backward LSTM learns temporal features with the answer to predict the best quality answer. Moreover, spatial Bi-LSTM captures past and future dependencies for a better understanding of context and to improve the effectiveness of answer selection. For extracting meaningful information from noisy text data, data is preprocessed following standard steps such as tokenization, parsing, lemmatization, stop words removal, part of speech tagging and entities extraction. Word embeddings-based Paragraph to vector (par2vec) has additional input nodes to represent paragraph information in vector for context understanding. The empirical analysis carried out on the SemEval CQA dataset shows that the proposed model outperforms state-of-art answer selection approaches.

Supervised Attention Network for Arbitrary-Shaped Text Detection in Edge-Fainted Noisy Scene Images

Text mining in noisy situations, like poor contrast and fainted edges, is one of the challenging areas of research in the domain of social networks and computer vision. Scene text detection is a complicated task as text regionhaving varying span in term of size, orientation, aspect ratio, color, font, and script. Furthermore, the contrast of a scene image varies drastically in noisy situations due to poor illumination and image filtering. This faints the text edges and make the task of detection more challenging. In this article, we bring forward a semantic edge supervised spatial-channel attention network, known as SESANet, for detecting arbitrary-shaped text instances in noisy scene images with faint text edges. Our network learns multiscale (MS) supervised edge semantic, pixel-wise spatial structure information, and interchannel dependencies for precisely localizing the text masks in scene images with poor contrast and illumination. Our network is efficient, precise, and fast in nature. SESANet captures rich, dense, discriminative, and MS semantic information. The experimental results show the success of the proposed network. It shows a superior performance with regard to recall on the publicly available benchmark datasets. A new dataset scene images, named as Edge-fainted Noisy Arbitrary-shaped Scene Text (EFNAST) dataset, having varying noise density, poor contrast, low illumination, and faint edges is created.

Predicting Location of Tweets Using Machine Learning Approaches

Twitter, one of the most popular microblogging platforms, has tens of millions of active users worldwide, generating hundreds of millions of posts every day. Twitter posts, referred to as “tweets”, the short and the noisy text, bring many challenges with them, such as in the case of some emergency or disaster. Predicting the location of these tweets is important for social, security, human rights, and business reasons and has raised noteworthy consideration lately. However, most Twitter users disable the geo-tagging feature, and their home locations are neither standardized nor accurate. In this study, we applied four machine learning techniques named Logistic Regression, Random Forest, Multinomial Naïve Bayes, and Support Vector Machine with and without the utilization of the geo-distance matrix for location prediction of a tweet using its textual content. Our extensive experiments on our vast collection of Arabic tweets From Saudi Arabia with different feature sets yielded promising results with 67% accuracy.

An unsupervised perplexity-based method for boilerplate removal

AbstractThe availability of large web-based corpora has led to significant advances in a wide range of technologies, including massive retrieval systems or deep neural networks. However, leveraging this data is challenging, since web content is plagued by the so-called boilerplate: ads, incomplete or noisy text and rests of the navigation structure, such as menus or navigation bars. In this work, we present a novel and efficient approach to extract useful and well-formed content from web-scraped data. Our approach takes advantage of Language Models and their implicit knowledge about correctly formed text, and we demonstrate here that perplexity is a valuable artefact that can contribute in terms of effectiveness and efficiency. As a matter of fact, the removal of noisy parts leads to lighter AI or search solutions that are effective and entail important reductions in resources spent. We exemplify here the usefulness of our method with two downstream tasks, search and classification, and a cleaning task. We also provide a Python package with pre-trained models and a web demo demonstrating the capabilities of our approach.

Automatic Knowledge Integration Method of English Translation Corpus Based on Kmeans Algorithm

Abstract We propose a feature extraction method based on the Kmeans algorithm based on the text characteristics in the English translation corpus. The article first uses a sparse autoencoder unsupervised learning method to reduce dimensionality. It then uses the Kmeans clustering algorithm for text clustering. The experimental results prove that the text features extracted by the sparse autoencoder based on the Kmeans algorithm can be used for English translation corpus knowledge clustering to achieve automatic integration. And this method can effectively solve the problems of high-dimensional, sparse, and noisy texts in the English translation corpus. The algorithm mentioned in the article can significantly improve the accuracy of the clustering results.