Text Information Extraction Research Articles

Power text information extraction based on multi-task learning

In order to improve the analysis and processing speed of power system fault text in actual business scenarios, a power fault text information automatic extraction model based on pre-training and multi-task learning is proposed. The pre-training model is used to learn the context information of power text words, and the first-order and second-order fusion features of words are mined to enhance the representation ability of features. The multi-task learning framework is used to combine the learning of named entity recognition and relationship extraction tasks to achieve mutual complementation and mutual promotion between entity recognition and relationship extraction, thereby improving the performance of power fault text information extraction. Finally, the model is verified by the daily business data of a power grid data center. Compared with other models, the accuracy and recall rate of power fault text entity recognition and relationship extraction are improved.

Three-stage binarization of color document images based on discrete wavelet transform and generative adversarial networks

The efficient extraction of text information from the background in degraded color document images is an important challenge in the preservation of ancient manuscripts. The imperfect preservation of ancient manuscripts has led to different types of degradation over time, such as page yellowing, staining, and ink bleeding, seriously affecting the results of document image binarization. This work proposes an effective three-stage network method to image enhancement and binarization of degraded documents using generative adversarial networks (GANs). Specifically, in Stage-1, we first split the input images into multiple patches, and then split these patches into four single-channel patch images (gray, red, green, and blue). Then, three single-channel patch images (red, green, and blue) are processed by the discrete wavelet transform (DWT) with normalization. In Stage-2, we use four independent generators to separately train GAN models based on the four channels on the processed patch images to extract color foreground information. Finally, in Stage-3, we train two independent GAN models on the outputs of Stage-2 and the resized original input images (512 × 512) as the local and global predictions to obtain the final outputs. The experimental results show that the Avg-Score metrics of the proposed method are 77.64, 77.95, 79.05, 76.38, 75.34, and 77.00 on the (H)-DIBCO 2011, 2013, 2014, 2016, 2017, and 2018 datasets, which are at the state-of-the-art level. The implementation code for this work is available at https://github.com/abcpp12383/ThreeStageBinarization.

Handwritten text recognition and information extraction from ancient manuscripts using deep convolutional and recurrent neural network

Handwritten text recognition and information extraction from ancient manuscripts using deep convolutional and recurrent neural network

Review of machine learning with sentimental analysis method for cross-model stock price prediction

Stock market prediction has been a popular area of research for years. Machine learning, as a fast-developing popular algorithm, is applied to stock market prediction by many previous researchers to better solve the time series involving problems over the changing prices. Different from traditional machine learning algorithms that focus on stock prices only, this paper gives a brief description and review of stock price predictions models that contain sentimental analysis over the recent paper works. Different from the prices in number format, sentimental analysis is more based on textual information extraction and mining, converting them into usable input to pass to a prediction model. This extends the prediction model's takeable input domain and strengthens the accuracy. To better classify the differences between the models, discussion and introduction are given based on different model types about whether they are traditional type or deep neural network embedded. Even though traditional types of models are more popular for sentimental analysis and neural networks perform better in prediction tasks, traditional methods are relatively easy to build or train with more explainability, compared to deep learning models suitable for larger data sets.

Estimating Leaf Area Index in Apple Orchard by UAV Multispectral Images with Spectral and Texture Information

The Leaf Area Index (LAI) strongly influences vegetation evapotranspiration and photosynthesis rates. Timely and accurately estimating the LAI is crucial for monitoring vegetation growth. The unmanned aerial vehicle (UAV) multispectral digital camera platform has been proven to be an effective tool for this purpose. Currently, most remote sensing estimations of LAIs focus on cereal crops, with limited research on economic crops such as apples. In this study, a method for estimating the LAI of an apple orchard by extracting spectral and texture information from UAV multispectral images was proposed. Specifically, field measurements were conducted to collect LAI data for 108 sample points during the final flowering (FF), fruit setting (FS), and fruit expansion (FE) stages of apple growth in 2023. Concurrently, UAV multispectral images were obtained to extract spectral and texture information (Gabor transform). The Support Vector Regression Recursive Feature Elimination (SVR-REF) was employed to select optimal features as inputs for constructing models to estimate the LAI. Finally, the optimal model was used for LAI mapping. The results indicate that integrating spectral and texture information effectively enhances the accuracy of LAI estimation, with the relative prediction deviation (RPD) for all models being greater than 2. The Categorical Boosting (CatBoost) model established for FF exhibits the highest accuracy, with a validation set R2, root mean square error (RMSE), and RPD of 0.867, 0.203, and 2.482, respectively. UAV multispectral imagery proves to be valuable in estimating apple orchard LAIs, offering real-time monitoring of apple growth and providing a scientific basis for orchard management.

AI-powered dining: text information extraction and machine learning for personalized menu recommendations and food allergy management

AI-powered dining: text information extraction and machine learning for personalized menu recommendations and food allergy management

Infrared Image Enhancement Method of Substation Equipment Based on Self-Attention Cycle Generative Adversarial Network (SA-CycleGAN)

During the acquisition of infrared images in substations, low-quality images with poor contrast, blurred details, and missing texture information frequently appear, which adversely affects subsequent advanced visual tasks. To address this issue, this paper proposes an infrared image enhancement algorithm for substation equipment based on a self-attention cycle generative adversarial network (SA-CycleGAN). The proposed algorithm incorporates a self-attention mechanism into the CycleGAN model’s transcoding network to improve the mapping ability of infrared image information, enhance image contrast, and reducing the number of model parameters. The addition of an efficient local attention mechanism (EAL) and a feature pyramid structure within the encoding network enhances the generator’s ability to extract features and texture information from small targets in infrared substation equipment images, effectively improving image details. In the discriminator part, the model’s performance is further enhanced by constructing a two-channel feature network. To accelerate the model’s convergence, the loss function of the original CycleGAN is optimized. Compared to several mainstream image enhancement algorithms, the proposed algorithm improves the quality of low-quality infrared images by an average of 10.91% in color degree, 18.89% in saturation, and 29.82% in feature similarity indices. Additionally, the number of parameters in the proposed algorithm is reduced by 37.89% compared to the original model. Finally, the effectiveness of the proposed method in improving recognition accuracy is validated by the Centernet target recognition algorithm.

Mechanobiological factors in fermentation: Understanding the impact on food texture and nutrient release

Evaluating people’s food intake is crucial for establishing the link between diet and disease. Deprivation of vital nutrients causes the body to deteriorate organs and increases the risk of severe diseases that manifest in maturity. Making healthy food choices is one of the best ways to avoid developing chronic diseases, such as diabetes, heart disease, stroke, and even certain types of cancer. Hence, this paper proposes a deep learning-based automated nutrition classification system (DL-ANCS) for predicting food ingredients and nutrition. The DL-ANCS with Internet of Things (IoT) sensors to quantify food nutrition and a smartphone app to compile ingredient nutritional information. It is possible to take pictures of food eaten using the camera that comes with most mobile phones. It is possible to automatically identify the food items for record keeping using image processing. The effectiveness of the proposed DL-ANCS relies on its ability to accurately classify food items in these photos utilizing meal prediction algorithms and DL-ANCS. This research introduces a novel approach to extracting texture information from food photos to show how these features improve the accuracy of a nutritional assessment system that runs on mobile phones. The proposed method improvesthe food texture ratio of 98.7%, effectiveness ratio of 99.2%, accuracy ratio of 95.89%, food ingredient predictionsand their nutritional compatibility ratio of 96.8%, and food component classifications ratio of 97.29%.

Introducing MagBERT: A language model for magnesium textual data mining and analysis

Magnesium (Mg) based materials hold immense potential for various applications due to their lightweight and high strength-to-weight ratio. However, to fully harness the potential of Mg alloys, structured analytics are essential to gain valuable insights from centuries of accumulated knowledge. Efficient information extraction from the vast corpus of scientific literature is crucial for this purpose. In this work, we introduce MagBERT, a BERT-based language model specifically trained for Mg-based materials. Utilizing a dataset of approximately 370,000 abstracts focused on Mg and its alloys, MagBERT is designed to understand the intricate details and specialized terminology of this domain. Through rigorous evaluation, we demonstrate the effectiveness of MagBERT for information extraction using a fine-tuned named entity recognition (NER) model, named MagNER. This NER model can extract mechanical, microstructural, and processing properties related to Mg alloys. For instance, we have created an Mg alloy dataset that includes properties such as ductility, yield strength, and ultimate tensile strength (UTS), along with standard alloy names. The introduction of MagBERT is a novel advancement in the development of Mg-specific language models, marking a significant milestone in the discovery of Mg alloys and textual information extraction. By making the pre-trained weights of MagBERT publicly accessible, we aim to accelerate research and innovation in the field of Mg-based materials through efficient information extraction and knowledge discovery.

Ancient paintings inpainting based on dual encoders and contextual information

Deep learning-based inpainting models have achieved success in restoring natural images, yet their application to ancient paintings encounters challenges due to the loss of texture, lines, and color. To address these issues, we introduce an ancient painting inpainting model based on dual encoders and contextual information to overcome the lack of feature extraction and detail texture recovery when restoring ancient paintings. Specifically, the proposed model employs a gated encoding branch that aims to minimize information loss and effectively capture semantic information from ancient paintings. A dense multi-scale feature fusion module is designed to extract texture and detail information at various scales, while dilated depthwise separable convolutions are utilized to reduce parameters and enhance computational efficiency. Furthermore, a contextual feature aggregation module is incorporated to extract contextual features, enhancing the overall consistency of the inpainting results. Finally, a color loss function is introduced to ensure color consistency in the restored area, harmonizing it with the surrounding region. The experimental results indicate that the proposed model effectively restores the texture details of ancient paintings, outperforming other methods both qualitatively and quantitatively. Additionally, the model is tested on real damaged ancient paintings to validate its practicality and efficacy.

Kinship verification in childhood images using curvelet transformed features

Kinship verification poses a formidable challenge due to significant age disparities between parents and children, resulting in diminished facial resemblances and consequently reduced model accuracy. In response, we introduce a novel methodology focusing on childhood imagery, wherein images of both parents and children during their early stages are leveraged, leading to more pronounced feature similarities. Our approach introduces a novel feature descriptor utilizing the curvelet transform (CLT) to extract facial descriptors from sub-bands, effectively capturing unique facial features. Additionally, we employ a convolutional neural network (CNN) to extract texture information, further enhancing the discriminative power of our method. Statistical measures are subsequently applied to evaluate image similarity, with a dedicated training phase utilized to determine an optimal threshold for kinship classification during the testing phase. Extensive evaluations conducted across multiple databases, including KINFACEW-I, II, FIW, TSKINFACE, UBKinface, and childhood image dataset, showcase the superior performance of our proposed model compared to existing state-of-the-art approaches. Our methodology not only enhances kinship verification accuracy but also underscores the significance of childhood images in mitigating age-related challenges within the domain.

Vocabulary Matters: An Annotation Pipeline and Four Deep Learning Algorithms for Enzyme Named Entity Recognition.

Enzymes are indispensable in many biological processes, and with biomedical literature growing exponentially, effective literature review becomes increasingly challenging. Natural language processing methods offer solutions to streamline this process. This study aims to develop an annotated enzyme corpus for training and evaluating enzyme named entity recognition (NER) models. A novel pipeline, combining dictionary matching and rule-based keyword searching, automatically annotated enzyme entities in >4800 full-text publications. Four deep learning NER models were created with different vocabularies (BioBERT/SciBERT) and architectures (BiLSTM/transformer) and evaluated on 526 manually annotated full-text publications. The annotation pipeline achieved an F1-score of 0.86 (precision = 1.00, recall = 0.76), surpassed by fine-tuned transformers for F1-score (BioBERT: 0.89, SciBERT: 0.88) and recall (0.86) with BiLSTM models having higher precision (0.94) than transformers (0.92). The annotation pipeline runs in seconds on standard laptops with almost perfect precision, but was outperformed by fine-tuned transformers in terms of F1-score and recall, demonstrating generalizability beyond the training data. In comparison, SciBERT-based models exhibited higher precision, and BioBERT-based models exhibited higher recall, highlighting the importance of vocabulary and architecture. These models, representing the first enzyme NER algorithms, enable more effective enzyme text mining and information extraction. Codes for automated annotation and model generation are available from https://github.com/omicsNLP/enzymeNER and https://zenodo.org/doi/10.5281/zenodo.10581586.

A Comprehensive Approach to Brain Tumor Classification in MRI: Unifying Classical Local Binary Patterns and Convolution Neural Networks

Brain tumors, among the most perilous neurological disorders affecting the human nervous system, are categorized into glioma, meningioma, and Pituitary types. The proposed system combines Classical Local Binary Patterns (CLBP), Histogram of Oriented Gradients (HOG), and Convolutional Neural Networks (CNN) to extract texture information from MRI images. The methodology comprises three key steps: image pre-processing, feature extraction, and classification. By evaluating a publicly accessible brain tumor dataset, the proposed approach attains an impressive accuracy of 96%. The findings highlight the potential uses of the CLBP+CNN approach in clinical diagnosis and treatment planning, demonstrating the possibility of accurate and efficient brain tumor classification. The proposal introduces future extension methods like CLBPs (DLBP, ӨLBP), where the DLBP approach incorporates a 'D' parameter specifying the distance between neighboring pixels. The ӨLBP Method evaluates the pixel value by changing the 'Ө' value, i.e., 150, 450, 900 and 1200. The classification of tumors involved the application of various classification methods, namely ANN (Artificial Neural Networks), AlDE (Ant Lion Optimizer Differential Evolution), and LDA (Linear Discriminant Analysis). This classification is executed using feature extractions derived from CLBP (DLBP & ӨLBP) applied to MRI images within the dataset.

PGCL: Prompt guidance and self-supervised contrastive learning-based method for Visual Question Answering

Recent works have demonstrated the efficacy of Chain-of-Thought (CoT), which comprises multimodal information, in multiple complex reasoning tasks. CoT, involving multiple stages of reasoning, has also been applied to Visual Question Answering (VQA) for scientific questions. Existing research on CoT in science-oriented VQA primarily concentrates on the extraction and integration of visual and textual information. However, they overlook the fact that image-question pairs, categorized by different attributes (such as subject, topic, category, skill, grade, and difficulty), emphasize distinct text information, visual information, and reasoning capabilities. Therefore, this work proposes a novel VQA method termed PGCL, founded on the prompt guidance strategy and self-supervised contrastive learning. PGCL strategically excavates and integrates text and visual information based on attribute information. Specifically, two prompt templates are first crafted. They are subsequently combined with the attribution information and the interference information of image-question pairs to generate a series of prompt positive and prompt negative samples respectively. The mining of visual and text representations is then guided by constructed prompts. These prompt-guided representations are integrated and enhanced via transformer architecture and self-supervised contrastive learning. The fused features are eventually learned to predict answers for VQA. Sufficient experiments have convincingly substantiated the individual contributions of the components within PGCL, as well as the performance of PGCL.

Medical nearest-word embedding technique implemented using an unsupervised machine learning approach for Bengali language

Abstract The rapid growth of natural language processing (NLP) applications, such as text summarization, speech recognition, information extraction, and machine translation, has led to the development of structured query language (SQL) for extracting information from structured data. However, due to limited resources, converting Natural Language (NL) queries to SQL in Bengali is challenging. This article proposes an unsupervised machine learning model to find semantically Bengali closed words that can generate SQL from NL queries in Bengali. The main objective of the proposed system is to provide support in the creation of patient-oriented explanations and educational resources by simplifying intricate medical terminology. The major findings of the proposed system are as follows: The use of machine translation in the field of medicine facilitates the dissemination of healthcare information to a diverse international audience and improves the performance of entity recognition tasks, including the identification of medical conditions, drugs, or procedures within clinical notes or electronic health data. This system allows a naive user to extract health-related information from a healthcare-structured database without any knowledge of SQL. The system accepts a query and generates a response according to the query in Bengali language. Query tokenization and stop word removal are carried out in the preprocessing stage, and unsupervised machine learning techniques are implemented to process the input query sentence. Tokenized words are converted into vectors using the skip-gram model, with noise-contrastive estimation (NCE) applied to discriminate between actual and irrelevant words. Stochastic gradient descent (SGD) optimizes the model by randomly choosing a small amount of data from the dataset and using cosine similarity to measure closer words. The semantically closer words are found using an unsupervised learning method to generate the SQL.

LRFNet: A real-time medical image fusion method guided by detail information

Multimodal medical image fusion (MMIF) technology plays a crucial role in medical diagnosis and treatment by integrating different images to obtain fusion images with comprehensive information. Deep learning-based fusion methods have demonstrated superior performance, but some of them still encounter challenges such as imbalanced retention of color and texture information and low fusion efficiency. To alleviate the above issues, this paper presents a real-time MMIF method, called a lightweight residual fusion network. First, a feature extraction framework with three branches is designed. Two independent branches are used to fully extract brightness and texture information. The fusion branch enables different modal information to be interactively fused at a shallow level, thereby better retaining brightness and texture information. Furthermore, a lightweight residual unit is designed to replace the conventional residual convolution in the model, thereby improving the fusion efficiency and reducing the overall model size by approximately 5 times. Finally, considering that the high-frequency image decomposed by the wavelet transform contains abundant edge and texture information, an adaptive strategy is proposed for assigning weights to the loss function based on the information content in the high-frequency image. This strategy effectively guides the model toward preserving intricate details. The experimental results on MRI and functional images demonstrate that the proposed method exhibits superior fusion performance and efficiency compared to alternative approaches. The code of LRFNet is available at https://github.com/HeDan-11/LRFNet.

Image Analysis and Classification Using HRSVM-CNN for Land-Cover Classification by Using Remote Sensing Images

Objective: To effectively analyze diverse satellite images and derive valuable insights, it's crucial to employ efficient methods for classification and image processing. However, due to imperfections in image formats and sensor data, satellite imagery often contains flaws and inaccuracies, requiring extra steps to enhance its quality. The proposed solution involves two key techniques: segmenting the input image using a Hybrid HRSVM-CNN and classifying the resulting high-resolution remote sensing image using a Convolutional Neural Network. This combined approach addresses the challenges posed by image inconsistencies and aims to improve the accuracy and efficiency of current methods for satellite image analysis. Methods: In this research, a high-resolution Support Vector Machine-Convolutional Neural Network (Hybrid HRSVM-CNN) and texture characteristics are used to create an automated land identification method for satellite Remote sensing (RS) images. This approach's main focus is segmentation using the Bendlet Transform and Improved Chan-Vese, and it also does classification using a Hybrid HRSVM-CNN based on feature extraction and gray-level co-occurrence matrix algorithm. Findings: The proposed classification method's accuracy was evaluated against several other classification algorithms, including Semi-Supervised Graph Based Method (SSG), Conditional Random Fields (CRF), k-Nearest Neighbor (KNN), and Bi-layer Graph-based Learning (BLGL), Support Vector Machine (SVM), and Artificial Neural Networks (ANN). When compared with existing methods, the findings of the proposed method display excellent accuracy of 98.83%. Novelty: To start, an adaptive median filter is used to pre-process the satellite Remote sensing images, removing unwanted noises and other impacts. Following pre-processing, the image is segmented using the Bendlet Transform and Improved Chan-Vese algorithms. Gray-level co-occurrence matrix is utilized to extract texture information, and the Hybrid HRSVM-CNN is then used to categorize the various types of land. These applications frequently have a variety of issues that affect the categorization accuracy. A few significant factors, like location, irregularity, form, and diameter, reduce the process's overall accuracy. This research focuses on presenting a unique Land-Cover classification model in order to address such problems. The UC Merced Land Use dataset was considered in this research. Keywords: Bendlet Transform and Improved Chan­Vese Segmentation, Hybrid HRSVM­CNN, Land detection system, Satellite remote sensing images, Adaptive median filter, Gray­ level co­occurrence matrix

Multimodal learning with only image data: A deep unsupervised model for street view image retrieval by fusing visual and scene text features of images

AbstractAs one of the classic tasks in information retrieval, the core of image retrieval is to identify the images sharing similar features with a query image, aiming to enable users to find the required information from a large number of images conveniently. Street view image retrieval, in particular, finds extensive applications in many fields, such as improvements to navigation and mapping services, formulation of urban development planning scheme, and analysis of historical evolution of buildings. However, the intricate foreground and background details in street view images, coupled with a lack of attribute annotations, render it among the most challenging issues in practical applications. Current image retrieval research mainly uses the visual model that is completely dependent on the image visual features, and the multimodal learning model that necessitates additional data sources (e.g., annotated text). Yet, creating annotated datasets is expensive, and street view images, which contain a large amount of scene texts themselves, are often unannotated. Therefore, this paper proposes a deep unsupervised learning algorithm that combines visual and text features from image data for improving the accuracy of street view image retrieval. Specifically, we employ text detection algorithms to identify scene text, utilize the Pyramidal Histogram of Characters encoding predictor model to extract text information from images, deploy deep convolutional neural networks for visual feature extraction, and incorporate a contrastive learning module for image retrieval. Upon testing across three street view image datasets, the results demonstrate that our model holds certain advantages over the state‐of‐the‐art multimodal models pre‐trained on extensive datasets, characterized by fewer parameters and lower floating point operations. Code and data are available at https://github.com/nwuSY/svtRetrieval.

DNN-HHOA: Deep Neural Network Optimization-Based Tabular Data Extraction from Compound Document Images

Text information extraction from a tabular structure within a compound document image (CDI) is crucial to help better understand the document. The main objective of text extraction is to extract only helpful information since tabular data represents the relation between text lying in a tuple. Text from an image may be of low contrast, different style, size, alignment, orientation, and complex background. This work presents a three-step tabular text extraction process, including pre-processing, separation, and extraction. The pre-processing step uses the guide image filter to remove various kinds of noise from the image. Improved binomial thresholding (IBT) separates the text from the image. Then the tabular text is recognized and extracted from CDI using deep neural network (DNN). In this work, weights of DNN layers are optimized by the Harris Hawk optimization algorithm (HHOA). Obtained text and associated information can be used in many ways, including replicating the document in digital format, information retrieval, and text summarization. The proposed process is applied comprehensively to UNLV, TableBank, and ICDAR 2013 image datasets. The complete procedure is implemented in Python, and precision metrics performance is verified.

Local to global purification strategy to realize collaborative camouflaged object detection

The purpose of camouflaged object detection is to detect objects in images that are not easily perceived by human eyes. Aiming at the problems of low recognition performance and unsatisfied texture information extraction in the complex environment in the current camouflaged object detection algorithms, we propose to improve the accuracy by simultaneously detecting a group of images containing the same camouflaged category. Therefore, we put forward a novel method termed local to global purification network (LGPNet) for collaborative camouflaged object detection. Our method comprises two main modules: the Local Detail Mining module (LDM) and the Global Intra-group Feature Extraction module (GIFE). The LDM is designed to exploit diversified detail information via different adaptive kernels and receptive field mechanisms locally, and the GIFE module is invented for feature enhancement and multi-level information aggregation. Specifically, the GIFE first utilizes channel attention and spatial attention mechanisms to enhance high-level semantic information and then aggregates the intra-group characteristics by level. Extensive experiments on CoCOD8K dataset and 4 COD benchmark datasets illustrate the effectiveness and superiority of our method compared to SOTAs.