Construct Training Data Research Articles

Document‐to‐Document Retrieval Using Self‐Retrieval Learning and Automatic Keyword Extraction

In this study, we propose self‐retrieval learning, a self‐supervised learning method that does not require an annotated dataset. In self‐retrieval learning, keywords extracted from documents are used as queries to construct training data that imitate the relationship between query and corpus, such that the documents themselves are retrieved. In the usual supervised learning for information retrieval, a pair of query and corpus document is required as training data, but self‐retrieval learning does not require such data. In addition, it does not use information such as reference lists or other documents connected to the query, but only the text of the documents in the target domain. In our experiments, self‐retrieval learning was performed on the EU and UK legal document retrieval task using a retrieval model called DRMM. We found that self‐retrieval learning not only does not require supervised datasets, but also outperforms supervised learning with the same model in terms of retrieval accuracy. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

Enhancing Cross-Lingual Sarcasm Detection by a Prompt Learning Framework with Data Augmentation and Contrastive Learning

Given their intricate nature and inherent ambiguity, sarcastic texts often mask deeper emotions, making it challenging to discern the genuine feelings behind the words. The proposal of the sarcasm detection task is to assist us with more accurately understanding the true intention of the speaker. Advanced methods, such as deep learning and neural networks, are widely used in the field of sarcasm detection. However, most research mainly focuses on sarcastic texts in English, as other languages lack corpora and annotated datasets. To address the challenge of low-resource languages in sarcasm detection tasks, a zero-shot cross-lingual transfer learning method is proposed in this paper. The proposed approach is based on prompt learning and aims to assist the model with understanding downstream tasks through prompts. Specifically, the model uses prompt templates to construct training data into cloze-style questions and then trains them using a pre-trained cross-lingual language model. Combining data augmentation and contrastive learning can further improve the capacity of the model for cross-lingual transfer learning. To evaluate the performance of the proposed model, we utilize a publicly accessible sarcasm dataset in English as training data in a zero-shot cross-lingual setting. When tested with Chinese as the target language for transfer, our model achieves F1-scores of 72.14% and 76.7% on two test datasets, outperforming the strong baselines by significant margins.

A novel spectral model for prediction of nongray radiative properties in gaseous combustion

Radiative heat transfer during gaseous combustion displays very strong spectral or “nongray” behavior, which is both very difficult to characterize and to evaluate. A novel spectral model based on the full-spectrum correlated k-distribution (FSCK) method combined with the multilayer perceptron (MLP), i.e., NFM model, is developed for the prediction of nongray radiative properties in gaseous combustion. Followed by the nature of FSCK method, the NFM model is designed as a simple MLP structure that has only two outputs to directly give both correlated k-value and corresponding ka-value. This not only avoids on-the-fly calculations of a-values but also saves storage by the simple structure. Moreover, by constructing training data with a number of small mole fractions, the errors caused by nonlinear effects are found to be mitigated during the use of NFM model. Several test cases have been carried out to compare the developed NFM model and other spectral tools including FSCK look-up tables and traditional FSCK MLP (TFM) model. Results show that the NFM model can achieve excellent accuracy that is even better than FSCK look-up tables at a tiny computational cost that is far less than that of TFM model.

ATN-Res2Unet: an advanced deep learning network for the elimination of saturation artifacts in endoscopy optical coherence tomography

Endoscopic optical coherence tomography (OCT) possesses the capability to non-invasively image internal lumens; however, it is susceptible to saturation artifacts arising from robust reflective structures. In this study, we introduce an innovative deep learning network, ATN-Res2Unet, designed to mitigate saturation artifacts in endoscopic OCT images. This is achieved through the integration of multi-scale perception, multi-attention mechanisms, and frequency domain filters. To address the challenge of obtaining ground truth in endoscopic OCT, we propose a method for constructing training data pairs. Experimental in vivo data substantiates the effectiveness of ATN-Res2Unet in reducing diverse artifacts while preserving structural information. Comparative analysis with prior studies reveals a notable enhancement, with average quantitative indicators increasing by 45.4–83.8%. Significantly, this study marks the inaugural exploration of leveraging deep learning to eradicate artifacts from endoscopic OCT images, presenting considerable potential for clinical applications.

Uncovering and Mitigating the Hidden Chasm: A Study on the Text-Text Domain Gap in Euphemism Identification

Euphemisms are commonly used on social media and darknet marketplaces to evade platform regulations by masking their true meanings with innocent ones. For instance, “weed” is used instead of “marijuana” for illicit transactions. Thus, euphemism identification, i.e., mapping a given euphemism (“weed”) to its specific target word (“marijuana”), is essential for improving content moderation and combating underground markets. Existing methods employ self-supervised schemes to automatically construct labeled training datasets for euphemism identification. However, they overlook the text-text domain gap caused by the discrepancy between the constructed training data and the test data, leading to performance deterioration. In this paper, we present the text-text domain gap and explain how it forms in terms of the data distribution and the cone effect. Moreover, to bridge this gap, we introduce a feature alignment network (FA-Net), which can both align the in-domain and cross-domain features, thus mitigating the domain gap from training data to test data and improving the performance of the base models for euphemism identification. We apply this FA-Net to the base models, obtaining markedly better results, and creating a state-of-the-art model which beats the large language models.

BOK-VQA: Bilingual outside Knowledge-Based Visual Question Answering via Graph Representation Pretraining

The current research direction in generative models, such as the recently developed GPT4, aims to find relevant knowledge information for multimodal and multilingual inputs to provide answers. Under these research circumstances, the demand for multilingual evaluation of visual question answering (VQA) tasks, a representative task of multimodal systems, has increased. Accordingly, we propose a bilingual outside-knowledge VQA (BOK-VQA) dataset in this study that can be extended to multilingualism. The proposed data include 17K images, 17K question-answer pairs for both Korean and English and 280K instances of knowledge information related to question-answer content. We also present a framework that can effectively inject knowledge information into a VQA system by pretraining the knowledge information of BOK-VQA data in the form of graph embeddings. Finally, through in-depth analysis, we demonstrated the actual effect of the knowledge information contained in the constructed training data on VQA.

Deep Learning Radiomics Analysis of CT Imaging for Differentiating Between Crohn's Disease and Intestinal Tuberculosis.

This study aimed to develop and evaluate a CT-based deep learning radiomics model for differentiating between Crohn's disease (CD) and intestinal tuberculosis (ITB). A total of 330 patients with pathologically confirmed as CD or ITB from the First Affiliated Hospital of Zhengzhou University were divided into the validation dataset one (CD: 167; ITB: 57) and validation dataset two (CD: 78; ITB: 28). Based on the validation dataset one, the synthetic minority oversampling technique (SMOTE) was adopted to create balanced dataset as training data for feature selection and model construction. The handcrafted and deep learning (DL) radiomics features were extracted from the arterial and venous phases images, respectively. The interobserver consistency analysis, Spearman's correlation, univariate analysis, and the least absolute shrinkage and selection operator (LASSO) regression were used to select features. Based on extracted multi-phase radiomics features, six logistic regression models were finally constructed. The diagnostic performances of different models were compared using ROC analysis and Delong test. The arterial-venous combined deep learning radiomics model for differentiating between CD and ITB showed a high prediction quality with AUCs of 0.885, 0.877, and 0.800 in SMOTE dataset, validation dataset one, and validation dataset two, respectively. Moreover, the deep learning radiomics model outperformed the handcrafted radiomics model in same phase images. In validation dataset one, the Delong test results indicated that there was a significant difference in the AUC of the arterial models (p = 0.037), while not in venous and arterial-venous combined models (p = 0.398 and p = 0.265) as comparing deep learning radiomics models and handcrafted radiomics models. In our study, the arterial-venous combined model based on deep learning radiomics analysis exhibited good performance in differentiating between CD and ITB.

Landslide susceptibility mapping of the Main Boundary Thrust region in Mandre - Khursanibari section of Arghakhanchi and Palpa Districts, Lumbini Province, Nepal

The Mandre-Khursanibari section of Arghakhanchi and Palpa districts consists of numerous landslides, small to large. Prospecting weak geological sections like the location of the Main Boundary Thrust (MBT) and inherently weak lithological settings in addition to other factors are the main objectives of the present research. In this study, the results of the landslide susceptibility analysis using frequency ratios (FR) are assessed. The Mandre-Khursanibari segment was subjected to landslide detection using imagery (Google Earth) with a spatial resolution of 50 cm. The landslide inventory was used to construct training and testing data. Based on topographic, geological, and land-use maps, nine contributing variables were identified. The FR ratings were established using the contributing elements and training data. The map illustrating landslide risk was produced by integrating the influencing components that determined FR ratings. The ROC-AUC curve shows a validation rate of 82.2 % in the present model of LSI. The three most important elements in the occurrence of the landslide, out of the nine potential causes, are distance from the thrust (MBT), land use, and distance from the road. However, the role of adverse lithological setting is found to be a minor in many sections of this region. The MBT is one of the main causes of landslides in this area.

학습 데이터 자동 구축을 통한 양방향 게이트 순환 유닛 기반 도로명주소 파싱 모델

학습 데이터 자동 구축을 통한 양방향 게이트 순환 유닛 기반 도로명주소 파싱 모델

In-Vehicle Speech Recognition for Voice-Driven UAV Control in a Collaborative Environment of MAV and UAV

Most conventional speech recognition systems have mainly concentrated on voice-driven control of personal user devices such as smartphones. Therefore, a speech recognition system used in a special environment needs to be developed in consideration of the environment. In this study, a speech recognition framework for voice-driven control of unmanned aerial vehicles (UAVs) is proposed in a collaborative environment between manned aerial vehicles (MAVs) and UAVs, where multiple MAVs and UAVs fly together, and pilots on board MAVs control multiple UAVs with their voices. Standard speech recognition systems consist of several modules, including front-end, recognition, and post-processing. Among them, this study focuses on recognition and post-processing modules in terms of in-vehicle speech recognition. In order to stably control UAVs via voice, it is necessary to handle the environmental conditions of the UAVs carefully. First, we define control commands that the MAV pilot delivers to UAVs and construct training data. Next, for the recognition module, we investigate an acoustic model suitable for the characteristics of the UAV control commands and the UAV system with hardware resource constraints. Finally, two approaches are proposed for post-processing: grammar network-based syntax analysis and transaction-based semantic analysis. For evaluation, we developed a speech recognition system in a collaborative simulation environment between a MAV and an UAV and successfully verified the validity of each module. As a result of recognition experiments of connected words consisting of two to five words, the recognition rates of hidden Markov model (HMM) and deep neural network (DNN)-based acoustic models were 98.2% and 98.4%, respectively. However, in terms of computational amount, the HMM model was about 100 times more efficient than DNN. In addition, the relative improvement in error rate with the proposed post-processing was about 65%.

Improving vertical resolution of vintage seismic data by a weakly supervised method based on cycle generative adversarial network

Seismic vertical resolution is critical for accurately identifying subsurface structures and reservoir properties. Improving the vertical resolution of vintage seismic data with strongly supervised deep learning is challenging due to scarce or costly labels. To remedy the label-lacking problem, we develop a weakly supervised deep-learning method to improve vintage seismic data with poor resolution by extrapolating from nearby high-resolution seismic data. Our method uses a cycle generative adversarial network with an improved identity loss function. In addition, we contribute a pseudo-3D training data construction strategy that reduces discontinuity artifacts caused by accessing 3D field data with a 2D network. We determine the feasibility of our method on 2D synthetic data and achieve results comparable to the classic time-varying spectrum whitening method on field poststack migration data while effectively recovering more high-frequency information.

중학교 수학 학습을 위한 질의응답 챗봇의 설계와 활용

Objectives The purpose of this study is to determine the most effective approach for constructing training data comprising questions and answers for the development of a mathematical question-answering chatbot. This involves a theoretical analysis of mathematical knowledge and an empirical analysis of questions posed by students using the chatbot.
 Methods The study targeted second-year middle school students and created a total of 1,936 pairs of math-related question-answer sets virtually. The chatbot was trained using the Doc2Vec technique, which is based on similarity. During the data construction process, various considerations related to mathematical knowledge and math learning were taken into account to design different types of questions. The data was then used for training the chatbot. Additionally, students utilized the trained chatbot during their classes for seven months, and their input questions were collected and analyzed.
 Results A comparison between the collected dataset of 1,636 questions from 92 middle school students and the types of question-answer pairs used for training the chatbot revealed that conceptual knowledge questions accounted for over 70% of the student-generated data. Students also posed questions to test the chatbot's capabilities and inquire about attitudes and emotional aspects related to mathematics. Within the category of conceptual knowledge questions, deeper inquiries involving comparisons, extensions, and connections of concepts were observed.
 Conclusions The study revealed that when students used the question-answering chatbot, inquiries related to c conceptual understanding of mathematics when students interact with the question-answering chatbot highlights the importance of focusing on conceptual questions. Furthermore, the activity of questioning concepts holds the potential to evolve into more advanced inquiries, fostering connections and extensions of mathematical concepts. Further research is needed to explore more suitable and effective ways of responding to students through the chatbot and utilizing it in educational contexts.

Multi-scale feature fusion kernel estimation with masked interpolation loss for real-world remote sensing images super-resolution

ABSTRACT In recent years, the application of deep learning to remote sensing image super-resolution has achieved promising results. However, most deep learning-based methods are often trained on remote sensing datasets constructed by bicubic downsampling, and their recovery effects on real-world remote sensing images are often unsatisfactory. This is because the process of generating low-resolution (LR) images from high-resolution (HR) images to construct training data pairs (LR-HR) using simple bicubic downsampling cannot reflect the degradation process of real-world remote sensing dataset images. In this paper, we propose a multi-scale feature fusion kernel estimation with masked interpolation loss for real-world remote sensing images super-resolution (MFFILSR) to address this problem. MFFILSR is divided into two stages: degenerate kernel estimation and SR network training. In the first stage, we propose a multi-scale feature fusion kernel estimation network that can effectively fuse multi-scale information, making the estimated downsampling kernel closer to the degradation patterns of real-world remote sensing images. In the second stage, we introduce a masked interpolation loss during generator training, by masking the interpolation loss, the artefacts of generated images can be effectively reduced. Extensive experiments show that MFFILSR has satisfactory super-resolution reconstruction performance for real-world remote sensing images.

Automatic Summarization for Academic Articles using Deep Learning and Reinforcement Learning with Viewpoints

The purpose of this research is to develop a Viewpoint Refinement in Automatic Summarization (VPRAS) system for research articles. The system will reflect viewpoints of survey to support surveys stage for researchers and students. We collect academic articles using web scraping technology and construct training data by combining sections and sentences through analysis of the article's PDF structure. We use machine learning techniques to classify sentences in Japanese articles into viewpoints. In addition to supervised learning, we introduce reinforcement learning and Dynamic Programming (DP) to extract important sentences for each viewpoint. Finally, we implemented an agent to automatically extract summary sentences based on a reward function.

UAV remote sensing monitoring of pine forest diseases based on improved Mask R-CNN

ABSTRACT Timely and accurate monitoring of pine forest diseases is greatly important to pine forest disease prevention and forest protection. A method for unmanned aerial vehicle (UAV) remote sensing monitoring of pine forest diseases is proposed based on improved Mask region-based convolutional neural network (R-CNN) to solve the problem of low accuracy of the existing methods due to complex background and nonevident characteristics of pine forest diseases. The Mask R-CNN with multitask capabilities is utilized as backbone network to build the monitoring model. An improved multiscale receptive field block module is added to the Mask R-CNN to extract the detailed features of pine forest diseases and reduce the missed detection of pine forest diseases. Meanwhile, a multilevel fusion residual pyramid structure is adopted to integrate low-level and high-level features to obtain distinguishable features of pine forest diseases to reduce the misdetection and misidentification of complex backgrounds. In addition, the proposed method uses cutting mixing splicing method to construct training data set to increase the number of background objects in the training images and reduce the impact of complex backgrounds on the monitoring results. Experimental results show that the proposed method can monitor pine forest diseases more accurately than methods of SSD, Faster R-CNN, MaskScoring R-CNN, and the original Mask R-CNN. Compared with the method of backbone network, the precision of the proposed method has increased by 22.4%, the recall has increased by 3.5%, and the F1-score has increased by 14.4%.

Automatic electron hologram acquisition of catalyst nanoparticles using particle detection with image processing and machine learning

To enable better statistical analysis of catalyst nanoparticles by high-resolution electron holography, we improved the particle detection accuracy of our previously developed automated hologram acquisition system by using an image classifier trained with machine learning. The detection accuracy of 83% was achieved with the small training data of just 232 images showing nanoparticles by utilizing transfer learning based on VGG16 to train the image classifier. Although the construction of training data generally requires much effort, the time needed to select the training data candidates was significantly shortened by utilizing a pattern matching technique. Experimental results showed that the high-resolution hologram acquisition efficiency was improved by factors of about 100 and 6 compared to a scan method and a pattern-matching-only method, respectively.

Identification of Ship Hydrodynamic Derivatives Based on LS-SVM with Wavelet Threshold Denoising

Nowadays, system-based simulation is one of the main methods for ship manoeuvring prediction. Great efforts are usually devoted to the determination of hydrodynamic derivatives as required for the mathematical models used for such methods. System identification methods can be applied to determine hydrodynamic derivatives. The purpose of this work is to present a parameter identification study based on least-squares support-vector machines (LS-SVMs) to obtain hydrodynamic derivatives for an Abkowitz-type model. An approach for constructing training data is used to reduce parameter drift. In addition, wavelet threshold denoising is applied to filter out the noise from the sample data during data pre-processing. Most of the resulting derivatives are very close to the original ones—especially for linear derivatives. Although the errors of high-order derivatives seem large, the final predicted results of the turning circle and zigzag manoeuvres agree pretty well with the reference ones. This indicates that the used methods are effective in obtaining manoeuvring hydrodynamic derivatives.

Memory Model for Morphological Semantics of Visual Stimuli Using Sparse Distributed Representation

Recent achievements on CNN (convolutional neural networks) and DNN (deep neural networks) researches provide a lot of practical applications on computer vision area. However, these approaches require construction of huge size of training data for learning process. This paper tries to find a way for continual learning which does not require prior high-cost training data construction by imitating a biological memory model. We employ SDR (sparse distributed representation) for information processing and semantic memory model, which is known as a representation model of firing patterns on neurons in neocortex area. This paper proposes a novel memory model to reflect remembrance of morphological semantics of visual input stimuli. The proposed memory model considers both memory process and recall process separately. First, memory process converts input visual stimuli to sparse distributed representation, and in this process, morphological semantic of input visual stimuli can be preserved. Next, recall process can be considered by comparing sparse distributed representation of new input visual stimulus and remembered sparse distributed representations. Superposition of sparse distributed representation is used to measure similarities. Experimental results using 10,000 images in MNIST (Modified National Institute of Standards and Technology) and Fashion-MNIST data sets show that the sparse distributed representation of the proposed model efficiently keeps morphological semantic of the input visual stimuli.

A Low Complexity Learning-Based Channel Estimation for OFDM Systems With Online Training

In this paper, we devise a highly efficient machine learning-based channel estimation for orthogonal frequency division multiplexing (OFDM) systems, in which the training of the estimator is performed online. A simple learning module is employed for the proposed learning-based estimator. The training process is thus much faster and the required training data is reduced significantly. Besides, a training data construction approach utilizing least square (LS) estimation results is proposed so that the training data can be collected during the data transmission. The feasibility of this novel construction approach is verified by theoretical analysis and simulations. Based on this construction approach, two alternative training data generation schemes are proposed. One scheme transmits additional block pilot symbols to create training data, while the other scheme adopts a decision-directed method and does not require extra pilot overhead. Simulation results show the robustness of the proposed channel estimation method. Furthermore, the proposed method shows better adaptation to practical imperfections compared with the conventional minimum mean-square error (MMSE) channel estimation. It outperforms the existing machine learning-based channel estimation techniques under varying channel conditions.

Repeatability enhancement of time-lapse seismic data via a convolutional autoencoder

SUMMARY In an ideal case, the time-lapse differences in 4-D seismic data should only reflect the changes of the subsurface geology. Practically, however, undesirable discrepancies are generated because of various reasons. Therefore, proper time-lapse processing techniques are required to improve the repeatability of time-lapse seismic data and to capture accurate seismic information to analyse target changes. In this study, we propose a machine learning-based time-lapse seismic data processing method improving repeatability. A training data construction method, training strategy and machine learning network architecture based on a convolutional autoencoder are proposed. Uniform manifold approximation and projection are applied to the training and target data to analyse the features corresponding to each data point. When the feature distribution of the training data is different from the target data, we implement data augmentation to enhance the diversity of the training data. The method is verified through numerical experiments using both synthetic and field time-lapse seismic data, and the results are analysed with several methods, including a comparison of repeatability metrics. From the results of the numerical experiments, we can conclude that the proposed convolutional autoencoder can enhance the repeatability of the time-lapse seismic data and increase the accuracy of observed variations in seismic signals generated from target changes.