Amount Of Training Data Research Articles

Depth estimation and 3D reconstruction from UAV-borne imagery: Evaluation on the UseGeo dataset

Depth estimation and 3D model reconstruction from aerial imagery is an important task in photogrammetry, remote sensing, and computer vision. To compare the performance of different image-based approaches, this study presents a benchmark for UAV-based aerial imagery using the UseGeo dataset. The contributions include the release of various evaluation routines on GitHub, as well as a comprehensive comparison of baseline approaches, such as methods for offline multi-view 3D reconstruction resulting in point clouds and triangle meshes, online multi-view depth estimation, as well as single-image depth estimation using self-supervised deep learning. With the release of our evaluation routines, we aim to provide a universal protocol for the evaluation of depth estimation and 3D reconstruction methods on the UseGeo dataset. The conducted experiments and analyses show that each method excels in a different category: the depth estimation from COLMAP outperforms that of the other approaches, ACMMP achieves the lowest error and highest completeness for point clouds, while OpenMVS produces triangle meshes with the lowest error. Among the online methods for depth estimation, the approach from the Plane-Sweep Library outperforms the FaSS-MVS approach, while the latter achieves the lowest processing time. And even though the particularly challenging nature of the dataset and the small amount of training data leads to a significantly higher error in the results of the self-supervised single-image depth estimation approach, it outperforms all other approaches in terms of processing time and frame rate. In our evaluation, we have also considered modern learning-based approaches that can be used for image-based 3D reconstruction, such as NeRFs. However, due to the significantly lower quality of the resulting 3D models, we have only included a qualitative comparison between NeRF-based and conventional approaches in the scope of this work.

Early fault detection for rolling bearings: A meta‐learning approach

AbstractEarly fault detection (EFD) of rolling bearings aims at detecting the early symptoms of faults by monitoring small deviations of health states. Accurate EFD enables predictive maintenance and contributes to the stability of mechanical systems. In recent years, machine learning based methods have shown impressive performance on EFD. Most of the current machine learning‐based methods assume the availability for a large amount of data. However, in practice, the authors may only have a very limited amount of training data, which makes it hard to learn a reliable machine learning model. To address this concern, in this work, the authors propose to tackle EFD via meta learning. Specifically, the authors first formulate EFD as a few‐shot learning problem and then propose to tackle this problem with a metric‐based meta learning method. Furthermore, ensemble learning is further leveraged to improve the detection robustness. For the proposed method, the distribution difference from the working conditions and the bearings are considered. The experimental results on two bearing datasets show that the proposed method can achieve better EFD performance, that is, detecting incipient faults earlier while bringing in lower false alarms, compared with several frequently used EFD methods.

Sensitivity analysis of distributed photovoltaic system capacity estimation based on artificial neural network

Residential solar photovoltaic (PV) system installations are expected to continue increasing due to their growing cost competitiveness and supportive government policies. However, excessive installations of unknown behind-the-meter solar panels present a challenge for accurate load prediction and reliable operations of power networks. To address such growing concerns of distribution network operators (DNOs), this research proposes a novel model for distributed PV system capacity estimations. Innovative extracted features from 24-hour substation net load curves were fed into a deep neural network to estimate the PV capacity linked to the substation feeder. A comprehensive study into the sensitivity of the model’s accuracy to specific temporal scales of data collection, number of households served by a substation, and proportion of PV-equipped properties was conducted. This study revealed that a model developed to be used exclusively in summer achieved a 18.1% decrease in estimation root mean squared error (RMSE) compared to an all-year model, whilst using only a third of the training data amount. Similarly, compared to an all-year model, RMSE decreased by 26.9% when only data from Mondays to Thursdays were used to train and test the model. Also, for the all-year model, the most accurate estimations occur when 20% to 80% of households have PV systems installed and estimation percentage error tend to remain constant at around 10% when more than 20% of households have PV systems installed. A machine learning-ready dataset of substations with known PV capacity and experiment results are both useful to inform DNOs on the potential of the proposed method in reducing grid operation costs.

Generative artificial intelligence for distributed learning to enhance smart grid communication

Machine learning models are the backbone of smart grid optimization, but their effectiveness hinges on access to vast amounts of training data. However, smart grids face critical communication bottlenecks due to the ever-increasing volume of data from distributed sensors. This paper introduces a novel approach leveraging Generative Artificial Intelligence (GenAI), specifically a type of pre-trained Foundation Model (FM) architecture suitable for time series data due to its efficiency and privacy-preserving properties. These GenAI models are distributed to agents, or data holders, empowering them to fine-tune the foundation model with their local datasets. By fine-tuning the foundation model, the updated model can produce synthetic data that mirrors real-world grid conditions. The server aggregates fine-tuned model from all agents and then generates synthetic data which considers all data collected in the grid. This synthetic data can be used to train global machine learning models for specific tasks like anomaly detection and energy optimization. Then, the trained task models are distributed to agents in the grid to leverage them. The paper highlights the advantages of GenAI for smart grid communication, including reduced communication burden, enhanced privacy through anonymized data transmission, and improved efficiency and scalability. By enabling a distributed and intelligent communication architecture, GenAI introduces a novel way for a more secure, efficient, and sustainable energy future.

A multi-step loss meta-learning method based on multi-scale feature extraction for few-shot fault diagnosis

Existing deep learning (DL) algorithms are based on a large amount of training data and they face challenges in effectively extracting fault features when dealing with few-shot fault diagnoses. Model-agnostic meta-learning (MAML) also faces some challenges, including the limited capability of the basic convolutional neural network (CNN) with a single convolutional kernel to extract fault features comprehensively, as well as the instability of model training due to the inner and outer double-layer loops. To address these issues, this paper presents a multi-step loss meta-learning method based on multi-scale feature extraction (MFEML). Firstly, an improved multi-scale feature extraction module (IMFEM) is designed to solve the problem of the insufficient feature extraction capability of the CNN. Secondly, the multi-step loss is used to reconstruct the meta-loss to address the issue of MAML training instability. Finally, the experimental results of two datasets demonstrate the effectiveness of the MFEML.

Interpretable Prediction of SARS-CoV-2 Epitope-Specific TCR Recognition Using a Pre-Trained Protein Language Model.

The emergence of the novel coronavirus, designated as severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2), has posed a significant threat to public health worldwide. There has been progress in reducing hospitalizations and deaths due to SARS-CoV-2. However, challenges stem from the emergence of SARS-CoV-2 variants, which exhibit high transmission rates, increased disease severity, and the ability to evade humoral immunity. Epitope-specific T-cell receptor (TCR) recognition is key in determining the T-cell immunogenicity for SARS-CoV-2 epitopes. Although several data-driven methods for predicting epitope-specific TCR recognition have been proposed, they remain challenging due to the enormous diversity of TCRs and the lack of available training data. Self-supervised transfer learning has recently been proven useful for extracting information from unlabeled protein sequences, increasing the predictive performance of fine-tuned models, and using a relatively small amount of training data. This study presents a deep-learning model generated by fine-tuning pre-trained protein embeddings from a large corpus of protein sequences. The fine-tuned model showed markedly high predictive performance and outperformed the recent Gaussian process-based prediction model. The output attentions captured by the deep-learning model suggested critical amino acid positions in the SARS-CoV-2 epitope-specific TCRβ sequences that are highly associated with the viral escape of T-cell immune response.

Rician Noise Removal in Low-noise Condition via a Deep Unfolding Network

Nuclear magnetic resonance (NMR) signals as the key role of research on denoising, are mainly influenced by Rician noise. Restoring clean Rician medical images from signal-related Rician noise is a challenging task with great practical significance. In this paper, an energy function based on MAP estimation is proposed, mainly for conditions of low noise, where the noise distribution is approximately Gaussian. Then the mathematical model is embedded into the network structure to realize the knot of knowledge-driven and network learning. The experiment shows that the proposed model only uses simple and lightweight network modules and a small amount of training data, which has a better denoising effect and has achieved satisfactory results under the two evaluation indexes.

Machine learning-based time-lapse 1D seismic full-waveform inversion with efficient training data generation in a carbon capture and storage monitoring

Machine learning (ML)-based time-lapse and seismic full-waveform inversion has recently gained attention in carbon capture and storage (CCS) research because of its potential to overcome the limitations of theory-based inversion and its excellent imaging performance for CO2 plumes. We propose a ML-based time-lapse 1D inversion method with efficient training data generation. The proposed method exploits time-lapse seismic data to infer time-lapse velocity changes and uses a 1D inversion strategy to enable the UNet3+ model to learn the general characteristics of velocity changes between baseline and monitoring phases. Our proposed method quickly generates various 1D velocity models and rapidly provides numerous near-offset data by using 1D ray-based modeling method. This allows us to overcome a challenge in constructing of 2D velocity models based on limited geological assumptions and forward modeling to generate seismic gathers, which requires massive computational resources. However, because 2D effects such as diffraction caused by the CO2 plume only affect the monitoring data, inaccurate predictions in the 1D inversion occurred near the plume, following the diffraction trajectory. We corrected this effect using a diffraction removal process for 2D target data, thereby improving the prediction accuracy of the ML model. The ML model was trained with a large amount of training data generated using the proposed method and validated on 2D target data with three different CO2 injection plume shapes obtained from a modified Marmousi2 velocity model. As a result, information about the CO2 plume was reasonably delineated, although the magnitude of the time-lapse velocity change near the plume boundary was reduced by small amplitude losses during the diffraction removal process. Our results suggest that the proposed method is a powerful tool for CCS monitoring, which requires forward modeling for each iterative survey.

Pollen identification through convolutional neural networks: First application on a full fossil pollen sequence.

The automation of pollen identification has seen vast improvements in the past years, with Convolutional Neural Networks coming out as the preferred tool to train models. Still, only a small portion of works published on the matter address the identification of fossil pollen. Fossil pollen is commonly extracted from organic sediment cores and are used by paleoecologists to reconstruct past environments, flora, vegetation, and their evolution through time. The automation of fossil pollen identification would allow paleoecologists to save both time and money while reducing bias and uncertainty. However, Convolutional Neural Networks require a large amount of data for training and databases of fossilized pollen are rare and often incomplete. Since machine learning models are usually trained using labelled fresh pollen associated with many different species, there exists a gap between the training data and target data. We propose a method for a large-scale fossil pollen identification workflow. Our proposed method employs an accelerated fossil pollen extraction protocol and Convolutional Neural Networks trained on the labelled fresh pollen of the species most commonly found in Northeastern American organic sediments. We first test our model on fresh pollen and then on a full fossil pollen sequence totalling 196,526 images. Our model achieved an average per class accuracy of 91.2% when tested against fresh pollen. However, we find that our model does not perform as well when tested on fossil data. While our model is overconfident in its predictions, the general abundance patterns remain consistent with the traditional palynologist IDs. Although not yet capable of accurately classifying a whole fossil pollen sequence, our model serves as a proof of concept towards creating a full large-scale identification workflow.

Classification of Simple CNN Model and ResNet50

Abstract: In recent years, Convolutional Neural Networks (CNNs) have emerged as powerful tools for image classification tasks, achieving state-of-the-art performance in various domains. Among the plethora of CNN architectures, the Simple CNN model and ResNet50 stand out as widely used architectures with distinct characteristics. In this study, we present a comparative analysis of these two architectures in terms of their performance, computational efficiency, and robustness for classification tasks.The Simple CNN model represents a straightforward convolutional neural network architecture with a sequential arrangement of convolutional, pooling, and fully connected layers. On the other hand, ResNet50 introduces the concept of residual learning, leveraging skip connections to address the vanishing gradient problem and facilitate the training of deeper networks.We conduct experiments on benchmark datasets, evaluating the classification accuracy, training time, and model complexity of both architectures. Our findings reveal insights into the strengths and weaknesses of each model. While the Simple CNN model demonstrates simplicity and ease of implementation, ResNet50 exhibits superior performance, particularly in scenarios with a large amount of training data and complex feature representations.

Retrieval for Extremely Long Queries and Documents with RPRS: A Highly Efficient and Effective Transformer-based Re-Ranker

Retrieval with extremely long queries and documents is a well-known and challenging task in information retrieval and is commonly known as Query-by-Document (QBD) retrieval. Specifically designed Transformer models that can handle long input sequences have not shown high effectiveness in QBD tasks in previous work. We propose a Re-Ranker based on the novel Proportional Relevance Score (RPRS) to compute the relevance score between a query and the top- k candidate documents. Our extensive evaluation shows RPRS obtains significantly better results than the state-of-the-art models on five different datasets. Furthermore, RPRS is highly efficient, since all documents can be pre-processed, embedded, and indexed before query time that gives our re-ranker the advantage of having a complexity of O(N) , where N is the total number of sentences in the query and candidate documents. Furthermore, our method solves the problem of the low-resource training in QBD retrieval tasks as it does not need large amounts of training data and has only three parameters with a limited range that can be optimized with a grid search even if a small amount of labeled data is available. Our detailed analysis shows that RPRS benefits from covering the full length of candidate documents and queries.

Data Augmentation for Sample Efficient and Robust Document Ranking

Contextual ranking models have delivered impressive performance improvements over classical models in the document ranking task. However, these highly over-parameterized models tend to be data-hungry and require large amounts of data even for fine-tuning. In this article, we propose data-augmentation methods for effective and robust ranking performance. One of the key benefits of using data augmentation is in achieving sample efficiency or learning effectively when we have only a small amount of training data. We propose supervised and unsupervised data augmentation schemes by creating training data using parts of the relevant documents in the query-document pairs. We then adapt a family of contrastive losses for the document ranking task that can exploit the augmented data to learn an effective ranking model. Our extensive experiments on subsets of the MS MARCO and TREC-DL test sets show that data augmentation, along with the ranking-adapted contrastive losses, results in performance improvements under most dataset sizes. Apart from sample efficiency, we conclusively show that data augmentation results in robust models when transferred to out-of-domain benchmarks. Our performance improvements in in-domain and more prominently in out-of-domain benchmarks show that augmentation regularizes the ranking model and improves its robustness and generalization capability.

Automatic flaw detection in sectoral scans using machine learning

Phased array ultrasonic testing (PAUT) requires highly trained and qualified personnel to interpret and analyze images. It takes a solid understanding of wave propagation physics to comprehend the generated images. As such, the inspector's judgment and level of experience have a significant impact on the analysis's outcome. In addition, the procedure is prone to error and laborious. AI had shown to be effective in computer vision in a variety of classification and detection tasks. Regarding PAUT, studies have also demonstrated that machine learning may be able to identify defects with a level of accuracy that is on par or even superior to that of trained and qualified inspectors. Nonetheless, the use of computer vision in PAUT remains very limited. The primary cause of this is the challenge accessing large databases of labelled inspections. In fact, a considerable amount of training data is required for machine learning. While it is easy to access sizeable, labelled databases of MRI scans or photographs for instance, that is not the case in PAUT because inspection results are usually confidential. In this project, a large database was generated using mock-ups commonly used to train and evaluate inspectors. The different defects contained in these mock-ups were used to train a machine learning model. The data was acquired with several different probes centered at different frequencies. Each acquisition was performed using Full Matrix Capture (FMC). The post-processing of the data contained in the FMC allows to compute any sectoral scan from its focal laws. As a result, a comprehensive database composed of hundreds of thousands of sectoral scans was generated from these few FMC acquisitions. The completeness of this database facilitated robust training of a defect detection model for PAUT sectoral scans. The evaluation of the model demonstrated its ability to generalize even to defect types it had never been trained on. Furthermore, the detection performance remained consistent even in high noise conditions where the Contrast-to-Noise Ratio (CNR) was very low.

An improved semantic segmentation algorithm for high-resolution remote sensing images based on DeepLabv3+

High-precision and high-efficiency Semantic segmentation of high-resolution remote sensing images is a challenge. Existing models typically require a significant amount of training data to achieve good classification results and have numerous training parameters. A novel model called MST-DeepLabv3+ was suggested in this paper for remote sensing image classification. It’s based on the DeepLabv3+ and can produce better results with fewer train parameters. MST-DeepLabv3+ made three improvements: (1) Reducing the number of model parameters by substituting MobileNetV2 for the Xception in the DeepLabv3+’s backbone network. (2) Adding the attention mechanism module SENet to increase the precision of semantic segmentation. (3) Increasing Transfer Learning to enhance the model's capacity to recognize features, and raise the segmentation accuracy. MST-DeepLabv3+ was tested on international society for photogrammetry and remote sensing (ISPRS) dataset, Gaofen image dataset (GID), and practically applied to the Taikang cultivated land dataset. On the ISPRS dataset, the mean intersection over union (MIoU), overall accuracy (OA), Precision, Recall, and F1-score are 82.47%, 92.13%, 90.34%, 90.12%, and 90.23%, respectively. On the GID dataset, these values are 73.44%, 85.58%, 84.10%, 84.86%, and 84.48%, respectively. The results were as high as 90.77%, 95.47%, 95.28%, 95.02%, and 95.15% on the Taikang cultivated land dataset. The experimental results indicate that MST-DeepLabv3+ effectively improves the accuracy of semantic segmentation of remote sensing images, recognizes the edge information with more completeness, and significantly reduces the parameter size.

Classification of battery compounds using structure-free Mendeleev encodings

Machine learning is a valuable tool that can accelerate the discovery and design of materials occupying combinatorial chemical spaces. However, the prerequisite need for vast amounts of training data can be prohibitive when significant resources are needed to characterize or simulate candidate structures. Recent results have shown that structure-free encoding of complex materials, based entirely on chemical compositions, can overcome this impediment and perform well in unsupervised learning tasks. In this study, we extend this exploration to supervised classification, and show how structure-free encoding can accurately predict classes of material compounds for battery applications without time consuming measurement of bonding networks, lattices or densities.Scientific contribution The comprehensive evaluation of structure-free encodings of complex materials in classification tasks, including binary and multi-class separation, inclusive of three classifiers based on different logic function, is measured four metrics and learning curves. The encoding is applied to two data sets from computational and experimental sources, and the outcomes visualised using 5 approaches to confirms the suitability and superiority of Mendeleev encoding. These methods are general and accessible using source software, to provide simple, intuitive and interpretable materials informatics outcomes to accelerate materials design.

Q-SupCon: Quantum-Enhanced Supervised Contrastive Learning Architecture within the Representation Learning Framework

In the evolving landscape of data privacy regulations, the challenge of providing extensive data for robust deep classification models arises. The accuracy of these models relies on the amount of training data, due to the multitude of parameters that require tuning. Unfortunately, obtaining such ample data proves challenging, particularly in domains like medical applications, where there is a pressing need for robust models for early disease detection but a shortage of labeled data. Nevertheless, the classical supervised contrastive learning models, have shown the potential to address this challenge up to a certain limit, by utilizing deep encoder models. However, recent advancements in quantum machine learning enable the extraction of meaningful representations from extremely limited and simple data. Thus, replacing classical counterparts in classical or hybrid quantum-classical supervised contrastive models enhances feature learning capability with minimal data. Therefore, this work proposes the Q-SupCon model, a fully quantum-powered supervised contrastive learning model comprising a quantum data augmentation circuit, quantum encoder, quantum projection head, and quantum variational classifier, enabling efficient image classification with minimal labeled data. Furthermore, the novel model attains 80%, 60%, and 80% test accuracy on MNIST, KMNIST, and FMNIST datasets, marking a significant advancement in addressing the data scarcity challenge.

Feature decoupling and regeneration towards wifi-based human activity recognition

Gesture recognition using WiFi is vital for human–computer interaction, smart homes, smart spaces, and elderly care. WiFi signals are non-stationary, sensitive to the environment, and traditional pattern recognition-based HAR methods incur exorbitant training and deployment costs due to their reliance on the quality and quantity of training data. While there has been extensive research on enhancing HAR techniques, the efficiency of the system is still limited by the scarcity of training data and identity-action feature coupling. This research focuses on attaining gesture recognition across multiple users with a limited number of samples. Our findings show that the conventional data augmentation methods are incapable of facilitating the system in attaining sample diversity. As a result, we propose a training-free augmentation strategy as a means to provide adequate training data. Unlike the conventional data enhancement approach, this scheme aims to develop data processing methods that differentiate between various samples in practical applications. Consequently, it enhances data pertinent to specific HAR assignments effectively. To effectively extract action features in WiFi samples, an unsupervised cross-user domain sample generation (CUDSG) model is proposed. This model generates virtual gesture samples for new user domains by decoupling and recombining gesture and identity features. This extends the sensing boundary of the system to new user domains without requiring a significant number of users to participate. Model performance was evaluated using various classifiers, such as SVM, KNN and CNN. The results demonstrate a significant improvement in average classification accuracy from 57.3% to 98.4%. This indicates that CUDSG is a highly effective tool for enhancing the performance of existing gesture recognition techniques.

Language model based on deep learning network for biomedical named entity recognition

Biomedical Named Entity Recognition (BioNER) is one of the most basic tasks in biomedical text mining, which aims to automatically identify and classify biomedical entities in text. Recently, deep learning-based methods have been applied to Biomedical Named Entity Recognition and have shown encouraging results. However, many biological entities are polysemous and ambiguous, which is one of the main obstacles to the task of biomedical named entity recognition. Deep learning methods require large amounts of training data, so the lack of data also affect the performance of model recognition. To solve the problem of polysemous words and insufficient data, for the task of biomedical named entity recognition, we propose a multi-task learning framework fused with language model based on the BiLSTM-CRF architecture. Our model uses a language model to design a differential encoding of the context, which could obtain dynamic word vectors to distinguish words in different datasets. Moreover, we use a multi-task learning method to collectively share the dynamic word vector of different types of entities to improve the recognition performance of each type of entity. Experimental results show that our model reduces the false positives caused by polysemous words through differentiated coding, and improves the performance of each subtask by sharing information between different entity data. Compared with other state-of-the art methods, our model achieved superior results in four typical training sets, and achieved the best results in F1 values.

Klasifikasi Menggunakan Algoritma K-Nearest Neighbor pada Imbalance Class Data dengan SMOTE. (Studi Kasus: Nasabah Bank Perkreditan Rakyat “X”)

<p class="AbstractEnglish">Rural Banks (Bank Perkreditan Rakyat/BPR) provide financial services to micro-businesses and low repayment communities, especially in rural areas. The main activity of the bank is lending. Customer credit classification is expected to assist BPR in anticipating potential bad loans. K-Nearest Neighbor classify current and potential bad credit status based on customer data from BPR “X” in Central Java in October 2022. K-Nearest Neighbor is effective against a large amount of training data and works based on the nearest neighbor. There is an imbalance class data which causes the classification process to focus more on the majority class. Imbalance class data is handled using Synthetic Minority Oversampling Technique (SMOTE) as an oversampling approach. Classification with the addition of SMOTE can improve the evaluation of classification accuracy, especially G-mean. G-mean is the most comprehensive measurement in term of accuracy, sensitivity and specificity in evaluating classification performance on imbalance class data. The results of this research were able to increase g-mean to 58.55% and sensitivity to 45.46% by implementing SMOTE. Based on the classification results, it is concluded that K-Nearest Neighbor with SMOTE at k = 19 and a proportion of training data to test data of 70:30 is a more appropriate classification model to use for customer credit status.</p><p class="AbstractEnglish"> </p><p><strong>Keywords: </strong>Credit Status; K-Nearest Neighbor; Imbalance Class Data; SMOTE</p>

Reliability-Based Design Optimization of the PEMFC Flow Field with Consideration of Statistical Uncertainty of Design Variables

Recently, with the fourth industrial revolution, the research cases that search for optimal design points based on neural networks or machine learning have rapidly increased. In addition, research on optimization is continuously reported in the field of fuel cell research using hydrogen as fuel. However, in the case of optimization research, it often requires a large amount of training data, which means that it is more suitable for numerical research such as CFD simulation rather than time-consuming research such as actual experiments. As is well known, the design range of fuel cell flow channels is extremely small, ranging from hundreds of microns to several millimeters, which means the small tolerance could cause fatal performance loss. In this study, the general optimization study was further improved in terms of reliability by considering stochastic tolerances that may occur in actual industry. The optimization problem was defined to maximize stack power, which is employed as objective function, under the constraints such as pressure drop and current density standard deviation; the performance of the optimal point through general optimization was about 3.252 kW/L. In the reliability-based optimization problem, the boundary condition for tolerance was set to 0.1 mm and tolerance was assumed to occur along a normal distribution. The optimal point to secure 99% reliability for the given constraints was 2.918 kW/L, showing significantly lower performance than the general optimal point.