Amount Of Training Data Research Articles

Construction grammar and procedural semantics for human-interpretable grounded language processing

Abstract Grounded language processing is a crucial component in many artificial intelligence systems, as it allows agents to communicate about their physical surroundings. State-of-the-art approaches typically employ deep learning techniques that perform end-to-end mappings between natural language expressions and representations grounded in the environment. Although these techniques achieve high levels of accuracy, they are often criticized for their lack of interpretability and their reliance on large amounts of training data. As an alternative, we propose a fully interpretable, data-efficient architecture for grounded language processing. The architecture is based on two main components. The first component comprises an inventory of human-interpretable concepts learned through task-based communicative interactions. These concepts connect the sensorimotor experiences of an agent to meaningful symbols that can be used for reasoning operations. The second component is a computational construction grammar that maps between natural language expressions and procedural semantic representations. These representations are grounded through their integration with the learned concepts. We validate the architecture using a variation on the CLEVR benchmark, achieving an accuracy of 96 %. Our experiments demonstrate that the integration of a computational construction grammar with an inventory of interpretable grounded concepts can effectively achieve human-interpretable grounded language processing in the CLEVR environment.

Post-Operative Brain MRI Resection Cavity Segmentation Model and Follow-Up Treatment Assistance

Post-operative brain magnetic resonance imaging (MRI) segmentation is inherently challenging due to the diverse patterns in brain tissue, which makes it difficult to accurately identify resected areas. Therefore, there is a crucial need for a precise segmentation model. Due to the scarcity of post-operative brain MRI scans, it is not feasible to use complex models that require a large amount of training data. This paper introduces an innovative approach for accurately segmenting and quantifying post-operative brain resection cavities in MRI scans. The proposed model, named Attention-Enhanced VGG-U-Net, integrates VGG16 initial weights in the encoder section and incorporates a self-attention module in the decoder, offering improved accuracy for postoperative brain MRI segmentation. The attention mechanism enhances its accuracy by concentrating on a specific area of interest. The VGG16 model is comparatively lightweight, has pre-trained weights, and allows the model to extract incredibly detailed information from the input. The model is trained on publicly available post-operative brain MRI data and achieved a Dice coefficient value of 0.893. The model is then assessed using a clinical dataset of postoperative brain MRIs. The model facilitates the quantification of the resected regions and enables comparisons with each brain region based on pre-operative images. The capabilities of the model assist radiologists in evaluating surgical success and directing follow-up procedures.

The Price Prediction of Vegetables by Using Machine Learning

Vegetable prices are great important in ensuring people's livelihood. With the progress of science and technology, machine learning algorithms have been applied in the field of vegetable price prediction. This paper analyzed the model that has been used in vegetable price prediction and its results and compared the accuracy of each model. The study in this paper concluded that LSTM is suitable for handling sequence data with long-term dependence but this model requires a large amount of data for training. ARIMA is suitable for short-term predictions but may not fit well with non-stationary and nonlinear data. In addition, this paper introduces other algorithms. For example, the paper shows How to use Lasso regression to compress the input variables and how to reduce model complexity and improve model generalization ability. The BP neural network was used to fit the output variables to improve the model fitting ability. This paper further studies and summarizes the combination of each model and the accuracy of the combined model for predicting vegetable prices. It can be seen that the prediction error of the L-LBNN model is very small, which can be effectively used in the price prediction of vegetables. The purpose of this study is to summarize the current situation of predicting vegetable prices with machine learning algorithms to provide guidance and inspiration for researchers and to promote the development of the use of machine learning in the field of price prediction of vegetables.

Detection and coverage estimation of purple nutsedge in turf with image classification neural networks.

Accurate detection of weeds and estimation of their coverage is crucial for implementing precision herbicide applications. Deep learning (DL) techniques are typically used for weed detection and coverage estimation by analyzing information at the pixel or individual plant level, which requires a substantial amount of annotated data for training. This study aims to evaluate the effectiveness of using image-classification neural networks (NNs) for detecting and estimating weed coverage in bermudagrass turf. Weed-detection NNs, including DenseNet, GoogLeNet and ResNet, exhibited high overall accuracy and F1 scores (≥0.971) throughout the k-fold cross-validation. DenseNet outperformed GoogLeNet and ResNet with the highest overall accuracy and F1 scores (0.977). Among the evaluated NNs, DenseNet showed the highest overall accuracy and F1 scores (0.996) in the validation and testing data sets for estimating weed coverage. The inference speed of ResNet was similar to that of GoogLeNet but noticeably faster than DenseNet. ResNet was the most efficient and accurate deep convolution neural network for weed detection and coverage estimation. These results demonstrated that the developed NNs could effectively detect weeds and estimate their coverage in bermudagrass turf, allowing calculation of the herbicide requirements for variable-rate herbicide applications. The proposed method can be employed in a machine vision-based autonomous site-specific spraying system of smart sprayers. © 2024 Society of Chemical Industry.

The Evaluation of a Code-Switched Sepedi-English Automatic Speech Recognition System

Speech technology is a field that encompasses various techniques and tools used to enable machines to interact with speech, such as automatic speech recognition (ASR), spoken dialog systems, and others, allowing a device to capture spoken words through a microphone from a human speaker. End-to-end approaches such as Connectionist Temporal Classification (CTC) and attention-based methods are the most used for the development of ASR systems. However, these techniques were commonly used for research and development for many high-resourced languages with large amounts of speech data for training and evaluation, leaving low-resource languages relatively underdeveloped. While the CTC method has been successfully used for other languages, its effectiveness for the Sepedi language remains uncertain. In this study, we present the evaluation of the SepediEnglish code-switched automatic speech recognition system. This end-to-end system was developed using the Sepedi Prompted Code Switching corpus and the CTC approach. The performance of the system was evaluated using both the NCHLT Sepedi test corpus and the Sepedi Prompted Code Switching corpus. The model produced the lowest WER of 41.9%, however, the model faced challenges in recognizing the Sepedi only text

A two-stage framework for Arabic social media text misinformation detection combining data augmentation and AraBERT

Misinformation can profoundly impact the reputation of an entity, and eliminating its spread has become a critical concern across various applications. Social media, often a primary source of information, can significantly influence individuals’ perspectives through content from less credible sources. The utilization of machine-learning (ML) algorithms can facilitate automated, large-scale analysis of textual content, contributing to the rapid and efficient processing of extensive datasets for informed decision-making. Since the performance of ML models is highly affected by the size of the training data, many research papers have presented different approaches to solve the problem of limited dataset size. The data augmentation (DA) approach is one of these strategies, aiming to enhance ML model performance by increasing the amount of training data. DA generates new instances by applying different transformations to the original data instances. While many DA techniques have been investigated for various languages, such as English, achieving an enhancement of the classification model’s performance on the new augmented dataset compared to the original dataset, there is a lack of studies on the Arabic language due to its unique characteristics. This paper introduces a novel two-stage framework designed for the automated identification of misinformation in Arabic textual content. The first stage aims to identify the optimal representation of features before feeding them to the ML model. Diverse representations of tweet content are explored, including N-grams, content-based features, and source-based features. The second stage focuses on investigating the DA effect through the back-translation technique applied to the original training data. Back-translation entails translating sentences from the target language (in this case, Arabic) into another language and then back to Arabic. As a result of this procedure, new examples for training are created by introducing variances in the text. The study utilizes support vector machine (SVM), naive Bayes, logistic regression (LR), and random forest (RF) as baseline algorithms. Additionally, AraBERT transformer pre-trained language models are used to relate the instance’s label and feature representation of the input. Experimental outcomes demonstrate that misinformation detection, coupled with data augmentation, enhances accuracy by a noteworthy margin 5 to 12% compared to baseline machine-learning algorithms and pre-trained models. Remarkably, the results show the superiority of the N-grams approach over traditional state-of-the-art feature representations concerning accuracy, recall, precision, and F-measure metrics. This suggests a promising avenue for improving the efficacy of misinformation detection mechanisms in the realm of Arabic text analysis.

Hybrid supervised and reinforcement learning for the design and optimization of nanophotonic structures.

From higher computational efficiency to enabling the discovery of novel and complex structures, deep learning has emerged as a powerful framework for the design and optimization of nanophotonic circuits and components. However, both data-driven and exploration-based machine learning strategies have limitations in their effectiveness for nanophotonic inverse design. Supervised machine learning approaches require large quantities of training data to produce high-performance models and have difficulty generalizing beyond training data given the complexity of the design space. Unsupervised and reinforcement learning-based approaches on the other hand can have very lengthy training or optimization times associated with them. Here we demonstrate a hybrid supervised learning and reinforcement learning approach to the inverse design of nanophotonic structures and show this approach can reduce training data dependence, improve the generalizability of model predictions, and significantly shorten exploratory training times. The presented strategy thus addresses several contemporary deep learning-based challenges, while opening the door for new design methodologies that leverage multiple classes of machine learning algorithms to produce more effective and practical solutions for photonic design.

Small molecule autoencoders: architecture engineering to optimize latent space utility and sustainability

Autoencoders are frequently used to embed molecules for training of downstream deep learning models. However, evaluation of the chemical information quality in the latent spaces is lacking and the model architectures are often arbitrarily chosen. Unoptimized architectures may not only negatively affect latent space quality but also increase energy consumption during training, making the models unsustainable. We conducted systematic experiments to better understand how the autoencoder architecture affects the reconstruction and latent space quality and how it can be optimized towards the encoding task as well as energy consumption. We can show that optimizing the architecture allows us to maintain the quality of a generic architecture but using 97% less data and reducing energy consumption by around 36%. We additionally observed that representing the molecules as SELFIES reduced the reconstruction performance compared to SMILES and that training with enumerated SMILES drastically improved latent space quality. Scientific Contribution: This work provides the first comprehensive systematic analysis of how choosing the autoencoder architecture affects the reconstruction performance of small molecules, the chemical information content of the latent space as well as the energy required for training. Demonstrated on the MOSES benchmarking dataset it provides first valuable insights into how autoencoders for the embedding of small molecules can be designed to optimize their utility and simultaneously become more sustainable, both in terms of energy consumption as well as the required amount of training data. All code, data and model checkpoints are made available on Zenodo (Oestreich et al. Small molecule autoencoders: architecture engineering to optimize latent space utility and sustainability. Zenodo, 2024). Furthermore, the top models can be found on GitHub with scripts to encode custom molecules: https://github.com/MarieOestreich/small-molecule-autoencoders.Graphical

Deep Learning Network for Remote Monitoring of Thermal Exchange Tunnels

Monitoring and predicting ground settlement throughout tunnel construction is critical to ensuring the safe and accurate use of urban tunnel systems. The accurate and efficient diagnosis of such settlement can decrease hazards while improving the safety and dependability of these initiatives. However, typical tunnel inspection procedures are time-consuming, costly, and heavily reliant on human subjectivity. The trained model's accuracy was evaluated by comparing its findings across extended operating durations using the same and different thermal operational patterns as those utilized for training. Deep learning, one of the most powerful Artificial Intelligence approaches, is required for the tunnel's settlement predicting challenge. Nevertheless, deep neural networks frequently want huge quantities of training data. In the method we used, CNN-LSTM models were trained on datasets of various sizes and attributes. The results suggest that both of the proposed models may achieve a little inaccuracy under specific situations.

Predicting ocean pressure field with a physics-informed neural network.

Ocean sound pressure field prediction, based on partially measured pressure magnitudes at different range-depths, is presented. Our proposed machine learning strategy employs a trained neural network with range-depth as input and outputs complex acoustic pressure at the location. We utilize a physics-informed neural network (PINN), fitting sampled data while considering the additional information provided by the partial differential equation (PDE) governing the ocean sound pressure field. In vast ocean environments with kilometer-scale ranges, pressure fields exhibit rapidly fluctuating phases, even at frequencies below 100 Hz, posing a challenge for neural networks to converge to accurate solutions. To address this, we utilize the envelope function from the parabolic-equation technique, fundamental in ocean sound propagation modeling. The envelope function shows slower variations across ranges, enabling PINNs to predict sound pressure in an ocean waveguide more effectively. Additional PDE information allows PINNs to capture PDE solutions even with a limited amount of training data, distinguishing them from purely data-driven machine learning approaches that require extensive datasets. Our approach is validated through simulations and using data from the SWellEx-96 experiment.

Using artificial neural networks to predict indoor particulate matter and TVOC concentration in an office building: Model selection and method development

Indoor particulate matter and TVOC pose a significant threat to the health of staff working in office buildings. The accurate prediction of indoor particulate matter and TVOC concentrations is crucial for the automatic operation of ventilation and filtration system to control these indoor pollutants. Artificial neural networks have the advantages of high accuracy, real-time monitoring, and multi-source data integrations. Their performances in predicting particulate matter and TVOC concentrations in office buildings need to be further studied. In this study, on-site sampling was conducted in a typical office building at northern China. A standardized database of indoor parameters for four different offices was created with a total of 6072 datasets for each parameter. Three artificial neural network models were used in this study: the back propagation neural network (BP-ANN), the multi-layer neural network (MLNN), as well as the long-term and short-term memory neural network (LSTM). Among these three models, the performance of the MLNN model was the best in predicting PM2.5 and PM10 concentrations. It achieved a FB ranging from -0.02 to -0.01, an NMSE ranging from 0.46 to 0.49 μg/m3, and an R2 between 0.78 and 0.81. The MLNN model and the random forest (RF) classification method were further used to predict indoor TVOC concentrations. The RF model achieved a relatively better performance with a prediction accuracy of 89.2 %. In addition, the models’ generalization abilities were further evaluated by using some smaller datasets. For the MLNN model, when predicting indoor PM2.5 concentration, as the amount of training data decreased from 80 % to 20 %, its FB decreased from 0.41 to 0.03, its NMSE changed from 1.53 μg/m3 to 0.53 μg/m3, and its R2 decreased from 0.69 to 0.07. The results in this study can contribute to the use of artificial intelligence algorithm in office buildings aiming at indoor pollutants control.

Self-Supervised Deep Unrolled Reconstruction Using Regularization by Denoising.

Deep learning methods have been successfully used in various computer vision tasks. Inspired by that success, deep learning has been explored in magnetic resonance imaging (MRI) reconstruction. In particular, integrating deep learning and model-based optimization methods has shown considerable advantages. However, a large amount of labeled training data is typically needed for high reconstruction quality, which is challenging for some MRI applications. In this paper, we propose a novel reconstruction method, named DURED-Net, that enables interpretable self-supervised learning for MR image reconstruction by combining a self-supervised denoising network and a plug-and-play method. We aim to boost the reconstruction performance of Noise2Noise in MR reconstruction by adding an explicit prior that utilizes imaging physics. Specifically, the leverage of a denoising network for MRI reconstruction is achieved using Regularization by Denoising (RED). Experiment results demonstrate that the proposed method requires a reduced amount of training data to achieve high reconstruction quality among the state-of-the-art approaches utilizing Noise2Noise.

Enhancing Portfolio Performance through Financial Time-Series Decomposition-Based Variational Encoder-Decoder Data Augmentation

The objective of portfolio diversification is to reduce risk and potentially enhance returns by spreading investments across different asset classes. Existing portfolio diversification models have traditionally been trained on historical financial time series data. However, several issues arise with historical financial time series data, making it challenging to train models effectively to achieve the portfolio diversification objective: an insufficient amount of training data and the uncertainty deficiency problem, wherein the uncertainty that existed in the past is not visible in the present. Insufficient datasets, characterized by small data size, result in information asymmetry and compromise portfolio performance. This limitation underscores the importance of adopting a pattern-centric data augmentation approach, capable of unveiling hidden patterns and structures within the financial time series data. To address these challenges, this paper introduces the financial time series decomposition-based variational encoder-decoder (FED) method to augment financial time series data, overcoming the limitations of insufficient training data and providing a more realistic and dynamic simulation of the financial market environment. By decomposing the data into distinct components, such as trend, dispersion, and residual, FED leverages pattern-centric data augmentation within the financial time series data. In the environment generated using the FED method, this paper proposes a two-class portfolio diversification, called FED2Port. It integrates stochastic elements into the reward function, enabling a reinforcement learning algorithm to learn from a comprehensive spectrum of financial market uncertainties. The experimental results demonstrate that the proposed model significantly enhances portfolio performance.

A Neural Network Computational Spectrometer Trained by a Small Dataset with High-Correlation Optical Filters.

A computational spectrometer is a novel form of spectrometer powerful for portable in situ applications. In the encoding part of the computational spectrometer, filters with highly non-correlated properties are requisite for compressed sensing, which poses severe challenges for optical design and fabrication. In the reconstruction part of the computational spectrometer, conventional iterative reconstruction algorithms are featured with limited efficiency and accuracy, which hinders their application for real-time in situ measurements. This study proposes a neural network computational spectrometer trained by a small dataset with high-correlation optical filters. We aim to change the paradigm by which the accuracy of neural network computational spectrometers depends heavily on the amount of training data and the non-correlation property of optical filters. First, we propose a presumption about a distribution law for the common large training dataset, in which a unique widespread distribution law is shown when calculating the spectrum correlation. Based on that, we extract the original dataset according to the distribution probability and form a small training dataset. Then a fully connected neural network architecture is constructed to perform the reconstruction. After that, a group of thin film filters are introduced to work as the encoding layer. Then the neural network is trained by a small dataset under high-correlation filters and applied in simulation. Finally, the experiment is carried out and the result indicates that the neural network enabled by a small training dataset has performed very well with the thin film filters. This study may provide a reference for computational spectrometers based on high-correlation optical filters.

Assessment of Disordered Linker Predictions in the CAID2 Experiment.

Disordered linkers (DLs) are intrinsically disordered regions that facilitate movement between adjacent functional regions/domains, contributing to many key cellular functions. The recently completed second Critical Assessments of protein Intrinsic Disorder prediction (CAID2) experiment evaluated DL predictions by considering a rather narrow scenario when predicting 40 proteins that are already known to have DLs. We expand this evaluation by using a much larger set of nearly 350 test proteins from CAID2 and by investigating three distinct scenarios: (1) prediction residues in DLs vs. in non-DL regions (typical use of DL predictors); (2) prediction of residues in DLs vs. other disordered residues (to evaluate whether predictors can differentiate residues in DLs from other types of intrinsically disordered residues); and (3) prediction of proteins harboring DLs. We find that several methods provide relatively accurate predictions of DLs in the first scenario. However, only one method, APOD, accurately identifies DLs among other types of disordered residues (scenario 2) and predicts proteins harboring DLs (scenario 3). We also find that APOD's predictive performance is modest, motivating further research into the development of new and more accurate DL predictors. We note that these efforts will benefit from a growing amount of training data and the availability of sophisticated deep network models and emphasize that future methods should provide accurate results across the three scenarios.

Tabular text embedding for Vietnamese text-based person search

Vietnamese text-based person search is still a challenging problem with the limited dataset of Vietnamese descriptions. The current popular approach to this problem is Deep Neural Networks (DNNs), and recently, transformer networks have been more favored because of their outperformance over CNN and RNN networks for both vision and natural language processing tasks. However, DNN, or transformer networks, require a large amount of training data and computing time for efficient learning of visual and textual features. This brings a burden for implementing Vietnamese text-based person search by DNN, or transformer networks. Towards building a Vietnamese text-based person search system on a scarce resource dataset of Vietnamese descriptive sentences with low computing cost, in this work, we propose to apply the transformer-based architecture named TabTransformer for contextual embedding of the noun phrases chunked from the Vietnamese descriptive sentences. This is the first time the TabTransformer network has been deployed together with CNN and RNN architectures for Vietnamese text-based person search. The experimental results on a limited dataset of 3000VnPersonSearch show the better recognition accuracy of the proposed method compared to the baseline method by about 7.5% at Rank 1. In addition, the computing time of our method is more effective than the baseline method.

Graph-SeTES: A graph based search task extraction using Siamese network

Search task extraction is a sub-process for query suggestion/reformulation, personalized recommendation, and advertisement in search engines and e-commerce platforms. However, they face both internal and external challenges. Internal challenges include short and misspelled queries and incomplete keywords. External challenges include an unknown number of clusters and a limited number of labeled datasets. Deep-learning models require a large amount of data for training; however, search task datasets are rare and small. To overcome these limitations, we proposed Graph-SeTES, which integrates feature extraction with a decision network that utilizes both distance metrics and decision networks. The existing graph-clustering algorithms use the similarities between search query pairs. Because the Siamese network (SN) finds similarities between two objects, it fits search task extraction. Compared with existing models, SN requires fewer parameters for training due to the shared weights. However, it yields good results even with less labeled data, overcoming the external challenges. We benefit from both distance metrics and narrowing of the linear layers for decision networks. Graph-SeTES was compared with state-of-the-art models, and it outperformed its counterparts. The results were 6% better than those of the best baseline on the CSTE dataset, which maintained this performance difference on the WSMC12 dataset.

Automatic Text Labeling Method Based on Large Language Models

With the increasing demand for large amounts of training data for model development, this paper proposes LLM4Label, an automatic text labeling method based on large language models, to assist human labelers in annotating text data. LLM4Label first selects the most representative seed data using a clustering algorithm based on text similarity. It then constructs prompt dialogues with few-shot prompts to stimulate the language model’s performance on entity labeling tasks, enabling it to automatically and efficiently label more data. Finally, LLM4Label introduces human feedback to correct un- certain labeling results and retrains the model with the corrected annotations. Experiments show that LLM4Label achieves high- quality labeled data at low human labeling cost. The proposed method provides an effective way to obtain sizable and high- quality annotated datasets with minimal manual effort, which can strongly support downstream natural language processing tasks.

A study of data heterogeneity in federated learning

Data-driven artificial intelligence algorithms cannot do without large amounts of training data. However, the characteristics of data privacy and decentralization make constructing large-scale training data costly, which restricts the further application of artificial intelligence algorithms in different downstream fields. To address the above problems, federated learning has gradually attracted more and more research interests in recent years, which aims to utilize decentralized private data for model training while preserving privacy. However, the non-independent and homogeneous distribution of data across devices causes federated learning to face problems such as data imbalance and label bias, which in turn affects the generalization performance of the model. The problem of data heterogeneity has become a major key challenge in federated learning, and this paper aims to explore the impact of data heterogeneity on federated learning and to synthesize recent research results in this area. By analyzing different solution approaches from the aspects of adaptive data distribution, adding regularization terms, contrastive Learning, and multi-task learning, a comprehensive overview is provided for researchers. This paper further summarizes the existing challenges of data heterogeneity in the research field of federated learning and discusses its potential development directions.

A Transfer Learning Approach Based on Radar Rainfall for River Water-Level Prediction

River water-level prediction is crucial for mitigating flood damage caused by torrential rainfall. In this paper, we attempt to predict river water levels using a deep learning model based on radar rainfall data instead of data from upstream hydrological stations. A prediction model incorporating a two-dimensional convolutional neural network (2D-CNN) and long short-term memory (LSTM) is constructed to exploit geographical and temporal features of radar rainfall data, and a transfer learning method using a newly defined flow–distance matrix is presented. The results of our evaluation of the Oyodo River basin in Japan show that the presented transfer learning model using radar rainfall instead of upstream measurements has a good prediction accuracy in the case of torrential rain, with a Nash–Sutcliffe efficiency (NSE) value of 0.86 and a Kling–Gupta efficiency (KGE) of 0.83 for 6-h-ahead forecast for the top-four peak water-level height cases, which is comparable to the conventional model using upstream measurements (NSE = 0.84 and KGE = 0.83). It is also confirmed that the transfer learning model maintains its performance even when the amount of training data for the prediction site is reduced; values of NSE = 0.82 and KGE = 0.82 were achieved when reducing the training torrential-rain-period data from 12 to 3 periods (with 105 periods of data from other rivers for transfer learning). The results demonstrate that radar rainfall data and a few torrential rain measurements at the prediction location potentially enable us to predict river water levels even if hydrological stations have not been installed at the prediction location.