Overcome Data Scarcity Research Articles

Doing More with Less: Overcoming Data Scarcity for POI Recommendation via Cross-Region Transfer

Variability in social app usage across regions results in a high skew of the quantity and the quality of check-in data collected, which in turn is a challenge for effective location recommender systems. In this article, we present Axolotl ( Automated cross Location-network Transfer Learning ), a novel method aimed at transferring location preference models learned in a data-rich region to significantly boost the quality of recommendations in a data-scarce region. Axolotl predominantly deploys two channels for information transfer: (1) a meta-learning based procedure learned using location recommendation as well as social predictions, and (2) a lightweight unsupervised cluster-based transfer across users and locations with similar preferences. Both of these work together synergistically to achieve improved accuracy of recommendations in data-scarce regions without any prerequisite of overlapping users and with minimal fine-tuning. We build Axolotl on top of a twin graph-attention neural network model used for capturing the user- and location-conditioned influences in a user-mobility graph for each region. We conduct extensive experiments on 12 user mobility datasets across the US, Japan, and Germany, using three as source regions and nine of them (that have much sparsely recorded mobility data) as target regions. Empirically, we show that Axolotl achieves up to 18% better recommendation performance than the existing state-of-the-art methods across all metrics.

Automated defect detection in digital radiography of aerospace welds using deep learning

Aerospace welds are non-destructively evaluated (NDE) during manufacturing to identify defective parts that may pose structural risks, often using digital radiography. The analysis of these digital radiographs is time consuming and costly. Attempts to automate the analysis using conventional computer vision methods or shallow machine learning have not, thus far, provided performance equivalent to human inspectors due to the high reliability requirements and low contrast to noise ratio of the defects. Modern approaches based on deep learning have made considerable progress towards reliable automated analysis. However, limited data sets render current machine learning solutions insufficient for industrial use. Moreover, industrial acceptance would require performance demonstration using standard metrics in non-destructive evaluation, such as probability of detection (POD), which are not commonly used in previous studies. In this study, data augmentation with virtual flaws was used to overcome data scarcity, and compared with conventional data augmentation. A semantic segmentation network was trained to find defects from computed radiography data of aerospace welds. Standard evaluation metrics in non-destructive testing were adopted for the comparison. Finally, the network was deployed as an inspector’s aid in a realistic environment to predict flaws from production radiographs. The network achieved high detection reliability and defect sizing performance, and an acceptable false call rate. Virtual flaw augmentation was found to significantly improve performance, especially for limited data set sizes, and for underrepresented flaw types even at large data sets. The deployed prototype was found to be easy to use indicating readiness for industry adoption.

Plain Template Insertion: Korean-Prompt-Based Engineering for Few-Shot Learners

Prompt-based learning is a method used for language models to interpret natural language by remembering the prior knowledge acquired and the training objective. Recent prompt-based few-shot learners have achieved superior performance by alleviating the catastrophic forgetting that occurs in pretrained language models. Few-shot learning contributes towards solving the data scarcity problem, an enormous challenge in AI systems and a significant consideration in natural language processing research. In spite of the significance of few-shot learning, research on Korean language-based few-shot learning is insufficient, and whether the prompt-based approach is appropriate for the Korean language has not been thoroughly verified. As a step toward realizing a Korean-prompt-based few-shot learner, we attempt to apply prompt engineering to the Korean language understanding benchmark dataset and introduce plain template insertion to overcome data scarcity in a more practical few-shot setting. The contributions of this study are as follows: (1) presumably, this is the first study to apply prompt-based few-shot learning to Korean benchmark datasets. With 32 few-shot settings, it improves performance by +14.88, +29.04, and +1.81 in the natural language inference, semantic textual similarity, and topic classification tasks. (2) We present prompt engineering, which merely inserts a plain template and increases data efficiency without training example selection, augmentation, reformulation, and retrieval. (3) Our approach is robust to the Korean prompt’s contextual information and sentence structure and is applicable to both hard- and soft-prompt.

CRNNTL: Convolutional Recurrent Neural Network and Transfer Learning for QSAR Modeling in Organic Drug and Material Discovery.

Molecular latent representations, derived from autoencoders (AEs), have been widely used for drug or material discovery over the past couple of years. In particular, a variety of machine learning methods based on latent representations have shown excellent performance on quantitative structure–activity relationship (QSAR) modeling. However, the sequence feature of them has not been considered in most cases. In addition, data scarcity is still the main obstacle for deep learning strategies, especially for bioactivity datasets. In this study, we propose the convolutional recurrent neural network and transfer learning (CRNNTL) method inspired by the applications of polyphonic sound detection and electrocardiogram classification. Our model takes advantage of both convolutional and recurrent neural networks for feature extraction, as well as the data augmentation method. According to QSAR modeling on 27 datasets, CRNNTL can outperform or compete with state-of-art methods in both drug and material properties. In addition, the performances on one isomers-based dataset indicate that its excellent performance results from the improved ability in global feature extraction when the ability of the local one is maintained. Then, the transfer learning results show that CRNNTL can overcome data scarcity when choosing relative source datasets. Finally, the high versatility of our model is shown by using different latent representations as inputs from other types of AEs.

Evaluation of Areal Monthly Average Precipitation Estimates from MERRA2 and ERA5 Reanalysis in a Colombian Caribbean Basin

Global reanalysis dataset estimations of climate variables constitute an alternative for overcoming data scarcity associated with sparsely and unevenly distributed hydrometeorological networks often found in developing countries. However, reanalysis datasets require detailed validation to determine their accuracy and reliability. This paper evaluates the performance of MERRA2 and ERA5 regarding their monthly rainfall products, comparing their areal precipitation averages with estimates based on ground measurement records from 49 rain gauges managed by the Institute of Hydrology, Meteorology, and Environmental Studies (IDEAM) and the Thiessen polygons method in the Sinu River basin, Colombia. The performance metrics employed in this research are the correlation coefficient, the bias, the normalized root mean square error (NRMSE), and the Nash–Sutcliffe efficiency (NSE). The results show that ERA5 generally outperforms MERRA2 in the study area. However, both reanalyses consistently overestimate the monthly averages calculated from IDEAM records at all time and spatial scales. The negative NSE values indicate that historical monthly averages from IDEAM records are better predictors than both MERRA2 and ERA5 rainfall products.

Transfer learning compensates limited data, batch effects and technological heterogeneity in single-cell sequencing.

Tremendous advances in next-generation sequencing technology have enabled the accumulation of large amounts of omics data in various research areas over the past decade. However, study limitations due to small sample sizes, especially in rare disease clinical research, technological heterogeneity and batch effects limit the applicability of traditional statistics and machine learning analysis. Here, we present a meta-transfer learning approach to transfer knowledge from big data and reduce the search space in data with small sample sizes. Few-shot learning algorithms integrate meta-learning to overcome data scarcity and data heterogeneity by transferring molecular pattern recognition models from datasets of unrelated domains. We explore few-shot learning models with large scale public dataset, TCGA (The Cancer Genome Atlas) and GTEx dataset, and demonstrate their potential as pre-training dataset in other molecular pattern recognition tasks. Our results show that meta-transfer learning is very effective for datasets with a limited sample size. Furthermore, we show that our approach can transfer knowledge across technological heterogeneity, for example, from bulk cell to single-cell data. Our approach can overcome study size constraints, batch effects and technical limitations in analyzing single-cell data by leveraging existing bulk-cell sequencing data.

Evaluation of the benefits of improved permeability estimation on high-resolution characterization of DNAPL distribution in aquifers with low-permeability lenses

High-resolution characterization of dense non-aqueous phase liquid (DNAPL) source zone architecture (SZA) is crucial for developing remediation strategies. Inverse modeling approaches are widely used to characterize DNAPL SZA. However, unknown parameters (permeability and DNAPL saturation) are usually assumed as Gaussian distribution in traditional geostatistical inverse methods (e.g., ensemble smoother with multiple data assimilation, ESMDA). This assumption may not be true in channels of fluvial deposits. The presence of clay lenses will bring significant challenges in reconstructing the DNAPL SZA using geostatistical inverse methods based on the Gaussian assumption. Moreover, the DNAPL characterization also suffers from a limited number of borehole data in the field. In this study, we use ESMDA's variant, ensemble smoother with multiple data assimilation-direct sampling (ESMDA-DS), to improve the estimation of non-Gaussian permeability field and evaluate the benefits of improved permeability estimation on the reconstruction of DNAPL source zone. The inversion technique is based on the iterative inversion framework to incorporate the clay content information into the DNAPL characterization. The performance of our proposed framework is examined and discussed through a numerical synthetic case with challenging pool-dominated SZA. To overcome data scarcity, we integrated hydrogeological and geophysical (electrical resistance tomography) datasets. Results show that the permeability is characterized well using ESMDA-DS method compared with ESMDA method. With the improved permeability estimation, the DNAPL source zone can be characterized in higher resolution. Compared with ESMDA method, the root mean square error (RMSE) of permeability is reduced by 38.5% using the ESMDA-DS method. Then the estimation RMSE of DNAPL saturation is reduced by 24.9% by improved permeability estimation. We also demonstrated the significant benefits of improved permeability estimation on DNAPL depletion behavior and plume evolution. Numerical simulations demonstrate with the improved permeability estimation, the DNAPL depletion prediction is highly improved, the prediction RMSE of the DNAPL dissolved plume is reduced by 34.5% after 50 years.

Document image layout analysis via explicit edge embedding network

Layout analysis from a document image plays an important role in document content understanding and information extraction systems. While many existing methods focus on learning knowledge with convolutional networks directly from color channels, we argue the importance of high-frequency structures in document images, especially edge information. In this paper, we present a novel document layout analysis framework with the Explicit Edge Embedding Network (E3Net). Specifically, the proposed network contains the edge embedding block and dynamic skip connection block to produce detailed features, as well as a lightweight fully convolutional subnet as the backbone for the effectiveness of the framework. The edge embedding block is designed to explicitly incorporate the edge information from the document images. The dynamic skip connection block aims to learn both color and edge representations with learnable weights. In contrast to the previous methods, we harness the model by using a synthetic document approach to overcome data scarcity. The combination of data augmentation and edge embedding is important toward a more compact representation than directly using the training images with only color channels. We conduct experiments using the proposed framework on three document layout analysis benchmarks and demonstrate its superiority in terms of effectiveness and efficiency over previous approaches.

Reproduction in the Archipelago of the Aegean 1920–2016: Long-Term Trends and Recent Upturns

Little is known about the reproductive behavior of the population in insular Greece because of the lack of reliable and unintermittent time series of vital statistics. By applying methods used in historical demography and by indirectly estimating demographic indexes from census returns we try to overcome data scarcity. Substantial differences were found between the islands and mainland Greece. Differences were also found between different groups of islands, with the North Aegean being more conservative as far as the procreative ethics are concerned. Out of wedlock childbearing is less common in North Aegean islands than in Dodecanese or in the Cyclades.

User-oriented hydrological indices for early warning systems with validation using post-event surveys: flood case studies in the Central Apennine District

Abstract. Floods and flash floods are complex events, depending on weather dynamics, basin physiographical characteristics, land use cover and water management. For this reason, the prediction of such events usually deals with very accurate model tuning and validation, which is usually site-specific and based on climatological data, such as discharge time series or flood databases. In this work, we developed and tested two hydrological-stress indices for flood detection in the Italian Central Apennine District: a heterogeneous geographical area, characterized by complex topography and medium-to-small catchment extension. The proposed indices are threshold-based and developed considering operational requirements of National Civil Protection Department end-users. They are calibrated and tested through the application of signal theory, in order to overcome data scarcity over ungauged areas, as well as incomplete discharge time series. The validation has been carried out on a case study basis, using flood reports from various sources of information, as well as hydrometric-level time series, which represent the actual hydrological quantity monitored by Civil Protection operators. Obtained results show that the overall accuracy of flood prediction is greater than 0.8, with false alarm rates below 0.5 and the probability of detection ranging from 0.51 to 0.80. Moreover, the different nature of the proposed indices suggests their application in a complementary way, as the index based on drained precipitation appears to be more sensitive to rapid flood propagation in small tributaries, while the discharge-based index is particularly responsive to main-channel dynamics.

Daily flow simulation in Thailand Part II: Unraveling effects of reservoir operation

Study regionUpper region of the Greater Chao Phraya River (GCPR) basin in Thailand. Study focusThe upper GCPR basin is highly regulated by multipurpose reservoirs, which obviously have altered the natural streamflow. Understanding quantitative effects of such alteration is crucial for effective water resource management. Therefore, this study aims to assess how reservoir operation affects the water balance, daily flow regime and extreme flows in this basin. For this purpose, we reconstructed streamflow in the naturalized (no reservoir) and baseline operation scenarios using the (∼1 km resolution) distributed model. To overcome data scarcity, we ran the model with global data and parameterization. A target storage-and-release-based reservoir operation module was applied in the baseline operation scenario. The model results were analyzed in comparison to observations in a wet year, a dry year, and the period 1989–2014. New hydrological insights for the regionThe reservoir operation resulted in more evaporation. It inverted the natural flow seasonality and smoothed the daily flow regime with decreasing high flows, increasing mean flows and low flows, greater baseflow contribution, and lower flashiness. It prevented or mitigated many historical extreme flow incidents. The annual flood peaks and minimum flows were markedly mitigated in terms of both magnitudes and frequencies, but their timing became more variable and difficult to predict. Altogether, the results highlighted the importance of effective decision making for real-time operation, which remain challenging in practice.

A Data Augmentation-Based Evaluation System for Regional Direct Economic Losses of Storm Surge Disasters.

The accurate prediction of storm surge disasters’ direct economic losses plays a positive role in providing critical support for disaster prevention decision-making and management. Previous researches on storm surge disaster loss assessment did not pay much attention to the overfitting phenomenon caused by the data scarcity and the excessive model complexity. To solve these problems, this paper puts forward a new evaluation system for forecasting the regional direct economic loss of storm surge disasters, consisting of three parts. First of all, a comprehensive assessment index system was established by considering the storm surge disasters’ formation mechanism and the corresponding risk management theory. Secondly, a novel data augmentation technique, k-nearest neighbor-Gaussian noise (KNN-GN), was presented to overcome data scarcity. Thirdly, an ensemble learning algorithm XGBoost as a regression model was utilized to optimize the results and produce the final forecasting results. To verify the best-combined model, KNN-GN-based XGBoost, we conducted cross-contrast experiments with several data augmentation techniques and some widely-used ensemble learning models. Meanwhile, the traditional prediction models are used as baselines to the optimized forecasting system. The experimental results show that the KNN-GN-based XGBoost model provides more precise predictions than the traditional models, with a 64.1% average improvement in the mean absolute percentage error (MAPE) measurement. It could be noted that the proposed evaluation system can be extended and applied to the geography-related field as well.

Exploring Factors Affecting Sustainable Innovation Performance of Food Firms. A Case of Korean Food Industry

This study examines the influence of government support, research and development (R&D) investment and cooperation networks on sustainable innovation performance of the medium-sized food firms in South Korea. Climate change, population growth, lifestyle change and consumer demands for new products allow firms to explore innovation for sustainable growth. In the food industry, where various stakeholders are involved in the value chain, government support, R&D investment and cooperation are affecting firms’ sustainable performance. However, there are few empirical studies that examine the relationship among them, especially regarding medium-sized food firms. This study aimed to analyze data regarding Korean food firms through a structural equation model. To overcome data scarcity, this study collected new data for 67 medium-sized food firms. The data is matched with the firms’ 10-year data on management, innovation performance, and cooperation networks. This study can contribute to filling the literature gap by exploring what innovation strategies should be used to improve food SMEs’ innovation performance in the latecomer countries, which have developed their food industries relatively recently.

Anticancer drug synergy prediction in understudied tissues using transfer learning.

Drug combination screening has advantages in identifying cancer treatment options with higher efficacy without degradation in terms of safety. A key challenge is that the accumulated number of observations in in-vitro drug responses varies greatly among different cancer types, where some tissues are more understudied than the others. Thus, we aim to develop a drug synergy prediction model for understudied tissues as a way of overcoming data scarcity problems. We collected a comprehensive set of genetic, molecular, phenotypic features for cancer cell lines. We developed a drug synergy prediction model based on multitask deep neural networks to integrate multimodal input and multiple output. We also utilized transfer learning from data-rich tissues to data-poor tissues. We showed improved accuracy in predicting synergy in both data-rich tissues and understudied tissues. In data-rich tissue, the prediction model accuracy was 0.9577 AUROC for binarized classification task and 174.3 mean squared error for regression task. We observed that an adequate transfer learning strategy significantly increases accuracy in the understudied tissues. Our synergy prediction model can be used to rank synergistic drug combinations in understudied tissues and thus help to prioritize future in-vitro experiments. Code is available at https://github.com/yejinjkim/synergy-transfer.

A Comparison of Architectures and Pretraining Methods for Contextualized Multilingual Word Embeddings

The lack of annotated data in many languages is a well-known challenge within the field of multilingual natural language processing (NLP). Therefore, many recent studies focus on zero-shot transfer learning and joint training across languages to overcome data scarcity for low-resource languages. In this work we (i) perform a comprehensive comparison of state-of-the-art multilingual word and sentence encoders on the tasks of named entity recognition (NER) and part of speech (POS) tagging; and (ii) propose a new method for creating multilingual contextualized word embeddings, compare it to multiple baselines and show that it performs at or above state-of-the-art level in zero-shot transfer settings. Finally, we show that our method allows for better knowledge sharing across languages in a joint training setting.

Overcoming Data Scarcity in Earth Science

The Data Scarcity problem is repeatedly encountered in environmental research. This may induce an inadequate representation of the response’s complexity in any environmental system to any input/change (natural and human-induced). In such a case, before getting engaged with new expensive studies to gather and analyze additional data, it is reasonable first to understand what enhancement in estimates of system performance would result if all the available data could be well exploited. The purpose of this Special Issue, “Overcoming Data Scarcity in Earth Science” in the Data journal, is to draw attention to the body of knowledge that leads at improving the capacity of exploiting the available data to better represent, understand, predict, and manage the behavior of environmental systems at meaningful space-time scales. This Special Issue contains six publications (three research articles, one review, and two data descriptors) covering a wide range of environmental fields: geophysics, meteorology/climatology, ecology, water quality, and hydrology.

Developing in-vehicular noise robust children ASR system using Tandem-NN-based acoustic modelling

Processing of children's speech is always challenging due to data scarcity and inefficient modelling input feature vectors. Accuracy of the modelling phase is always dependent upon extracted input features. In this paper, posterior probabilities are estimated over a phone set using first discriminatively trained model through neural-net pre-processor. This Neural Network (NN) classifier is first trained on original speech and then context-independent phone posterior probabilities are estimated on Tandem-NN system. The output vectors are employed as default features which are processed on Deep Neural Network-Hidden Markov Model (DNN-HMM) models. The original data-based system performance is improved by extending it using data augmentation. To see the robustness of the augmented speech various in-vehicle data are investigated and found that it is superior to that of other systems. Finally, we combine all augmented data to overcome data scarcity challenges to enhance system performance. It gives a relative improvement of 23.77% over the baseline system.

Developing in-vehicular noise robust children ASR system using Tandem-NN-based acoustic modelling

Processing of children's speech is always challenging due to data scarcity and inefficient modelling input feature vectors. Accuracy of the modelling phase is always dependent upon extracted input features. In this paper, posterior probabilities are estimated over a phone set using first discriminatively trained model through neural-net pre-processor. This Neural Network (NN) classifier is first trained on original speech and then context-independent phone posterior probabilities are estimated on Tandem-NN system. The output vectors are employed as default features which are processed on Deep Neural Network-Hidden Markov Model (DNN-HMM) models. The original data-based system performance is improved by extending it using data augmentation. To see the robustness of the augmented speech various in-vehicle data are investigated and found that it is superior to that of other systems. Finally, we combine all augmented data to overcome data scarcity challenges to enhance system performance. It gives a relative improvement of 23.77% over the baseline system.

A novel tool for Bayesian reliability analysis using AHP as a framework for prior elicitation

Quantitative risk assessment (QRA) is a powerful and popular technique to support risk-based decisions. Unfortunately, QRAs are often hampered by significant uncertainty in the frequency of failure estimation for physical assets. This uncertainty is largely due to lack of quality failure data in published sources. The failure data may be limited, incompatible and/or outdated. Consequently, there is a need for robust methods and tools that can incorporate all available information to facilitate reliability analysis of critical assets such as pipelines, pressure vessels, rotating equipment, etc. This paper presents a novel practical approach that can be used to help overcome data scarcity issues in reliability analysis. A Bayesian framework is implemented to cohesively integrate objective data with expert opinion with the aim toward deriving time to failure distributions for physical assets. The Analytic Hierarchy Process is utilized to aggregate time to failure estimates from multiple experts to minimize biases and address inconsistencies in their estimates. These estimates are summarized in the form of informative priors that are implemented in a Bayesian update procedure for the Weibull distribution. The flexibility of the proposed methodology allows for efficiently dealing with data limitations. Application of the proposed approach is illustrated using a case study.

Integrating sketch mapping and hot spot analysis to enhance capacity for community‐level flood and disaster risk management

This community‐based research aims to enhance local‐level flood management by utilising participatory geographic information systems (PGIS) methods to capture the spatial dimensions of community member flooding concerns in Hopkins Village, Belize. We offer a mixed methodology, applying participatory sketch mapping as a way to collect local knowledge about community perceptions of flooding in this data‐scarce context. We combine this local knowledge with quantitative geostatistical hot spot analysis of basic village infrastructure characteristics to reveal insights about community perceptions of and response to flood risk. The significance of this research lies in the application of PGIS methods to create two different primary datasets, which when analysed together offer a more complete story about community understanding and needs for flood management. One set of data (more qualitative in nature) originated from sketch maps with community members and answers descriptive questions about how people spatially conceptualise hydro‐meteorological hazards within their community. The other (more quantitative in nature) is the village's first publicly accessible infrastructure dataset (including information on building structures, roads, and drainage infrastructure) digitised by our research team from high‐resolution drone imagery. Attributes for the infrastructure dataset were developed in collaboration with community members to reflect their desires for data and information suited to conduct flood vulnerability assessment. Application of thematic coding and hot spot analysis to the data reveals concerns about hazards within their community and the measures they have taken to protect themselves in the absence of resources to develop community‐wide mitigation plans. Our work offers contributions to the participatory GIS literature by providing an example of sketch mapping methods to facilitate multi‐level management of community‐level flood risk in a data‐scarce setting, and demonstrating the multiple benefits of applied participatory GIS methods as a means of overcoming data scarcity.