Sparse Datasets Research Articles

Parallel learning by multitasking neural networks

Parallel learning, namely the simultaneous learning of multiple patterns, constitutes a modern challenge for neural networks. While this cannot be accomplished by standard Hebbian associative neural networks, in this paper we show how the multitasking Hebbian network (a variation on the theme of the Hopfield model, working on sparse datasets) is naturally able to perform this complex task. We focus on systems processing in parallel a finite (up to logarithmic growth in the size of the network) number of patterns, mirroring the low-storage setting of standard associative neural networks. When patterns to be reconstructed are mildly diluted, the network handles them hierarchically, distributing the amplitudes of their signals as power laws w.r.t. the pattern information content (hierarchical regime), while, for strong dilution, the signals pertaining to all the patterns are simultaneously raised with the same strength (parallel regime). Further, we prove that the training protocol (either supervised or unsupervised) neither alters the multitasking performances nor changes the thresholds for learning. We also highlight (analytically and by Monte Carlo simulations) that a standard cost function (i.e. the Hamiltonian) used in statistical mechanics exhibits the same minima as a standard loss function (i.e. the sum of squared errors) used in machine learning.

GMAT-DU: Traffic Anomaly Prediction With Fine Spatiotemporal Granularity in Sparse Data

The fine-grained prediction of traffic anomalies is crucial for Traffic Management Bureau to alleviate congestion and avoid public safety incidents. While in practice, the fine-grained prediction is very challenging due to two issues. 1) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Data sparsity</i> . At the fine-grained setting, missing data is inevitable and widespread on spatial and temporal dimension. Existing methods have weak performance as they do not handle missing data properly. 2) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Data distribution mutation</i> . At the fine-grained setting, the traffic conditions of adjacent road segments are sometimes completely different, invalidating existing spatiotemporal smoothing-based methods. This paper proposes GMAT-DU, a novel model that aims to predict traffic anomaly from sparse data in fine-grained manner. To solve the first issue, we propose a Decay Unrolling (DU) mechanism to make the model applicable to sparse datasets. The performance will be progressively enhanced by the spatiotemporal unrolling of high-impact neighbors. For the second issue, we combine the meta-features of roads with correlations between roads, which are learnt from road semantic information and historical spatiotemporal data, and make the model focusing on the high-impact neighbors by a Graph Meta-features based ATtention (GMAT) mechanism. Extensive experiments on two real-world datasets validate the effectiveness of our method. The experiment results show the significant advantages against the state-of-the-art models.

Poster] Identifying Maintenance, Repair and Overhaul Events based on ADS-B Data

Maintenance, Repair and Overhaul (MRO) plays a big role in aviation schedules and operational costs. In this work service difficulty reports are combined with aircraft ADS-B data to identify MRO activity based on an aircraft's layover information. The ``Inspection/Maintenance'' stage of operation and date of service difficulty reports allow to label layovers from ADS-B with the presence of MRO activity. Using a support vector machine this work classifies layovers based on their duration and time of day, achieving an 86% overall accuracy and 78% balanced accuracy. These results reflect the difficulty of working with this sparse and incomplete dataset, however, they allow us to study MRO schedules. The combination of these data sources offers new opportunities to find relations between aircraft trajectory information and MRO, which will be further investigated in future work.

A Deep Learning Approach Based on Physical Constraints for Predicting Soil Moisture in Unsaturated Zones

AbstractWater transport in the unsaturated zone is an important part of the hydrological cycle and is the link between the atmosphere‒soil‐groundwater for material and energy transport. The accurate prediction of soil moisture (SM) is essential for the rational exploitation of water resources. Data‐driven deep learning methods are widely used in many fields; however, the lack of physical mechanisms limits their application in hydrological fields, especially for SM prediction in unsaturated zones. To solve this problem, this study proposes a new deep learning method that introduces the water balance principle, Richard's equation, and SM boundary conditions as constraints to construct the new loss function that guides the training process of deep learning, called physics‐informed deep learning (PIDL). In tests consisting of a large number of data sets acquired from in situ observation sites in the field, PIDL exhibits higher accuracy than ordinary deep learning (long short‐term memory) and physical models, with 51.03% and 53.46% reduction in root mean square error of SM prediction, respectively. PIDL performance significantly improved in predicting scenarios that are difficult for ordinary deep learning to handle, such as sparse data sets, extreme values, and mutated values. In addition, PIDL maintains high accuracy over a longer prediction period. The addition of physical mechanisms allows deep learning to mine patterns not only from the data itself but also from a priori physical theoretical knowledge for guidance, and this hybrid modeling approach can also be generalized to prediction problems in other hydrological domains.

Hybrid Temporal Dynamics Feature Extraction in Recommendation Systems for Improved Ranking of Items

In today's retail landscape, shopping malls and e-commerce platforms employ various psychological tactics to influence customer behavior and increase profits. In line with these strategies, this paper introduces an innovative method for recognizing sentiment patterns, with a specific emphasis on the evolving temporal aspects of user interests within Recommendation Systems (RS). The projected method, called Temporal Dynamic Features based User Sentiment Pattern for Recommendation System (TDF-USPRS), aims to enhance the performance of RS by leveraging sentiment trends derived from a user's past preferences. TDF-USPRS utilizes a hybrid model combining Short Time Fourier Transform (STFT) and a layered architecture based on Bidirectional Long Short-Term Memory (BiLSTM) to retrieve temporal dynamics and discern a user's sentiment trend. Through an examination of a user's sequential history of item preferences, TDF-USPRS produces sentiment patterns to offer exceptionally pertinent recommendations, even in cases of sparse datasets. A variety of popular datasets, including as MovieLens, Amazon Rating Beauty, YOOCHOOSE, and CiaoDVD are utilised to assess the suggested technique. The TDF-USPRS model outperforms existing approaches, according to experimental data, resulting in recommendations with greater accuracy and relevance. Comparing the projected model to existing approaches, the projected model displays a 6.5% reduction in RMSE and a 4.5% gain in precision. Specifically, the model achieves an RMSE of 0.7623 and 0.996 on the MovieLens and CiaoDVD datasets, while attaining a precision score of 0.5963 and 0.165 on the YOOCHOOSE and Amazon datasets, respectively.

Fast Uncertainty Quantification by Sparse Data Learning From Multiphysics Systems

Data learning is a burgeoning area; here, we refer to it as a gorgeous blend of scientific computing and high-dimensional statistics. In this article, we propose a generalized polynomial chaos expansion (gPCE)-based surrogate model, with a sparse dataset, for multiphysics problems with associated uncertainty. This data-driven method avoids the dilemma of traditional intrusive methods, which usually require tremendous revisions over the backend source code. To circumvent the curse of dimensionality in uncertainty quantification, we perform sparse data learning for the relationship between the system output and the gPCE bases. Numerical experiments showcase that the surrogate model exhibits superior fast performance in system-output prediction, statistical-feature extraction, and sensitivity analysis of the 3-D multiphysics problems.

MSGNN: A Multi-structured Graph Neural Network model for real-time incident prediction in large traffic networks

This study addresses the problem of predicting traffic incidents across a large-scale road network to support network-wide real-time traffic management. To enable network-wide incident prediction, traditional approaches either build a separate model for each target link (road segment) across the network or build one big model containing all the links in the network, which are costly and inefficient. Instead, we propose a sub-area level incident prediction model that can predict an incident occurrence within ‘any’ given sub-area across the network using a single model, where this single network-wide model learns incident occurrence patterns from a large number of randomly sampled sub-areas across the whole network. For this, we develop a Multi-structured Graph Neural Network (MSGNN) model that effectively captures spatio-temporal relationships among links within each sub-area, where multiple graphs with different structures are formed to represent different data sources for the same geographic sub-area and the combined graph embeddings from those multiple graphs are taken as input to predict a binary classification label for incident occurrence as output. We conduct a comprehensive set of experiments for model evaluation. When compared with benchmark models, which are chosen from classical machine learning models as well as recent deep learning models, our model outperforms the other models in most of the test cases, showing superior performance consistently across different parameter settings (15-min and 60-min prediction horizons) and different study networks (Brisbane and Gold Coast). We also demonstrate the benefit of the multi-structured graph input architecture in flexibly fusing heterogeneous data sources to enhance the model accuracy, in conjunction with a novel clustering-based data imputation method that allows us to fully leverage even a sparse dataset with many missing data.

Sequential Bayesian updating for time-variant reliability analysis of ageing structures

This paper proposes a new method for time-variant reliability analysis of existing ageing structures under epistemic uncertainty from limited recurrent field measurement data. A growth model and a noise factor are introduced to account for the ageing changes of distribution parameters that take place while the reliability interval is sequentially updated using inspection data acquired over the service period. The method is applied to quantify the reliability of deteriorating concrete bridges under routine inspections of reinforcement steel corrosion. The interval estimation of corrosion and reliability shows narrower uncertainty bounds than a traditional frequentist approach, particularly when relatively sparse inspection datasets are available. In the case study presented, the relative variation of reliability bounds could be reduced by at least half when compared to a frequentist estimation.

Providing prediction reliability through deep neural networks for recommender systems

Deep learning-based recommendation approaches have shown significant improvement in the accuracy of recommender systems (RSs). However, beyond accuracy, reliability measures are gaining attention to evaluate the validity of predictions and enhance user satisfaction. Such measures can ensure that the recommended items are high-scoring items with high reliability. To integrate the native concept of reliability into a deep learning model, this paper proposes a deep neural network-based recommendation framework with prediction reliability. This framework filters out unreliable prediction ratings according to a pre-defined reliability threshold, ensuring the credibility and reliability of top-N recommendation. The proposed framework relies solely on user ratings for reliability, making it highly generalizable and scalable. Additionally, we design a data pre-processing method to address the issue of uneven distribution of ratings before model training, which effectively improves the effectiveness and fairness. The experiments on four benchmark datasets demonstrate that the proposed scheme is superior to other comparison methods in evaluation metrics. Furthermore, our framework performs better on sparse datasets than on dense datasets, indicating its ability to make strong predictions even with insufficient information.

3D inversion of multifrequency controlled-source electromagnetic data and its application to geothermal exploration in the Tianzhen region of the northern Datong Basin, China

Owing to its sensitivity to hydrothermal alteration and geothermal fluids, the distribution of subsurface electrical conductivity provides critical information for characterizing geothermal systems. For geothermal exploration, the controlled-source electromagnetic (CSEM) technique provides a crucial tool to image the subsurface resistivity structures, especially in areas with strong cultural noise. Usually, land-based CSEM surveys are carried out with dozens of operating frequencies to enhance spatial resolution. However, the 3D inversion of multifrequency CSEM data using the commonly used direct solver is challenging with limited computational resources. In this study, we implement a practical inversion strategy for interpreting 3D multifrequency CSEM data. By combining a hybrid direct-iterative solver with the inexact Gauss-Newton optimization, 3D inversion of CSEM data involving multiple frequencies can be performed effectively on typical workstations. First, we test the effectiveness of the developed approach using synthetic multifrequency 3D CSEM data generated for a simplified geothermal model. The inversion strategy is then applied to the multifrequency CSEM field data collected for the geothermal exploration at Tianzhen region in Shanxi Province, China. The comparison between the inversion results using the sparse data set (six frequencies) and the dense data set (12 frequencies) highlights the necessity of using data from more frequencies in the inversion to improve the resolution. The resulting 3D resistivity model clearly delineates the clay alteration layers and shallow thermal reservoirs of the potential high-temperature geothermal system within the study area, while its deep heat source is not revealed owing to the limited investigation depth of the survey.

What is the ecotoxicity of a given chemical for a given aquatic species? Predicting interactions between species and chemicals using recommender system techniques

ABSTRACT Ecotoxicological safety assessment of chemicals requires toxicity data on multiple species, despite the general desire of minimizing animal testing. Predictive models, specifically machine learning (ML) methods, are one of the tools capable of solving this apparent contradiction as they allow to generalize toxicity patterns across chemicals and species. However, despite the availability of large public toxicity datasets, the data is highly sparse, complicating model development. The aim of this study is to provide insights into how ML can predict toxicity using a large but sparse dataset. We developed models to predict LC50-values, based on experimental LC50-data covering 2431 organic chemicals and 1506 aquatic species from the ECOTOX-database. Several well-known ML techniques were evaluated and a new ML model was developed, inspired by recommender systems. This new model involves a simple linear model that learns low-rank interactions between species and chemicals using factorization machines. We evaluated the predictive performances of the developed models based on two validation settings: 1) predicting unseen chemical-species pairs, and 2) predicting unseen chemicals. The results of this study show that ML models can accurately predict LC50-values in both validation settings. Moreover, we show that the novel factorization machine approach can match well-tuned, complex, ML approaches.

Extremely missing numerical data in Electronic Health Records for machine learning can be managed through simple imputation methods considering informative missingness: A comparative of solutions in a COVID-19 mortality case study

Background and objectiveReusing Electronic Health Records (EHRs) for Machine Learning (ML) leads on many occasions to extremely incomplete and sparse tabular datasets, which can hinder the model development processes and limit their performance and generalization. In this study, we aimed to characterize the most effective data imputation techniques and ML models for dealing with highly missing numerical data in EHRs, in the case where only a very limited number of data are complete, as opposed to the usual case of having a reduced number of missing values. MethodsWe used a case study including full blood count laboratory data, demographic and survival data in the context of COVID-19 hospital admissions and evaluated 30 processing pipelines combining imputation methods with ML classifiers. The imputation methods included missing mask, translation and encoding, mean imputation, k-nearest neighbors’ imputation, Bayesian ridge regression imputation and generative adversarial imputation networks. The classifiers included k-nearest neighbors, logistic regression, random forest, gradient boosting and deep multilayer perceptron. ResultsOur results suggest that in the presence of highly missing data, combining translation and encoding imputation—which considers informative missingness—with tree ensemble classifiers—random forest and gradient boosting—is a sensible choice when aiming to maximize performance, in terms of area under curve. ConclusionsBased on our findings, we recommend the consideration of this imputer-classifier configuration when constructing models in the presence of extremely incomplete numerical data in EHR.

LADI: Landslide displacement interpolation through a spatial-temporal Kalman filter

Monitoring landslide displacement is critical towards understanding kinematics and evaluating risk. Both remote sensing technology and in-situ sensors are currently used for monitoring purposes, each with their own spatio-temporal advantages. However, the majority of current workflows analyze these data independently, missing a synergistic opportunity to combine complementary datasets to enhance analyses both spatially and temporally. This work presents the first landslide specific displacement interpolation technique using a novel approach, LADI (Landslide Displacement Interpolation), which creates a high-spatial, high-temporal resolution interpolation of landslide surface displacement by combining remotely-sensed observation data with and in-situ sensor data. At a minimum, LADI requires only a single in-situ monitoring station (control point) and a single high-resolution displacement dataset (calibration data) to function. LADI implements a series of Kalman filters, where interpolation is first performed spatially, then optionally in the temporal domain. LADI is demonstrated on both simulated and real-world landslide data to showcase the behavior, effectiveness, and applicability of the method. Results are compared to common spatial interpolation methods, which demonstrate that LADI achieves superior performance (average EPE = 0.018 m and 0.031 m of LADI and the best performing common spatial interpolation method respectively when number of interpolation control points = 4) in the real-world test sparse displacement dataset when the number of spatial control points is less than sixteen. Additionally, the performance gain of utilizing LADI is further enhanced over common interpolation methods when the interpolation control points are not equally distributed geographically (average EPE = 0.016 m and 0.049 m respectively) as is usually the case with in-situ landslide displacement monitoring stations.

HGCLMDA: Predicting mRNA-Drug Sensitivity Associations via Hypergraph Contrastive Learning.

The identification of drug sensitivity to mRNA interactions is crucial for drug development and disease treatment, but traditional experimental methods for verifying mRNA-drug sensitivity associations are labor-intensive and time-consuming. In this study, we present a hypergraph contrastive learning approach, HGCLMDA, to predict potential mRNA-drug sensitivity associations. HGCLMDA integrates a graph convolutional network-based method with a hypergraph convolutional network to mine high-order relationships between mRNA-drug association pairs. The proposed cross-view contrastive learning architecture improves the model's learning ability, and the inner product is used to obtain the mRNA-drug sensitivity association score. Our experiments on three mRNA-drug sensitivity association data sets show that HGCLMDA outperforms traditional graph convolutional network-based methods, graph augmentation-based contrastive learning methods, and state-of-the-art association prediction methods. The visualization experiment demonstrates the strong discrimination ability of the mRNA and drug embeddings learned by HGCLMDA, and experiments on sparse data sets showcase the performance and robustness of the method. In-depth analysis of hypergraph structures reveals a crucial role that hypergraphs play in enhancing the performance of models. The case study highlights the potential of HGCLMDA as a valuable tool for predicting mRNA-drug sensitivity associations. The interpretive analysis reveals that HGCLMDA effectively models the similarity between mRNA-mRNA and drug-drug interactions.

Local-Aware Hierarchical Attention for Sequential Recommendation

Modeling the dynamic preferences of users is a challenging and essential task in a recommendation system. Taking inspiration from the successful use of self-attention mechanisms in tasks within natural language processing, several approaches have initially explored integrating self-attention into sequential recommendation, demonstrating promising results. However, existing methods have overlooked the intrinsic structure of sequences, failed to simultaneously consider the local fluctuation and global stability of users’ interests, and lacked user information. To address these limitations, we propose LHASRec (Local-Aware Hierarchical Attention for Sequential Recommendation), a model that divides a user’s historical interaction sequences into multiple sessions based on a certain time interval and computes the weight values for each session. Subsequently, the calculated weight values are combined with the user’s historical interaction sequences to obtain a weighted user interaction sequence. This approach can effectively reflect the local fluctuation of the user’s interest, capture the user’s particular preference, and at the same time, consider the user’s general preference to achieve global stability. Additionally, we employ Stochastic Shared Embeddings (SSE) as a regularization technique to mitigate the overfitting issue resulting from the incorporation of user information. We conduct extensive experiments, showing that our method outperforms other competitive baselines on sparse and dense datasets and different evaluation metrics.

Machine Learning Solutions to Regional Surface Ocean δ18O‐Salinity Relationships for Paleoclimatic Reconstruction

AbstractStable isotope‐based reconstructions of past ocean salinity and hydroclimate depend on accurate, regionally constrained relationships between the stable oxygen isotopic composition of seawater (δ18Osw) and salinity in the surface ocean. An increasing number of δ18Osw observations suggest greater spatial variability in this relationship than previously considered, highlighting the need to reassess these relationships on a global scale. Here, we use available, paired δ18Osw and salinity data (N = 11,119) to create global interpolations of each variable. We then use a self‐organizing map, a specialized form of machine learning, to define 19 regions with unique δ18Osw‐salinity relationships in the surface (&lt;50 m) ocean. Inclusion of atmospheric moisture‐related variables and oceanic tracer data in additional self‐organizing map experiments indicates global surface δ18Osw‐salinity spatial patterns are strongly forced by the atmosphere, as the SOM spatial output is highly similar despite no overlapping input data. Our approach is a useful update to the previously delimited regions, and highlights the utility of neural network pattern extraction in spatiotemporally sparse data sets.

Artificial intelligence can dynamically adjust strategies for auxiliary diagnosing respiratory diseases and analyzing potential pathological relationships

Respiratory diseases are one of the leading causes of human death and exacerbate the global burden of non-communicable diseases. Finding a method to assist clinicians pre-diagnose these diseases is an urgent task. Existing artificial intelligence-based methods can improve the clinical diagnosis efficiency, but still face challenges. For example, the lack of interpretability, the problem of information redundancy or missing caused by only using static data, the difficulty of model to learn the interdependence between features, and the performance of model is limited by sparse datasets, etc. To alleviate these problems, we propose a novel RQPA-Net. It consists of Q&A diagnosis module (QAD) and pathological inference module (PI). The QAD is responsible for interacting with patients, adjusting inquiry strategies dynamically and collecting effective information for disease diagnosis. The designed multi-subspace network can alleviate the problem that classical method is difficult to understand the interdependence between features. The deep reinforcement learning designed also can alleviate the problem of classical methods lack of interpretability. The PI is responsible for reasoning potential pathological relationships between diseases or symptoms based on existing knowledge. Through integrating the advantages of deep learning and reinforcement learning techniques, PI can handle sparse datasets. Finally, for auxiliary diagnosis, the model achieves 0.9780 ± 0.0002 Recall, 0.9778 ± 0.0003 Acc, 0.9779 ± 0.0003 Precision and 0.9780 ± 0.0003 F1-score on the test set. In terms of assisting pathological analysis, compared with the end-to-end model, our model achieves higher comprehensive performance on different tasks and datasets with different degrees of sparsity. Even in sparse datasets, it can effectively infer potential associations between diseases or symptoms, and has higher potential clinical application. In this paper, we propose a novel network structure, which can not only assist doctors in diagnosing diseases, but also contribute to explore the potential disease mechanisms. It provides a new perspective for integrating AI technology and clinical practice.

Customer churn prediction model in cloud environment using DFE-WUNB: ANN deep feature extraction with Weight Updated Tuned Naïve Bayes classification with Block-Jacobi SVD dimensionality reduction

With progressing competitive market, different organizations were desperate to hold this churn rate as minimum value, hence to achieve this, building an effective (CCP) customer churn prediction model is essential. In order to address those issues in CCP, the study deliberated the churn prediction model using DFE-WUNB (Deep Feature Extraction with Weight Updated Tuned Naïve Bayes classifier) in a cloud-computing environment. Due to the huge non-linear features of the Telco customer churn dataset, the pre-processed features are deeply learned by the subsequent two models of ANN. In ANN, the input feature gets multiplied by the weight value, and the resultant output feature passes to the next dense layer in ANN. The deep feature extraction in ANN models facilitates precise accuracy in determining relevant churn features. However, the higher matrix dimensions of features exhibit complexity in prediction, hence this Block Jacobi SVD algorithm is applied to decrease the dimensions of features, such as to create a sparse dataset projected to get fit to the training model for efficient classification. The dimension-reduced features pass through enhanced weighing Naïve Bayes algorithm gets updated with ANN weights and tunes the parameters having greater and best weights to NB classifier, improvising the classification accuracy performance. The comparative assessment of the proposed DFE-WUNB churn prediction model delineated the efficiency of highly accurate churn prediction, outperforming other conventional churn prediction models.

Evaluation of classification models in limited data scenarios with application to additive manufacturing

This paper presents a novel framework that enables the generation of unbiased estimates for test loss using fewer labeled samples, effectively evaluating the predictive performance of classification models in data-limited applications. The framework’s key innovation lies in developing an adaptive sampling distribution that iteratively identifies influential testing samples based on interactions between learner and evaluator agents. Notably, the adaptive distribution dynamically adjusts the evaluator agent’s supervisory role by prioritizing inputs with discrepancies between the agents and considering the evaluator’s uncertainty. Comprehensive experimental analyses on synthetic data and two sparse data sets from material extrusion additive manufacturing problems validate the framework’s superiority over uniform and fixed sampling distributions. First, the proposed framework provides unbiased estimates of the test loss across various data sets, sampling ratios, and evaluator models. Second, the introduced adaptive sampling distribution significantly reduces the standard deviation of the test loss estimator compared to uniform sampling, achieving a 50% reduction for a 10% sampling ratio in the filament selection benchmark. Third, the framework demonstrates its efficacy in model selection to determine the optimal number of hidden units with a reduced number of test samples. Overall, this work offers a promising framework for evaluating classification models in applications where acquiring labeled data is time-consuming and resource-intensive, including materials science and engineering.

Multi-aspect features of items for time-ordered sequential recommendation

The key to sequential recommendation modeling is to capture dynamic users’ interests. Existing sequential recommendation methods (e.g., self-attention mechanism) have achieved extraordinary success in modeling users’ interests. However, these models ignore that users have different levels of preferences for different aspects of items, failing to capture users’ most concerning aspects. In addition, they are highly dependent on the quality of training data, which may lead to overfitting of the model when the training data is insufficient. To address the above issues, we propose a novel sequence-aware model (Multi-Aspect Features of Items for Time-Ordered Sequential Recommendation, MFITSRec), which combines the features of items with user behavior sequences to learn more complex item-item and item-attribute relationships. Moreover, the model uses a self-attention network based on an absolute time relationship, which can better represent the changes in users’ interests and capture users’ preferences for particular aspects of items. Extensive experiments on five datasets demonstrate that our model outperforms various baseline models. In particular, the model’s prediction accuracy has been significantly improved on sparse datasets.