Tabular Dataset Research Articles

The Graph Becomes the Dataset: Compiling Historical and Current RTG Data into One Place

Radioisotope thermoelectric generator (RTG) technology has a very long and incredibly successful history of providing heat and power to some of humankind’s most successful space exploration missions. Members of the community that support and maintain this technology often like to show the entire history of RTG performance on “The Graph.” Meant to impress, The Graph does its job in that regard. Unless you were connected to the right people, however, it was very difficult to get a copy of The Graph or its underlying data. This made it difficult for many to dive into the technical details or evaluate individual RTG performance from historical missions. To make matters worse, the performance for several missions in The Graph were incomplete. In 2020, with a little extra time on their hands thanks to COVID, the authors reached throughout the RTG community to locate and collate the missing RTG data. This information was then published as an open-source tabular Dataset. Users can now build their own version of The Graph or use the data for a more detailed analysis of RTG performance. Updates to the currently operating missions in The Dataset are expected to occur on a roughly annual basis. This Dataset represents the most comprehensive, authoritative, up-to-date repository of all RTG performance available today.

Table Transformers for imputing textual attributes

Missing data in tabular dataset is a common issue as the performance of downstream tasks usually depends on the completeness of the training dataset. Previous missing data imputation methods focus on numeric and categorical columns, but we propose a novel end-to-end approach called Table Transformers for Imputing Textual Attributes (TTITA) based on the transformer to impute unstructured textual columns using other columns in the table. We conduct extensive experiments on three datasets, and our approach shows competitive performance outperforming baseline models such as recurrent neural networks and Llama2. The performance improvement is more significant when the target sequence has a longer length. Additionally, we incorporate multi-task learning to simultaneously impute for heterogeneous columns, boosting the performance for text imputation. We also qualitatively compare with ChatGPT for realistic applications.

Attention-Driven Transfer Learning Model for Improved IoT Intrusion Detection

The proliferation of Internet of Things (IoT) devices has become inevitable in contemporary life, significantly affecting myriad applications. Nevertheless, the pervasive use of heterogeneous IoT gadgets introduces vulnerabilities to malicious cyber-attacks, resulting in data breaches that jeopardize the network’s integrity and resilience. This study proposes an Intrusion Detection System (IDS) for IoT environments that leverages Transfer Learning (TL) and the Convolutional Block Attention Module (CBAM). We extensively evaluate four prominent pre-trained models, each integrated with an independent CBAM at the uppermost layer. Our methodology is validated using the BoT-IoT dataset, which undergoes preprocessing to rectify the imbalanced data distribution, eliminate redundancy, and reduce dimensionality. Subsequently, the tabular dataset is transformed into RGB images to enhance the interpretation of complex patterns. Our evaluation results demonstrate that integrating TL models with the CBAM significantly improves classification accuracy and reduces false-positive rates. Additionally, to further enhance the system performance, we employ an Ensemble Learning (EL) technique to aggregate predictions from the two best-performing models. The final findings prove that our TL-CBAM-EL model achieves superior performance, attaining an accuracy of 99.93% as well as high recall, precision, and F1-score. Henceforth, the proposed IDS is a robust and efficient solution for securing IoT networks.

Proposal of self and semi-supervised learning for imbalanced classification of coronary heart disease tabular data

Triple Mixup is an augmentation policy in the hidden latent space we introduced in the Contrastive Mixup Self-Semi Supervised learning framework, to address the imbalanced data problem, for Cardiovascular Heart Diseases tabular dataset. Medical tabular datasets are known to present challenges as high imbalanced class, limited annotated quality samples due to the domain nature. Recent literature in Self and Semi supervised learning, has shown tremendous progress in learning useful representations, and leveraging unlabeled dataset and labeled dataset to train a learning model. Most existing methods are not feasible for tabular data due to the data augmentation scheme. In addition, the high imbalanced problem can show lower performance on machine learning algorithms. For this work, we propose the triple data augmentation method in hidden space to attack the unbalanced challenge in self-supervised and semi-supervised learning, from the possible applications of Contrastive Mixup, thus we will study the influence of it.

AdaVis: Adaptive and Explainable Visualization Recommendation for Tabular Data.

Automated visualization recommendation facilitates the rapid creation of effective visualizations, which is especially beneficial for users with limited time and limited knowledge of data visualization. There is an increasing trend in leveraging machine learning (ML) techniques to achieve an end-to-end visualization recommendation. However, existing ML-based approaches implicitly assume that there is only one appropriate visualization for a specific dataset, which is often not true for real applications. Also, they often work like a black box, and are difficult for users to understand the reasons for recommending specific visualizations. To fill the research gap, we propose AdaVis, an adaptive and explainable approach to recommend one or multiple appropriate visualizations for a tabular dataset. It leverages a box embedding-based knowledge graph to well model the possible one-to-many mapping relations among different entities (i.e., data features, dataset columns, datasets, and visualization choices). The embeddings of the entities and relations can be learned from dataset-visualization pairs. Also, AdaVis incorporates the attention mechanism into the inference framework. Attention can indicate the relative importance of data features for a dataset and provide fine-grained explainability. Our extensive evaluations through quantitative metric evaluations, case studies, and user interviews demonstrate the effectiveness of AdaVis.

Identify the most appropriate imputation method for handling missing values in clinical structured datasets: a systematic review

Background and objectivesComprehending the research dataset is crucial for obtaining reliable and valid outcomes. Health analysts must have a deep comprehension of the data being analyzed. This comprehension allows them to suggest practical solutions for handling missing data, in a clinical data source. Accurate handling of missing values is critical for producing precise estimates and making informed decisions, especially in crucial areas like clinical research. With data's increasing diversity and complexity, numerous scholars have developed a range of imputation techniques. To address this, we conducted a systematic review to introduce various imputation techniques based on tabular dataset characteristics, including the mechanism, pattern, and ratio of missingness, to identify the most appropriate imputation methods in the healthcare field.Materials and methodsWe searched four information databases namely PubMed, Web of Science, Scopus, and IEEE Xplore, for articles published up to September 20, 2023, that discussed imputation methods for addressing missing values in a clinically structured dataset. Our investigation of selected articles focused on four key aspects: the mechanism, pattern, ratio of missingness, and various imputation strategies. By synthesizing insights from these perspectives, we constructed an evidence map to recommend suitable imputation methods for handling missing values in a tabular dataset.ResultsOut of 2955 articles, 58 were included in the analysis. The findings from the development of the evidence map, based on the structure of the missing values and the types of imputation methods used in the extracted items from these studies, revealed that 45% of the studies employed conventional statistical methods, 31% utilized machine learning and deep learning methods, and 24% applied hybrid imputation techniques for handling missing values.ConclusionConsidering the structure and characteristics of missing values in a clinical dataset is essential for choosing the most appropriate data imputation technique, especially within conventional statistical methods. Accurately estimating missing values to reflect reality enhances the likelihood of obtaining high-quality and reusable data, contributing significantly to precise medical decision-making processes. Performing this review study creates a guideline for choosing the most appropriate imputation methods in data preprocessing stages to perform analytical processes on structured clinical datasets.

Machine Learning-Based Identification of Risk Factors of Keratoconus Progression Using Raw Corneal Tomography Data.

The purpose of this study was to identify early indicators of keratoconus progression in Pentacam data using machine learning (ML) techniques. A retrospective Pentacam tabular data set was created by retrieving 11,760 tomography tests performed in patients with keratoconus. Data for eyes labeled unstable based on their referral for cross-linking were differentiated from data for eyes labeled stable and not referred for follow-up procedures. A boosted decision tree was trained on the final data set using a cross-validation method. The final labeled data set included 1218 tomography tests. Training a ML model on a single test for each eye did not accurately predict disease progression, as indicated by the mean receiver-operating characteristic area under the curve of 0.59 ± 0.1, with precision of 0.27, recall of 0.3, and F1 score of 0.28. Training on serial tests for each eye included 819 tomography scans and yielded good prognostic abilities: a receiver-operating characteristic area under the curve of 0.75 ± 0.07, precision of 0.32, recall of 0.67, and F1 score of 0.43. In addition, 4 of the 55 Pentacam raw data parameters predominantly used the algorithm decision: age, central keratoconus index, Rs B, and D10 mm pachy. This study revealed specific dominant parameters attributing to the classification of stability, which are not routinely assessed in determining progression in common practice. Using ML techniques, keratoconus deterioration was evaluated algorithmically with training on multiple tests, yet was not predicted by a single tomography test. Hence, our study highlights novel factors to the current consideration of cross-linking referral and may serve as a supportive tool for clinicians.

Effective Data Distillation for Tabular Datasets (Student Abstract)

Data distillation is a technique of reducing a large dataset into a smaller dataset. The smaller dataset can then be used to train a model which can perform comparably to a model trained on the full dataset. Past works have examined this approach for image datasets, focusing on neural networks as target models. However, tabular datasets pose new challenges not seen in images. A sample in tabular dataset is a one dimensional vector unlike the two (or three) dimensional pixel grid of images, and Non-NN models such as XGBoost can often outperform neural network (NN) based models. Our contribution in this work is two-fold: 1) We show in our work that data distillation methods from images do not translate directly to tabular data; 2) We propose a new distillation method that consistently outperforms the baseline for multiple different models, including non-NN models such as XGBoost.

BERTground: A Transformer-Based Model of Background Spectra on the ISS-Based NICER Space Telescope

The Neutron star Interior Composition Explorer (NICER) is an International Space Station (ISS)-based Space Telescope developed by NASA and devoted to the study of high-energy X-Ray sources in the universe, including but not limited to neutron stars, pulsars, and black holes in stellar systems and active galactic nuclei (AGN). One prominent problem with NICER observations is the highly variable background spectra, obscuring actual signals of astrophysical sources and negatively affecting scientific analysis of the targets. Therefore, obtaining accurate estimations of the background spectra is crucial to filter the noise and facilitate better scientific discoveries of new astronomical objects. In this paper, we propose the very first Deep Neural Network architecture to model the NICER background spectra variation using information about the spacecraft and telescope associated with each observation. In particular, we develop a BERT-based architecture with tokenizers applied to different groups of features in our tabular dataset. We also introduce an adapted Tabular Deep Residual Network architecture as the predictor following the Transformer modules in our network. We show that our model outperforms the current state-of-the-art background model developed by the NICER team in most evaluation metrics. Finally, we discuss pathways and future work for the deployment of this model on NASA’s next versions of HEASARC Software packages.

Arithmetic Feature Interaction Is Necessary for Deep Tabular Learning

Until recently, the question of the effective inductive bias of deep models on tabular data has remained unanswered. This paper investigates the hypothesis that arithmetic feature interaction is necessary for deep tabular learning. To test this point, we create a synthetic tabular dataset with a mild feature interaction assumption and examine a modified transformer architecture enabling arithmetical feature interactions, referred to as AMFormer. Results show that AMFormer outperforms strong counterparts in fine-grained tabular data modeling, data efficiency in training, and generalization. This is attributed to its parallel additive and multiplicative attention operators and prompt-based optimization, which facilitate the separation of tabular samples in an extended space with arithmetically-engineered features. Our extensive experiments on real-world data also validate the consistent effectiveness, efficiency, and rationale of AMFormer, suggesting it has established a strong inductive bias for deep learning on tabular data. Code is available at https://github.com/aigc-apps/AMFormer.

Empowering prediction of miRNA–mRNA interactions in species with limited training data through transfer learning

MicroRNAs (miRNAs) play a crucial role in mRNA regulation. Identifying functionally important mRNA targets of a specific miRNA is essential for uncovering its biological function and assisting miRNA–based drug development. Datasets of high-throughput direct bona fide miRNA–target interactions (MTIs) exist only for a few model organisms, prompting the need for computational prediction. However, the scarcity of data poses a challenge in training accurate machine learning models for MTI prediction. In this study, we explored the potential of transfer learning technique (with ANN and XGB) to address the limited data challenge by leveraging the similarities in interaction rules between species. Furthermore, we introduced a novel approach called TransferSHAP for estimating the feature importance of transfer learning in tabular dataset tasks. We demonstrated that transfer learning improves MTI prediction accuracy for species with limited datasets and identified the specific interaction features the models employed to transfer information across different species.

Predicting dementia in Parkinson's disease on a small tabular dataset using hybrid LightGBM–TabPFN and SHAP

Predicting dementia in Parkinson's disease on a small tabular dataset using hybrid LightGBM–TabPFN and SHAP

Finding the best trade-off between performance and interpretability in predicting hospital length of stay using structured and unstructured data.

This study aims to develop high-performing Machine Learning and Deep Learning models in predicting hospital length of stay (LOS) while enhancing interpretability. We compare performance and interpretability of models trained only on structured tabular data with models trained only on unstructured clinical text data, and on mixed data. The structured data was used to train fourteen classical Machine Learning models including advanced ensemble trees, neural networks and k-nearest neighbors. The unstructured data was used to fine-tune a pre-trained Bio Clinical BERT Transformer Deep Learning model. The structured and unstructured data were then merged into a tabular dataset after vectorization of the clinical text and a dimensional reduction through Latent Dirichlet Allocation. The study used the free and publicly available Medical Information Mart for Intensive Care (MIMIC) III database, on the open AutoML Library AutoGluon. Performance is evaluated with respect to two types of random classifiers, used as baselines. The best model from structured data demonstrates high performance (ROC AUC = 0.944, PRC AUC = 0.655) with limited interpretability, where the most important predictors of prolonged LOS are the level of blood urea nitrogen and of platelets. The Transformer model displays a good but lower performance (ROC AUC = 0.842, PRC AUC = 0.375) with a richer array of interpretability by providing more specific in-hospital factors including procedures, conditions, and medical history. The best model trained on mixed data satisfies both a high level of performance (ROC AUC = 0.963, PRC AUC = 0.746) and a much larger scope in interpretability including pathologies of the intestine, the colon, and the blood; infectious diseases, respiratory problems, procedures involving sedation and intubation, and vascular surgery. Our results outperform most of the state-of-the-art models in LOS prediction both in terms of performance and of interpretability. Data fusion between structured and unstructured text data may significantly improve performance and interpretability.

A Multi-Agent Intrusion Detection System Optimized by a Deep Reinforcement Learning Approach with a Dataset Enlarged Using a Generative Model to Reduce the Bias Effect

Intrusion detection systems can defectively perform when they are adjusted with datasets that are unbalanced in terms of attack data and non-attack data. Most datasets contain more non-attack data than attack data, and this circumstance can introduce biases in intrusion detection systems, making them vulnerable to cyberattacks. As an approach to remedy this issue, we considered the Conditional Tabular Generative Adversarial Network (CTGAN), with its hyperparameters optimized using the tree-structured Parzen estimator (TPE), to balance an insider threat tabular dataset called the CMU-CERT, which is formed by discrete-value and continuous-value columns. We showed through this method that the mean absolute errors between the probability mass functions (PMFs) of the actual data and the PMFs of the data generated using the CTGAN can be relatively small. Then, from the optimized CTGAN, we generated synthetic insider threat data and combined them with the actual ones to balance the original dataset. We used the resulting dataset for an intrusion detection system implemented with the Adversarial Environment Reinforcement Learning (AE-RL) algorithm in a multi-agent framework formed by an attacker and a defender. We showed that the performance of detecting intrusions using the framework of the CTGAN and the AE-RL is significantly improved with respect to the case where the dataset is not balanced, giving an F1-score of 0.7617.

Tabular Machine Learning Methods for Predicting Gas Turbine Emissions

Predicting emissions for gas turbines is critical for monitoring harmful pollutants being released into the atmosphere. In this study, we evaluate the performance of machine learning models for predicting emissions for gas turbines. We compared an existing predictive emissions model, a first-principles-based Chemical Kinetics model, against two machine learning models we developed based on the Self-Attention and Intersample Attention Transformer (SAINT) and eXtreme Gradient Boosting (XGBoost), with the aim to demonstrate the improved predictive performance of nitrogen oxides (NOx) and carbon monoxide (CO) using machine learning techniques and determine whether XGBoost or a deep learning model performs the best on a specific real-life gas turbine dataset. Our analysis utilises a Siemens Energy gas turbine test bed tabular dataset to train and validate the machine learning models. Additionally, we explore the trade-off between incorporating more features to enhance the model complexity, and the resulting presence of increased missing values in the dataset.

Predicting Depression during the COVID-19 Pandemic Using Interpretable TabNet: A Case Study in South Korea

COVID-19 has further aggravated problems by compelling people to stay indoors and limit social interactions, leading to a worsening of the depression situation. This study aimed to construct a TabNet model combined with SHapley Additive exPlanations (SHAP) to predict depression in South Korean society during the COVID-19 pandemic. We used a tabular dataset extracted from the Seoul Welfare Survey with a total of 3027 samples. The TabNet model was trained on this dataset, and its performance was compared to that of several other machine learning models, including Random Forest, eXtreme Gradient Boosting, Light Gradient Boosting, and CatBoost. According to the results, the TabNet model achieved an Area under the receiver operating characteristic curve value (AUC) of 0.9957 on the training set and an AUC of 0.9937 on the test set. Additionally, the study investigated the TabNet model’s local interpretability using SHapley Additive exPlanations (SHAP) to provide post hoc global and local explanations for the proposed model. By combining the TabNet model with SHAP, our proposed model might offer a valuable tool for professionals in social fields, and psychologists without expert knowledge in the field of data analysis can easily comprehend the decision-making process of this AI model.

DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Data

Data preprocessing is a crucial step in the machine learning process that transforms raw data into a more usable format for downstream ML models. However, it can be costly and time-consuming, often requiring the expertise of domain experts. Existing automated machine learning (AutoML) frameworks claim to automate data preprocessing. However, they often use a restricted search space of data preprocessing pipelines which limits the potential performance gains, and they are often too slow as they require training the ML model multiple times. In this paper, we propose DiffPrep, a method that can automatically and efficiently search for a data preprocessing pipeline for a given tabular dataset and a differentiable ML model such that the performance of the ML model is maximized. We formalize the problem of data preprocessing pipeline search as a bi-level optimization problem. To solve this problem efficiently, we transform and relax the discrete, non-differential search space into a continuous and differentiable one, which allows us to perform the pipeline search using gradient descent with training the ML model only once. Our experiments show that DiffPrep achieves the best test accuracy on 15 out of the 18 real-world datasets evaluated and improves the model's test accuracy by up to 6.6 percentage points.

Maize yield prediction using federated random forest

Crop variety yield prediction is important for the global food supply. In recent years, machine learning models have been successfully applied in this domain. However, most seed companies typically have a limited number of trial sites they can set up and lack sufficient data for training machine learning models individually, which prevents them from benefiting from state-of-the-art AI technologies in the age of intelligent breeding. The primary aim of this study is to propose a novel approach based on a federated random forest algorithm that enables breeding institutions to collaborate without disclosing or sharing their own data, and to jointly model with tabular data including field breeding test phenotypes and environmental meteorology data stored locally by each participating institution. With a focus on a real tabular dataset obtained from maize field trials at 248 trial sites from the China National Crop Variety Tests from 2017 to 2021, this paper presents the first results that delve into phenotypic data and explore federated learning algorithms based on decision trees in the field of crop yield prediction. Empirical verification of the maize crop yield prediction scenario showed that the method not only performs better than each of the models trained on an individual data source, but also is virtually lossless in accuracy compared with traditional, data-centralized random forest approach. Additionally, the method provides a cost-effective and efficient alternative for joint breeding for breeders and breeding teams.

Patient-specific quality assurance failure prediction with deep tabular models

Purpose. Patient-specific quality assurance (PSQA) failures in radiotherapy can cause a delay in patient care and increase the workload and stress of staff. We developed a tabular transformer model based directly on the multi-leaf collimator (MLC) leaf positions (without any feature engineering) to predict IMRT PSQA failure in advance. This neural model provides an end-to-end differentiable map from MLC leaf positions to the probability of PSQA plan failure, which could be useful for regularizing gradient-based leaf sequencing optimization algorithms and generating a plan that is more likely to pass PSQA. Method. We retrospectively collected DICOM RT PLAN files of 968 patient plans treated with volumetric arc therapy. We constructed a beam-level tabular dataset with 1873 beams as samples and MLC leaf positions as features. We trained an attention-based neural network FT-Transformer to predict the ArcCheck-based PSQA gamma pass rates. In addition to the regression task, we evaluated the model in the binary classification context predicting the pass or fail of PSQA. The performance was compared to the results of the two leading tree ensemble methods (CatBoost and XGBoost) and a non-learned method based on mean-MLC-gap. Results. The FT-Transformer model achieves 1.44% Mean Absolute Error (MAE) in the regression task of the gamma pass rate prediction and performs on par with XGBoost (1.53 % MAE) and CatBoost (1.40 % MAE). In the binary classification task of PSQA failure prediction, FT-Transformer achieves 0.85 ROC AUC (compared to the mean-MLC-gap complexity metric achieving 0.72 ROC AUC). Moreover, FT-Transformer, CatBoost, and XGBoost all achieve 80% true positive rate while keeping the false positive rate under 20%. Conclusions. We demonstrated that reliable PSQA failure predictors can be successfully developed based solely on MLC leaf positions. FT-Transformer offers an unprecedented benefit of providing an end-to-end differentiable map from MLC leaf positions to the probability of PSQA failure.

A CNN-Based Novel Approach for Classification of Sacral Hiatus with GAN-Powered Tabular Data Set

Caudal epidural anaesthesia is usually the most well-known technique in obstetrics to deal with chronic back pain. Due to variations in the shape and size of the sacral hiatus (SH), its classification is a crucial and challenging task. Clinically, it is required in trauma, where surgeons must make fast and correct selections. Past studies have focused on morphometric and statistical analysis to classify it. Therefore, it is vital to automatically and accurately classify SH types through deep learning methods. To this end, we proposed the Multi-Task Process (MTP), a novel classification approach to classify the SH MTP that initially uses a small medical tabular data set obtained by manual feature extraction on computed tomography scans of the sacrums. Second, it augments the data set synthetically through a Generative Adversarial Network (GAN). In addition, it adapts a two-dimensional (2D) embedding algorithm to convert tabular features into images. Finally, it feeds images into Convolutional Neural Networks (CNNs). The application of MTP to six CNN models achieved remarkable classification success rates of approximately 90 % to 93 %. The proposed MTP approach eliminates the small medical tabular data problem that results in bone classification on deep models.