Ground Truth Dataset Research Articles

An explainable two-stage machine learning approach for precipitation forecast

A common post-processing approach to improve precipitation forecasts is to use machine learning models such as artificial neural networks (more specifically, multi-layer perceptrons) as black-box systems. These models utilize different sources of observations or predictors to generate an improved forecast in terms of desired metrics. However, most existing studies employ a single-stage regression model without considering explainability. The small number of studies with two-stage models that combine classification and regression utilize binary classification and still lack explainable artificial intelligence. Therefore, this study proposes a precipitation prediction system which (i) is composed of two stages for better predictions, (ii) compares the utility of binary and multi-class classification over the regression, and (iii) is explainable, unlike prior studies, in that individual predictions of machine learning-based forecasts are interpretable by humans. The proposed two-stage model first estimates the precipitation intensity category using binary or multi-class classification as the first stage and later utilizes precipitation intensity category information in a regression model, which is the second stage, to obtain daily precipitation magnitude. The utilized approach is made humanly interpretable (i.e., explainable) by providing insight into the model-wide importance of predictors and generation processes of the individual predictions (instance-level explanation). The proposed two-stage approach is compared against single-stage and black-box approaches in terms of prediction quality and explainability, where daily station-based observations are used as ground truth datasets. Experiments show that the proposed two-stage approach yields significant improvement (on average, RMSE reduced by 10.50%, and the correlation between numerical precipitation estimates and observed precipitation values increased by 7.5%) compared to the best-performing physical predictor (ECMWF). Analysis of explainability provides insights into the decisions of our two-stage approach, e.g., the usefulness of seasonality-related parameters, multi-class precipitation intensity classification as a first stage, and the predictors for each task (regression or classification).

Estimation of Landslide and Mudslide Susceptibility with Multi-Modal Remote Sensing Data and Semantics: The Case of Yunnan Mountain Area

Landslide and mudslide susceptibility predictions play a crucial role in environmental monitoring, ecological protection, settlement planning, etc. Currently, multi-modal remote sensing data have been used for precise landslide and mudslide disaster prediction with spatial details, spectral information, or terrain attributes. However, features regarding landslide and mudslide susceptibility are often hidden in multi-modal remote sensing images, beyond the features extracted and learnt by deep learning approaches. This paper reports our efforts to conduct landslide and mudslide susceptibility prediction with multi-modal remote sensing data involving digital elevation models, optical remote sensing, and an SAR dataset. Moreover, based on the results generated by multi-modal remote sensing data, we further conducted landslide and mudslide susceptibility prediction with semantic knowledge. Through the comparisons with the ground truth datasets created by field investigation, experimental results have proved that remote sensing data can only enhance deep learning techniques to detect the landslide and mudslide, rather than the landslide and mudslide susceptibility. Knowledge regarding the potential clues about landslide and mudslide, which would be critical for estimating landslide and mudslide susceptibility, have not been comprehensively investigated yet.

A High-Generalizability Machine Learning Framework for Analyzing the Homogenized Properties of Short Fiber-Reinforced Polymer Composites.

This study aims to develop a high-generalizability machine learning framework for predicting the homogenized mechanical properties of short fiber-reinforced polymer composites. The ensemble machine learning model (EML) employs a stacking algorithm using three base models of Extra Trees (ET), eXtreme Gradient Boosting machine (XGBoost), and Light Gradient Boosting machine (LGBM). A micromechanical model of a two-step homogenization algorithm is adopted and verified as an effective approach to composite modeling with randomly distributed fibers, which is integrated with finite element simulations for providing a high-quality ground-truth dataset. The model performance is thoroughly assessed for its accuracy, efficiency, interpretability, and generalizability. The results suggest that: (1) the EML model outperforms the base members on prediction accuracy, achieving R2 values of 0.988 and 0.952 on the train and test datasets, respectively; (2) the SHapley Additive exPlanations (SHAP) analysis identifies the Young's modulus of matrix, fiber, and fiber content as the top three factors influencing the homogenized properties, whereas the anisotropy is predominantly determined by the fiber orientations; (3) the EML model showcases good generalization capability on experimental data, and it has been shown to be more effective than high-fidelity computational models by significantly lowering computational costs while maintaining high accuracy.

Compliance Prediction for Structural Topology Optimization on the Basis of Moment Invariants and a Generalized Regression Neural Network.

Topology optimization techniques are essential for manufacturing industries, such as designing fiber-reinforced polymer composites (FRPCs) and structures with outstanding strength-to-weight ratios and light weights. In the SIMP approach, artificial intelligence algorithms are commonly utilized to enhance traditional FEM-based compliance minimization procedures. Based on an effective generalized regression neural network (GRNN), a new deep learning algorithm of compliance prediction for structural topology optimization is proposed. The algorithm learns the structural information using a fourth-order moment invariant analysis of the structural topology obtained from FEA at different iterations of classical topology optimization. A cantilever and a simply supported beam problem are used as ground-truth datasets, and the moment invariants are used as independent variables for input features. By comparing it with the well-known convolutional neural network (CNN) and deep neural network (DNN) models, the proposed GRNN model achieves a high prediction accuracy (R2 > 0.97) and drastically shortens the training and prediction cost. Furthermore, the GRNN algorithm exhibits excellent generalization ability on the prediction performance of the optimized topology with rotations and varied material volume fractions. This algorithm is promising for the replacement of the FEA calculation in the SIMP method, and can be applied to real-time optimization for advanced FRPC structure design.

Simulating structurally variable Nuclear Pore Complexes for Microscopy.

The Nuclear Pore Complex (NPC) is the only passageway for macromolecules between nucleus and cytoplasm, and an important reference standard in microscopy: it is massive and stereotypically arranged. The average architecture of NPC proteins has been resolved with pseudo-atomic precision, however observed NPC heterogeneities evidence a high degree of divergence from this average. Single Molecule Localization Microscopy (SMLM) images NPCs at protein-level resolution, whereupon image analysis software studies NPC variability. However the true picture of this variability is unknown. In quantitative image analysis experiments, it is thus difficult to distinguish intrinsically high SMLM noise from variability of the underlying structure. We introduce CIR4MICS ("ceramics", Configurable, Irregular Rings FOR MICroscopy Simulations), a pipeline that synthesizes ground truth datasets of structurally variable NPCs based on architectural models of the true NPC. Users can select one or more N- or C-terminally tagged NPC proteins, and simulate a wide range of geometric variations. We also represent the NPC as a spring-model such that arbitrary deforming forces, of user-defined magnitudes, simulate irregularly shaped variations. Further, we provide annotated reference datasets of simulated human NPCs, which facilitate a side-by-side comparison with real data. To demonstrate, we synthetically replicate a geometric analysis of real NPC radii and reveal that a range of simulated variability parameters can lead to observed results. Our simulator is therefore valuable to test the capabilities of image analysis methods, as well as to inform experimentalists about the requirements of hypothesis-driven imaging studies. Code: https://github.com/uhlmanngroup/cir4mics. Simulated data: BioStudies S-BSST1058. Supplementary data are available at.

Regional Models for Sentinel-2/MSI Imagery of Chlorophyll a and TSS, Obtained for Oligotrophic Issyk-Kul Lake Using High-Resolution LIF LiDAR Data

The development of regional satellite bio-optical models for natural waters with high temporal and spatial variability, such as inland seas, reservoirs, and coastal ocean waters, requires the implementation of an intermediate measuring link in the chain, “water sampling—bio-optical models”, and this link must have certain intermediate characteristics. The most crucial of them are the high-precision measurements of the main water quality parameters, such as the concentration of chlorophyll a (Chl a), colored dissolved organic matter (CDOM), and total suspended sediments (TSS) in the upper water layer, together with a high operational rate and the ability to cover a large water area in a short time, which corresponds to a satellite overpass. A possible solution is to utilize laser-induced fluorescence (LIF) of water constituents measured by a marine LiDAR in situ with a high sampling rate from a high-speed vessel. This allows obtaining a large ground-truth dataset of the main water quality parameters simultaneously with the satellite overpass within the time interval determined by NASA protocols. This method was successfully applied to the oligotrophic Issyk-Kul Lake in Kyrgyzstan, where we obtained more than 4000 and 1000 matchups for the Chl a and TSS, respectively. New preliminary regional bio-optical models were developed on the basis of a one-day survey and tested for archive Sentinel-2A data for 2022. This approach can be applied for regular monitoring and further correction in accordance with seasonal variability. The obtained results, together with previously published similar studies for eutrophic coastal and productive inland waters, emphasize the applicability of the presented method for the development or adjustment of regional bio-optical models for water bodies of a wide trophic range.

The image and ground truth dataset of Mongolian movable-type newspapers for text recognition

The image and ground truth dataset of Mongolian movable-type newspapers for text recognition

Groundwork for AI: Enforcing a benchmark for neoantigen prediction in personalized cancer immunotherapy.

This article expands on recent studies of machine learning or artificial intelligence (AI) algorithms that crucially depend on benchmark datasets, often called 'ground truths.' These ground-truth datasets gather input-data and output-targets, thereby establishing what can be retrieved computationally and evaluated statistically. I explore the case of the Tumor nEoantigen SeLection Alliance (TESLA), a consortium-based ground-truthing project in personalized cancer immunotherapy, where the 'truth' of the targets-immunogenic neoantigens-to be retrieved by the would-be AI algorithms depended on a broad technoscientific network whose setting up implied important organizational and material infrastructures. The study shows that instead of grounding an undisputable 'truth', the TESLA endeavor ended up establishing a contestable reference, the biology of neoantigens and how to measure their immunogenicity having slightly evolved alongside this four-year project. However, even if this controversy played down the scope of the TESLA ground truth, it did not discredit the whole undertaking. The magnitude of the technoscientific efforts that the TESLA project set into motion and the needs it ultimately succeeded in filling for the scientific and industrial community counterbalanced its metrological uncertainties, effectively instituting its contestable representation of 'true' neoantigens within the field of personalized cancer immunotherapy (at least temporarily). More generally, this case study indicates that the enforcement of ground truths, and what it leaves out, is a necessary condition to enable AI technologies in personalized medicine.

Deep Learning Models for Serendipity Recommendations: A Survey and New Perspectives

Serendipitous recommendations have emerged as a compelling approach to deliver users with unexpected yet valuable information, contributing to heightened user satisfaction and engagement. This survey presents an investigation of the most recent research in serendipity recommenders, with a specific emphasis on deep learning recommendation models. We categorize these models into three types, distinguishing their integration of the serendipity objective across distinct stages: pre-processing, in-processing, and post-processing. Additionally, we provide a review and summary of the serendipity definition, available ground truth datasets, and evaluation experiments employed in the field. We propose three promising avenues for future exploration: (1) leveraging user reviews to identify and explore serendipity, (2) employing reinforcement learning to construct a model for discerning appropriate timing for serendipitous recommendations, and (3) utilizing cross-domain learning to enhance serendipitous recommendations. With this review, we aim to cultivate a deeper understanding of serendipity in recommender systems and inspire further advancements in this domain.

One-dimensional ice shelf hardness inversion: Clustering behavior and collocation resampling in physics-informed neural networks

We investigate the use of Physics-Informed Neural Networks (PINNs) for ice shelf hardness inversion. PINNs were trained to invert for ice shelf hardness (B) profiles using synthetic datasets of one-dimensional (1D) ice shelf velocity (u) and thickness (h) contaminated by noise, with no training data provided for B. A hyperparameter γ is introduced in the objective function to adjust the relative weight of the neural network fit to training data and its fit to known physical laws. When the data is contaminated by noise, the equation loss should be weighted more heavily than the data loss to de-noise the predictions. However, we observe that PINN predictive performance is highly unreliable when γ is large. The prediction errors are distributed bimodally, with one cluster corresponding to accurate, “low-error” solutions, and the other consisting of “high-error” solutions that fit poorly to the ground truth datasets. The high-error PINN solutions are associated with the failure of the PINN to correctly learn the velocity spatial gradients that appear in the governing equations. We employ an alternative algorithm to improve the PINN inversion accuracy, which we call collocation resampling: the PINN is initially trained for a small number of iterations for which the collocation points for evaluating the equation loss are randomly resampled after each iteration; then, a fixed set of collocation points are used. This approach achieves predictive accuracies comparable to those attained by low-error solutions for the original training scheme with γ1−γ≫1, but with no clustering and minimal computational cost.

Deriving Ground Truth Labels for Regression Problems Using Annotator Precision

When training machine learning models with practical applications, a quality ground truth dataset is critical. Unlike in classification problems, there is currently no effective method for determining a single ground truth value or landmark from a set of annotations in regression problems. We propose a novel method for deriving ground truth labels in regression problems that considers the performance and precision of individual annotators when identifying each label separately. In contrast to the commonly accepted method of computing the global mean, our method does not assume each annotator to be equally capable of completing the specified task, but rather ensures that higher-performing annotators have a greater contribution to the final result. The ground truth selection method described within this paper provides a means of improving the quality of input data for machine learning model development by removing lower-quality labels. In this study, we objectively demonstrate the improved performance by applying the method to a simulated dataset where a canonical ground truth position can be known, as well as to a sample of data collected from crowd-sourced labels.

Multi-Modality Tensor Fusion Based Human Fatigue Detection

Multimodal learning is an expanding research area and aims to pursue a better understanding of given data by regarding different modals. Multimodal approaches for qualitative data are used for the quantitative proofing of ground-truth datasets and discovering unexpected phenomena. In this paper, we investigate the effect of multimodal learning schemes of quantitative data to assess its qualitative state. We try to interpret human fatigue levels through analyzing video, thermal image and voice data together. The experiment showed that the multimodal approach using three types of data was more effective than the method of using each dataset individually. As a result, we identified the possibility of predicting human fatigue states.

SiulMalaya: an annotated bird audio dataset of Malaysia lowland forest birds for passive acoustic monitoring

The laborious point count method of conducting bird surveys is still a common practice in Malaysia. An alternative method known as passive acoustic monitoring (PAM) is deployed in many countries by placing sound recorders at surveying sites to collect bird sounds. Studies revealed that the number of bird densities counted by human observers was agreeable with those obtained using PAM. However, one of the most essential gaps in conducting PAM is the lack of expert-verified bird-call databases. Therefore, the aim of this study is to construct the first annotated Malaysia lowland forest bird sounds called SiulMalaya to be used as ground-truth datasets for PAM-related activities. The raw bird sounds dataset was downloaded from Macaulay Library using the eBird platform. Data pre-processing was done to produce annotated audio tracks that can be used as training datasets for bird classification. SiulMalaya dataset was further validated by two bird experts from the Department of Wildlife and National Parks, Malaysia. A bird identification experiment was carried out to assess and validate SiulMalaya dataset using a convolutional neural network (CNN) learning model. Even though the accuracy of bird identification is slightly above 50%, the annotated dataset is shown to be viable for PAM-related operations.

SiulMalaya: an annotated bird audio dataset of Malaysia lowland forest birds for passive acoustic monitoring

The laborious point count method of conducting bird surveys is still a common practice in Malaysia. An alternative method known as passive acoustic monitoring (PAM) is deployed in many countries by placing sound recorders at surveying sites to collect bird sounds. Studies revealed that the number of bird densities counted by human observers was agreeable with those obtained using PAM. However, one of the most essential gaps in conducting PAM is the lack of expert-verified bird-call databases. Therefore, the aim of this study is to construct the first annotated Malaysia lowland forest bird sounds called SiulMalaya to be used as ground-truth datasets for PAM-related activities. The raw bird sounds dataset was downloaded from Macaulay Library using the eBird platform. Data pre-processing was done to produce annotated audio tracks that can be used as training datasets for bird classification. SiulMalaya dataset was further validated by two bird experts from the Department of Wildlife and National Parks, Malaysia. A bird identification experiment was carried out to assess and validate SiulMalaya dataset using a convolutional neural network (CNN) learning model. Even though the accuracy of bird identification is slightly above 50%, the annotated dataset is shown to be viable for PAM-related operations.

Reliability of whole mount radical prostatectomy histopathology as the ground truth for artificial intelligence assisted prostate imaging

The development of artificial intelligence–based imaging techniques for prostate cancer (PCa) detection and diagnosis requires a reliable ground truth, which is generally based on histopathology from radical prostatectomy specimens. This study proposes a comprehensive protocol for the annotation of prostatectomy pathology slides. To evaluate the reliability of the protocol, interobserver variability was assessed between five pathologists, who annotated ten radical prostatectomy specimens consisting of 74 whole mount pathology slides. Interobserver variability was assessed for both the localization and grading of PCa. The results indicate excellent overall agreement on the localization of PCa (Gleason pattern ≥ 3) and clinically significant PCa (Gleason pattern ≥ 4), with Dice similarity coefficients (DSC) of 0.91 and 0.88, respectively. On a per-slide level, agreement for primary and secondary Gleason pattern was almost perfect and substantial, with Fleiss Kappa of .819 (95% CI .659–.980) and .726 (95% CI .573–.878), respectively. Agreement on International Society of Urological Pathology Grade Group was evaluated for the index lesions and showed agreement in 70% of cases, with a mean DSC of 0.92 for all index lesions. These findings show that a standardized protocol for prostatectomy pathology annotation provides reliable data on PCa localization and grading, with relatively high levels of interobserver agreement. More complicated tissue characterization, such as the presence of cribriform growth and intraductal carcinoma, remains a source of interobserver variability and should be treated with care when used in ground truth datasets.

A Semantically Annotated 15-Class Ground Truth Dataset for Substation Equipment to Train Semantic Segmentation Models

The lack of annotated semantic segmentation datasets for electrical substations in the literature poses a significant problem for machine learning tasks; before training a model, a dataset is needed. This paper presents a new dataset of electric substations with 1660 images annotated with 15 classes, including insulators, disconnect switches, transformers and other equipment commonly found in substation environments. The images were captured using a combination of human, fixed and AGV-mounted cameras at different times of the day, providing a diverse set of training and testing data for algorithm development. In total, 50,705 annotations were created by a team of experienced annotators, using a standardized process to ensure accuracy across the dataset. The resulting dataset provides a valuable resource for researchers and practitioners working in the fields of substation automation, substation monitoring and computer vision. Its availability has the potential to advance the state of the art in this important area.

The Rise of GoodFATR: A Novel Accuracy Comparison Methodology for Indicator Extraction Tools

To adapt to a constantly evolving landscape of cyber threats, organizations actively need to collect Indicators of Compromise (IOCs), i.e., forensic artifacts that signal that a host or network might have been compromised. IOCs can be collected through open-source and commercial structured IOC feeds. But, they can also be extracted from a myriad of unstructured threat reports written in natural language and distributed using a wide array of sources such as blogs and social media. There exist multiple indicator extraction tools that can identify IOCs in natural language reports. But, it is hard to compare their accuracy due to the difficulty of building large ground truth datasets. This work presents a novel majority vote methodology for comparing the accuracy of indicator extraction tools, which does not require a manually-built ground truth. We implement our methodology into GoodFATR, an automated platform for collecting threat reports from a wealth of sources, extracting IOCs from the collected reports using multiple tools, and comparing their accuracy. GoodFATR supports 6 threat report sources: RSS, Twitter, Telegram, Malpedia, APTnotes, and ChainSmith. GoodFATR continuously monitors the sources, downloads new threat reports, extracts 41 indicator types from the collected reports, and filters non-malicious indicators to output the IOCs. We run GoodFATR over 15 months to collect 472,891 reports from the 6 sources; extract 978,151 indicators from the reports; and identify 618,217 IOCs. We analyze the collected data to identify the top IOC contributors and the IOC class distribution. We apply GoodFATR to compare the IOC extraction accuracy of 7 popular open-source tools with GoodFATR's own indicator extraction module.

A standardized quantitative analysis strategy for stable isotope probing metagenomics.

Stable isotope probing (SIP) facilitates culture-independent identification of active microbial populations within complex ecosystems through isotopic enrichment of nucleic acids. Many DNA-SIP studies rely on 16S rRNA gene sequences to identify active taxa, but connecting these sequences to specific bacterial genomes is often challenging. Here, we describe a standardized laboratory and analysis framework to quantify isotopic enrichment on a per-genome basis using shotgun metagenomics instead of 16S rRNA gene sequencing. To develop this framework, we explored various sample processing and analysis approaches using a designed microbiome where the identity of labeled genomes and their level of isotopic enrichment were experimentally controlled. With this ground truth dataset, we empirically assessed the accuracy of different analytical models for identifying active taxa and examined how sequencing depth impacts the detection of isotopically labeled genomes. We also demonstrate that using synthetic DNA internal standards to measure absolute genome abundances in SIP density fractions improves estimates of isotopic enrichment. In addition, our study illustrates the utility of internal standards to reveal anomalies in sample handling that could negatively impact SIP metagenomic analyses if left undetected. Finally, we present SIPmg, an R package to facilitate the estimation of absolute abundances and perform statistical analyses for identifying labeled genomes within SIP metagenomic data. This experimentally validated analysis framework strengthens the foundation of DNA-SIP metagenomics as a tool for accurately measuring the in situ activity of environmental microbial populations and assessing their genomic potential. IMPORTANCE Answering the questions, "who is eating what?" and "who is active?" within complex microbial communities is paramount for our ability to model, predict, and modulate microbiomes for improved human and planetary health. These questions can be pursued using stable isotope probing to track the incorporation of labeled compounds into cellular DNA during microbial growth. However, with traditional stable isotope methods, it is challenging to establish links between an active microorganism's taxonomic identity and genome composition while providing quantitative estimates of the microorganism's isotope incorporation rate. Here, we report an experimental and analytical workflow that lays the foundation for improved detection of metabolically active microorganisms and better quantitative estimates of genome-resolved isotope incorporation, which can be used to further refine ecosystem-scale models for carbon and nutrient fluxes within microbiomes.

Mapping smallholder cashew plantations to inform sustainable tree crop expansion in Benin

Cashews are grown by over 3 million smallholder farmers in >40 countries worldwide as a principal source of income. Expanding the area of cashew plantations and increasing productivity are critical to improving the livelihood of many smallholder communities. As the third largest cashew producer in Africa, Benin has nearly 200,000 smallholder cashew growers contributing 15% of the country's national export earnings. Expansion of the cashew industry is thus an essential economic driver and a governmental priority in Benin. However, a lack of information on where and how cashew trees grow across the country hinders decision-making that could support increased cashew production and poverty alleviation. By leveraging 2.4-m Planet Basemaps and 0.5-m aerial imagery, two newly developed deep learning algorithms, and large-scale ground truth datasets, we successfully produced the first-of-its-kind national map of cashew in Benin and characterized the expansion of cashew plantations between 2015 and 2021. In particular, we developed a SpatioTemporal Classification with Attention (STCA) model to map the distribution of cashew plantations with 2.4-m multi-temporal Planet Basemaps from 2019 to 2021, which can fully capture texture information from discriminative time steps during a growing season. The U-Net model was employed to map the distribution of cashew plantation with 0.5-m mono-temporal aerial imagery in 2015, which can achieve accurate and fast predictions even with limited training data. We further developed a Clustering Augmented Self-supervised Temporal Classification (CASTC) model to distinguish high-density versus low-density cashew plantations by automatic feature extraction and optimized clustering. Results show that the STCA model has an overall accuracy over 85% based on 1400 ground truth point samples from each year. The CASTC model achieved an overall accuracy of 76% based on 348 ground truth samples of planting density. We found that the cashew area in Benin has almost doubled to 519 ± 20 kha from 2015 to 2021 with 60% of new plantation development coming from cropland or fallow land, while encroachment of cashew plantations into protected areas has increased by 55%. Only about half of cashew plantations were high-density in 2021, suggesting high potential for intensification. Our study illustrates the power of combining high-resolution remote sensing imagery and state-of-the-art deep learning algorithms to better understand tree crops in the heterogeneous smallholder landscape, which can help efficiently allocate limited training and nursery resources for sustainable agricultural development.

RelaxNet: A structure-preserving neural network to approximate the Boltzmann collision operator

The extremely high dimensionality and nonlinearity in the Boltzmann equation bring tremendous difficulties to the study of rarefied gas dynamics. This paper addresses a neural network-based surrogate model that provides a structure-preserving approximation for the fivefold collision integral. The notion originates from the similarity in structure between the BGK-type relaxation model and residual neural network (ResNet) when a particle distribution function is treated as the input to the neural network function. Therefore, we extend the ResNet architecture and construct what we call the relaxation neural network (RelaxNet). Specifically, two feed-forward neural networks with physics-informed connections and activations are introduced as building blocks in RelaxNet, which provide bounded and physically realizable approximations of the equilibrium distribution and velocity-dependent relaxation time respectively. The evaluation of the collision term is thus significantly accelerated due to the fact that the convolution in the fivefold integral is replaced by tensor multiplication in the neural network. We fuse the mechanical advection operator and the RelaxNet-based collision operator into a unified model named the universal Boltzmann equation (UBE). We prove that UBE preserves the key structural properties in a many-particle system, i.e., positivity, conservation, invariance, H-theorem, and correct fluid dynamic limit. These properties promise that RelaxNet is superior to strategies that naively approximate the right-hand side of the Boltzmann equation using a machine learning model. A novel sampling strategy based on closure hierarchies of the moment system of the Boltzmann equation is developed to generate reliable and unbiased sampling for the supervised learning. The construction of the RelaxNet-based UBE and its solution algorithm are demonstrated in detail. Several numerical experiments, where the ground-truth datasets are produced by the Shakhov model, velocity-dependent ν-BGK model, and the full Boltzmann equation, are investigated. The capability of the current approach for simulating non-equilibrium flow physics is validated through satisfactory in- and out-of-distribution performance. The open-source codes to reproduce the numerical results are available under the MIT license [1].