Recent Success Of Deep Learning Research Articles

Implementation of deep neural networks learning on unmanned aerial vehicle based remote-sensing

Due to efficient and adaptable data collecting, unmanned aerial vehicle (UAV) has been a popular topic in computer vision (CV) and remote sensing (RS) in recent years. Inspiring by the recent success of deep learning (DL), several enhanced object identification and tracking methods have been broadly applied to a variety of UAV-related applications, including environmental monitoring, precision agriculture, and traffic management. In this research, we present efficient neural network (ENet), a unique deep neural network architecture designed exclusively for jobs demanding low latency operation. ENet is up to quicker, takes fewer floating-point operations per second (FLOPs), has fewer parameters, and offers accuracy comparable to or superior to that of previous models. We have tested it on the street and cityscapes reports on comparisons with current state-of-the-art approaches and the tradeoffs between a network's processing speed and accuracy. We give measurements of the proposed architecture's performance on embedded devices and offer software enhancements that might make ENet even quicker.

Dataset Distillation: A Comprehensive Review.

Recent success of deep learning is largely attributed to the sheer amount of data used for training deep neural networks. Despite the unprecedented success, the massive data, unfortunately, significantly increases the burden on storage and transmission and further gives rise to a cumbersome model training process. Besides, relying on the raw data for training per se yields concerns about privacy and copyright. To alleviate these shortcomings, dataset distillation (DD), also known as dataset condensation (DC), was introduced and has recently attracted much research attention in the community. Given an original dataset, DD aims to derive a much smaller dataset containing synthetic samples, based on which the trained models yield performance comparable with those trained on the original dataset. In this paper, we give a comprehensive review and summary of recent advances in DD and its application. We first introduce the task formally and propose an overall algorithmic framework followed by all existing DD methods. Next, we provide a systematic taxonomy of current methodologies in this area, and discuss their theoretical interconnections. We also present current challenges in DD through extensive empirical studies and envision possible directions for future works.

PointWavelet: Learning in Spectral Domain for 3-D Point Cloud Analysis.

With recent success of deep learning in 2-D visual recognition, deep-learning-based 3-D point cloud analysis has received increasing attention from the community, especially due to the rapid development of autonomous driving technologies. However, most existing methods directly learn point features in the spatial domain, leaving the local structures in the spectral domain poorly investigated. In this article, we introduce a new method, PointWavelet, to explore local graphs in the spectral domain via a learnable graph wavelet transform. Specifically, we first introduce the graph wavelet transform to form multiscale spectral graph convolution to learn effective local structural representations. To avoid the time-consuming spectral decomposition, we then devise a learnable graph wavelet transform, which significantly accelerates the overall training process. Extensive experiments on four popular point cloud datasets, ModelNet40, ScanObjectNN, ShapeNet-Part, and S3DIS, demonstrate the effectiveness of the proposed method on point cloud classification and segmentation.

Evolutionary multi-objective design of autoencoders for compact representation of histopathology whole slide images

The recent success of deep learning coincides with the surge of digital pathology as an effect of the pandemic. However, processing a massive volume of gigapixel histopathology images is daunting. Hence, establishing compact image representations for efficient computational pathology becomes paramount. Deep autoencoders have been frequently used to compress feature vectors extracted from pre-trained networks. However, the topology of autoencoders is generally set heuristically. In this paper, we propose a multi-objective evolutionary framework as a bi-level optimization scheme in which the outer level evolves autoencoders for the goal of dimension reduction while the inner level is to optimize their weights for training. Throughout the evolution process, multiple minimization objectives such as complexity, classification error, and code size are considered. These objectives can obtain compressed models to generate a very small set of deep features with the highest classification accuracy. We use images from The Cancer Genome Atlas (TCGA) repository provided by the National Cancer Institute to train and validate our optimization framework. We increased the classification accuracy by 8% while the image representation is 46,000 times shorter. We apply the framework to demonstrate higher accuracy, substantial memory reduction and faster processing for image retrieval compared to SOTA.

EnViTSA: Ensemble of Vision Transformer with SpecAugment for Acoustic Event Classification.

Recent successes in deep learning have inspired researchers to apply deep neural networks to Acoustic Event Classification (AEC). While deep learning methods can train effective AEC models, they are susceptible to overfitting due to the models' high complexity. In this paper, we introduce EnViTSA, an innovative approach that tackles key challenges in AEC. EnViTSA combines an ensemble of Vision Transformers with SpecAugment, a novel data augmentation technique, to significantly enhance AEC performance. Raw acoustic signals are transformed into Log Mel-spectrograms using Short-Time Fourier Transform, resulting in a fixed-size spectrogram representation. To address data scarcity and overfitting issues, we employ SpecAugment to generate additional training samples through time masking and frequency masking. The core of EnViTSA resides in its ensemble of pre-trained Vision Transformers, harnessing the unique strengths of the Vision Transformer architecture. This ensemble approach not only reduces inductive biases but also effectively mitigates overfitting. In this study, we evaluate the EnViTSA method on three benchmark datasets: ESC-10, ESC-50, and UrbanSound8K. The experimental results underscore the efficacy of our approach, achieving impressive accuracy scores of 93.50%, 85.85%, and 83.20% on ESC-10, ESC-50, and UrbanSound8K, respectively. EnViTSA represents a substantial advancement in AEC, demonstrating the potential of Vision Transformers and SpecAugment in the acoustic domain.

Deep Learning for Approximate Nearest Neighbour Search: A Survey and Future Directions

Approximate nearest neighbour search (ANNS) in high-dimensional space is an essential and fundamental operation in many applications from many domains such as multimedia database, information retrieval and computer vision. With the rapidly growing volume of data and the dramatically increasing demands of users, traditional heuristic-based ANNS solutions have been facing great challenges in terms of both efficiency and accuracy. Inspired by the recent successes of deep learning in many fields, substantial efforts have been devoted to applying deep learning techniques to ANNS for learning to index and learning to search, resulting in numerous algorithms that achieve state-of-the-art performance compared with conventional methods. In this survey paper, we comprehensively review the different types of deep learning-based ANNS methods according to two learning paradigms: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">learning to index</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">learning to search</i> . We provide a comprehensive overview and analysis of these methods in a systematic manner. Based on the overview, we point out that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">end-to-end learning</i> will be a new and promising research direction for deep learning-based ANNS, i.e., applying deep learning techniques to jointly learn the indexing and searching together, such that the underlying knowledge learned from data can directly contribute to the final searching performance. Finally, we conduct experiments and provide general performance analyses for the representative deep learning-based ANNS algorithms.

Classification of Alzheimer's Progression Using fMRI Data.

In the last three decades, the development of functional magnetic resonance imaging (fMRI) has significantly contributed to the understanding of the brain, functional brain mapping, and resting-state brain networks. Given the recent successes of deep learning in various fields, we propose a 3D-CNN-LSTM classification model to diagnose health conditions with the following classes: condition normal (CN), early mild cognitive impairment (EMCI), late mild cognitive impairment (LMCI), and Alzheimer's disease (AD). The proposed method employs spatial and temporal feature extractors, wherein the former utilizes a U-Net architecture to extract spatial features, and the latter utilizes long short-term memory (LSTM) to extract temporal features. Prior to feature extraction, we performed four-step pre-processing to remove noise from the fMRI data. In the comparative experiments, we trained each of the three models by adjusting the time dimension. The network exhibited an average accuracy of 96.4% when using five-fold cross-validation. These results show that the proposed method has high potential for identifying the progression of Alzheimer's by analyzing 4D fMRI data.

Automatic depression severity assessment with deep learning using parameter-efficient tuning.

To assist mental health care providers with the assessment of depression, research to develop a standardized, accessible, and non-invasive technique has garnered considerable attention. Our study focuses on the application of deep learning models for automatic assessment of depression severity based on clinical interview transcriptions. Despite the recent success of deep learning, the lack of large-scale high-quality datasets is a major performance bottleneck for many mental health applications. A novel approach is proposed to address the data scarcity problem for depression assessment. It leverages both pretrained large language models and parameter-efficient tuning techniques. The approach is built upon adapting a small set of tunable parameters, known as prefix vectors, to guide a pretrained model towards predicting the Patient Health Questionnaire (PHQ)-8 score of a person. Experiments were conducted on the Distress Analysis Interview Corpus - Wizard of Oz (DAIC-WOZ) benchmark dataset with 189 subjects, partitioned into training, development, and test sets. Model learning was done on the training set. Prediction performance mean and standard deviation of each model, with five randomly-initialized runs, were reported on the development set. Finally, optimized models were evaluated on the test set. The proposed model with prefix vectors outperformed all previously published methods, including models which utilized multiple types of data modalities, and achieved the best reported performance on the test set of DAIC-WOZ with a root mean square error of 4.67 and a mean absolute error of 3.80 on the PHQ-8 scale. Compared to conventionally fine-tuned baseline models, prefix-enhanced models were less prone to overfitting by using far fewer training parameters (<6% relatively). While transfer learning through pretrained large language models can provide a good starting point for downstream learning, prefix vectors can further adapt the pretrained models effectively to the depression assessment task by only adjusting a small number of parameters. The improvement is in part due to the fine-grain flexibility of prefix vector size in adjusting the model's learning capacity. Our results provide evidence that prefix-tuning can be a useful approach in developing tools for automatic depression assessment.

On potential challenges of V2X sidelink relaying under interference: link-level and system-level simulation with neural network assisted

The ever-increasing demand for high data rates and high connection densities in the vehicle communication network, along with the widespread adoption of radio access over the Third Generation Partnership Project (3GPP) standard, has been a major driver for the research on cellular vehicle-to-everything (C-V2X) communication. Nevertheless, Wi-Fi and other wireless communication technology work on the 5.9 GHz unlicensed band has also undergone booming proliferation over the years. C-V2X users dedicated band on the 5.9 GHz spectrum may thus suffer from both co-channel and adjacent channel interference, which cannot be negligible, especially in urban scenarios. To this end, 3GPP has standardized relay technology in New Radio (NR) V2X sidelink to extend the transmission range under interference. In this paper, through a link-level and system-level simulation study, we evaluate the sidelink performance in relaying scenarios under different interference. Motivated by the recent success of deep learning, a novel neural network is further introduced as a unified benchmark for interference mitigation evaluation. Numerical results show that there exist challenges in the real-time optimization of transmission scheme selection and power allocation in relay-assisted cases. The simulation also reveals that the interference incurred by NR on unlicensed spectrum (NR-U) signals and other sidelink signals is intractable to be suppressed, which may bring potential challenges in future works.

Protein structure prediction using the evolutionary algorithm USPEX.

Protein structure prediction is one of major problems of modern biophysics: current attempts to predict the tertiary protein structure from amino acid sequence are successful mostly when the use of big data and machine learning allows one to reduce the "prediction problem" to the "problem of recognition". Compared with recent successes of deep learning, classical predictive methods lag behind in their accuracy for the prediction of stable conformations. Therefore, in this work we extended the evolutionary algorithm USPEX to predict protein structure based on global optimization starting with the amino acid sequence. Moreover, we compared frequently used force fields for the task of protein structure prediction. Protein structure relaxation and energy calculations were performed using Tinker (with several different force fields) and Rosetta (with REF2015 force field) codes. To create new protein structure models in the USPEX algorithm, we developed novel variation operators. The test of the new method on seven proteins having (for simplicity) no cis-proline (with ω ≈ 0°) residues, and a length of up to 100 residues, revealed that our algorithm predicts tertiary structures of proteins with high accuracy. The comparison of the final potential energies of the predicted protein structures obtained using the USPEX and the Rosetta Abinitio approach showed that in most cases the developed algorithm found structures with close or even lower energy (Amber/Charmm/Oplsaal) and scoring function (REF2015). While USPEX has clearly demonstrated its ability to find very deep energy minima, our study showed that the existing force fields are not sufficiently accurate for accurate blind prediction of protein structures without further experimental verification.

A Revisiting Study of Appropriate Offline Evaluation for Top- N Recommendation Algorithms

In recommender systems, top- N recommendation is an important task with implicit feedback data. Although the recent success of deep learning largely pushes forward the research on top- N recommendation, there are increasing concerns on appropriate evaluation of recommendation algorithms. It therefore is important to study how recommendation algorithms can be reliably evaluated and thoroughly verified. This work presents a large-scale, systematic study on six important factors from three aspects for evaluating recommender systems. We carefully select 12 top- N recommendation algorithms and eight recommendation datasets. Our experiments are carefully designed and extensively conducted with these algorithms and datasets. In particular, all the experiments in our work are implemented based on an open sourced recommendation library, Recbole [ 139 ], which ensures the reproducibility and reliability of our results. Based on the large-scale experiments and detailed analysis, we derive several key findings on the experimental settings for evaluating recommender systems. Our findings show that some settings can lead to substantial or significant differences in performance ranking of the compared algorithms. In response to recent evaluation concerns, we also provide several suggested settings that are specially important for performance comparison.

Deep Reinforcement Learning With Graph Representation for Vehicle Repositioning

On-demand ride services, e.g., taxi companies, ridesourcing/transportation network companies (TNCs), play a vital role in nowadays transportation modes. The relocation/repositioning of idle vehicles is an important operational problem for service providers. With the recent success of deep learning, a large and growing body of literature emerged on learning-based transportation problems, including the relocation problem. In this work, we study the idle vehicle relocation problem to achieve better drivers’ profits and customers’ satisfaction. In detail, we formulate the learning problem on real-world road networks while recent repositioning works under a deep reinforcement learning scheme formulate the environment as grids. Meanwhile, a graph formulation enables learning with Graph Neural Network (GNN), which shows promising results in recent transportation studies. We empirically show that preserving the road network structure and learning the data representation with GNN contribute to improving the repositioning decision policy. Our simulation experiments are conducted on two real-world datasets, in New York City and Austin, respectively. The results show our formulation’s advantages in terms of reductions of order rejection rates and passenger waiting time, and increasing of vehicle occupancy rates and driver revenues.

Differential Private Deep Learning Models for Analyzing Breast Cancer Omics Data.

Proper analysis of high-dimensional human genomic data is necessary to increase human knowledge about fundamental biological questions such as disease associations and drug sensitivity. However, such data contain sensitive private information about individuals and can be used to identify an individual (i.e., privacy violation) uniquely. Therefore, raw genomic datasets cannot be publicly published or shared with researchers. The recent success of deep learning (DL) in diverse problems proved its suitability for analyzing the high volume of high-dimensional genomic data. Still, DL-based models leak information about the training samples. To overcome this challenge, we can incorporate differential privacy mechanisms into the DL analysis framework as differential privacy can protect individuals’ privacy. We proposed a differential privacy based DL framework to solve two biological problems: breast cancer status (BCS) and cancer type (CT) classification, and drug sensitivity prediction. To predict BCS and CT using genomic data, we built a differential private (DP) deep autoencoder (dpAE) using private gene expression datasets that performs low-dimensional data representation learning. We used dpAE features to build multiple DP binary classifiers to predict BCS and CT in any individual. To predict drug sensitivity, we used the Genomics of Drug Sensitivity in Cancer (GDSC) dataset. We extracted GDSC’s dpAE features to build our DP drug sensitivity prediction model for 265 drugs. Evaluation of our proposed DP framework shows that it achieves improved prediction performance in predicting BCS, CT, and drug sensitivity than the previously published DP work.

Abstract B014: Massively parallel functional assessment of label-free mutant pools is a universal approach to parametrize mechanistic models of drug resistance evolution

Abstract Systematically scanning the effects of amino acid substitutions across a protein is a powerful technique in protein engineering, synthetic biology, and the study of drug resistance. In general, these selection experiments increase growth rates to select for mutants. Current methods use molecular barcodes to circumvent the resolution barriers imposed by conventional NGS sequencing of point mutations in mutant pools. Barcodes introduce synonymous mutations in the genome and assume neutrality, but it is now well accepted that synonymous mutations affect protein function. In a different approach, barcodes on cDNA constructs require a cumbersome second cloning step during library generation. We describe an approach that harnesses a highly sensitive duplex sequencing strategy to skip this barcode integration step and directly measure the target in label-free deep mutant pools. Furthermore, we employ the use of single-mutant standards with well-studied growth dynamics. By tracking the deviations from the expected growth trajectory of these mutant standards, we correct for dose-response variations that often result in low replicate agreement in these functional screens. Our workflow is compatible with any pool generation strategy, and the parameters that we obtain are robust enough to be useful in detailed models of cellular dynamics in synthetic biology studies. We developed a method for the functional assessment of deep mutant pools in a single pooled experiment. The workflow includes generating a label-free library of mutants at a desired target and sequencing the mutagenized pool before and during drug treatment using duplex sequencing. We demonstrate our pooled approach by focusing on target-driven resistance in Chronic Myeloid Leukemia (CML). To this end, we studied the 17 most clinically abundant BCRABL resistant mutations in imatinib refractory CML at low frequencies of 1 in 15,000. The sensitivity afforded by duplex sequencing enabled the accurate quantification of growth rates of these mutants under imatinib selection, with all 17 mutants closely matching their expected growth trajectory as predicted by individual dose-response studies. Moreover, we use an inference-based approach that uses hill-curve information from our mutant standards to detect slight variations in attempted dose between pooled replicates. Correcting for these slight variations in attempted dose leads to a &amp;gt;10-fold improvement in replicate agreement. Here, we demonstrate a fast, label-free workflow that achieves high levels of sensitivity and specificity in measuring growth dynamics of deep mutant pools. Importantly, our method does not rely on synonymous, potentially error-inducing labels. Given recent successes in deep learning that predict structure and function in proteins, our approach can be used to generate large training sets of mutation-drug phenotypes. Predicting the rich phenotypic data generated using existing sequence and structural data could dramatically reduce the experiments that are necessary to generate predictive information on drug resistance. Citation Format: Haider Inam, Scott Leighow, Justin Pritchard. Massively parallel functional assessment of label-free mutant pools is a universal approach to parametrize mechanistic models of drug resistance evolution [abstract]. In: Proceedings of the AACR Special Conference on the Evolutionary Dynamics in Carcinogenesis and Response to Therapy; 2022 Mar 14-17. Philadelphia (PA): AACR; Cancer Res 2022;82(10 Suppl):Abstract nr B014.

Unsupervised Learning in Drug Design from Self-Organization to Deep Chemistry.

The availability of computers has brought novel prospects in drug design. Neural networks (NN) were an early tool that cheminformatics tested for converting data into drugs. However, the initial interest faded for almost two decades. The recent success of Deep Learning (DL) has inspired a renaissance of neural networks for their potential application in deep chemistry. DL targets direct data analysis without any human intervention. Although back-propagation NN is the main algorithm in the DL that is currently being used, unsupervised learning can be even more efficient. We review self-organizing maps (SOM) in mapping molecular representations from the 1990s to the current deep chemistry. We discovered the enormous efficiency of SOM not only for features that could be expected by humans, but also for those that are not trivial to human chemists. We reviewed the DL projects in the current literature, especially unsupervised architectures. DL appears to be efficient in pattern recognition (Deep Face) or chess (Deep Blue). However, an efficient deep chemistry is still a matter for the future. This is because the availability of measured property data in chemistry is still limited.

Deep Learning for Unmanned Aerial Vehicle-Based Object Detection and Tracking: A survey

Owing to effective and flexible data acquisition, unmanned aerial vehicle (UAV) has recently become a hotspot across the fields of computer vision (CV) and remote sensing (RS). Inspired by recent success of deep learning (DL), many advanced object detection and tracking approaches have been widely applied to various UAV-related tasks, such as environmental monitoring, precision agriculture, traffic management. This paper provides a comprehensive survey on the research progress and prospects of DL-based UAV object detection and tracking methods. More specifically, we first outline the challenges, statistics of existing methods, and provide solutions from the perspectives of DL-based models in three research topics: object detection from the image, object detection from the video, and object tracking from the video. Open datasets related to UAV-dominated object detection and tracking are exhausted, and four benchmark datasets are employed for performance evaluation using some state-of-the-art methods. Finally, prospects and considerations for the future work are discussed and summarized. It is expected that this survey can facilitate those researchers who come from remote sensing field with an overview of DL-based UAV object detection and tracking methods, along with some thoughts on their further developments.

The applications of deep learning algorithms on in silico druggable proteins identification

IntroductionThe top priority in drug development is to identify novel and effective drug targets. In vitro assays are frequently used for this purpose; however, traditional experimental approaches are insufficient for large-scale exploration of novel drug targets, as they are expensive, time-consuming and laborious. Therefore, computational methods have emerged in recent decades as an alternative to aid experimental drug discovery studies by developing sophisticated predictive models to estimate unknown drugs/compounds and their targets. The recent success of deep learning (DL) techniques in machine learning and artificial intelligence has further attracted a great deal of attention in the biomedicine field, including computational drug discovery. ObjectivesThis study focuses on the practical applications of deep learning algorithms for predicting druggable proteins and proposes a powerful predictor for fast and accurate identification of potential drug targets. MethodsUsing a gold-standard dataset, we explored several typical protein features and different deep learning algorithms and evaluated their performance in a comprehensive way. We provide an overview of the entire experimental process, including protein features and descriptors, neural network architectures, libraries and toolkits for deep learning modelling, performance evaluation metrics, model interpretation and visualization. ResultsExperimental results show that the hybrid model (architecture: CNN-RNN (BiLSTM) + DNN; feature: dictionary encoding + DC_TC_CTD) performed better than the other models on the benchmark dataset. This hybrid model was able to achieve 90.0% accuracy and 0.800 MCC on the test dataset and 84.8% and 0.703 on a nonredundant independent test dataset, which is comparable to those of existing methods. ConclusionWe developed the first deep learning-based classifier for fast and accurate identification of potential druggable proteins. We hope that this study will be helpful for future researchers who would like to use deep learning techniques to develop relevant predictive models.

Semi-supervised multiscale dual-encoding method for faulty traffic data detection

&lt;abstract&gt; &lt;p&gt;Inspired by the recent success of deep learning in multiscale information encoding, we introduce a variational autoencoder (VAE) based semi-supervised method for detection of faulty traffic data, which is cast as a classification problem. Continuous wavelet transform (CWT) is applied to the time series of traffic volume data to obtain rich features embodied in time-frequency representation, followed by a twin of VAE models to separately encode normal data and faulty data. The resulting multiscale dual encodings are concatenated and fed to an attention-based classifier, consisting of a self-attention module and a multilayer perceptron. For comparison, the proposed architecture is evaluated against five different encoding schemes, including (1) VAE with only normal data encoding, (2) VAE with only faulty data encoding, (3) VAE with both normal and faulty data encodings, but without attention module in the classifier, (4) siamese encoding, and (5) cross-vision transformer (CViT) encoding. The first four encoding schemes adopt the same convolutional neural network (CNN) architecture while the fifth encoding scheme follows the transformer architecture of CViT. Our experiments show that the proposed architecture with the dual encoding scheme, coupled with attention module, outperforms other encoding schemes and results in classification accuracy of 96.4%, precision of 95.5%, and recall of 97.7%.&lt;/p&gt; &lt;/abstract&gt;

Can DXA image-based deep learning model predict the anisotropic elastic behavior of trabecular bone?

3D image-based finite element (FE) and bone volume fraction (BV/TV)/fabric tensor modeling techniques are currently used to determine the apparent stiffness tensor of trabecular bone for assessing its anisotropic elastic behavior. Inspired by the recent success of deep learning (DL) techniques, we hypothesized that DL modeling techniques could be used to predict the apparent stiffness tensor of trabecular bone directly using dual-energy X-ray absorptiometry (DXA) images. To test the hypothesis, a convolutional neural network (CNN) model was trained and validated to predict the apparent stiffness tensor of trabecular bone cubes using their DXA images. Trabecular bone cubes obtained from human cadaver proximal femurs were used to obtain simulated DXA images as input, and the apparent stiffness tensor of the trabecular cubes determined by using micro-CT based FE simulations was used as output (ground truth) to train the DL model. The prediction accuracy of the DL model was evaluated by comparing it with the micro-CT based FE models, histomorphometric parameter based multiple linear regression models, and BV/TV/fabric tensor based multiple linear regression models. The results showed that DXA image-based DL model achieved high fidelity in predicting the apparent stiffness tensor of trabecular bone cubes (R2 = 0.905–0.973), comparable to or better than the histomorphometric parameter based multiple linear regression and BV/TV/fabric tensor based multiple linear regression models, thus supporting the hypothesis of this study. The outcome of this study could be used to help develop DXA image-based DL techniques for clinical assessment of bone fracture risk.

Using neural networks to accelerate the solution of the Boltzmann equation

One of the biggest challenges for simulating the Boltzmann equation is the evaluation of fivefold collision integral. Given the recent successes of deep learning and the availability of efficient tools, it is an obvious idea to try to substitute the calculation of the collision operator by the evaluation of a neural network. However, it is unlcear whether this preserves key properties of the Boltzmann equation, such as conservation, invariances, the H-theorem, and fluid-dynamic limits.In this paper, we present an approach that guarantees the conservation properties and the correct fluid dynamic limit at leading order. The concept originates from a recently developed scientific machine learning strategy which has been named “universal differential equations”. It proposes a hybridization that fuses the deep physical insights from classical Boltzmann modeling and the desirable computational efficiency from neural network surrogates. The construction of the method and the training strategy are demonstrated in detail. We conduct an asymptotic analysis and illustrate the multi-scale applicability of the method. The numerical algorithm for solving the neural network-enhanced Boltzmann equation is presented as well. Several numerical test cases are investigated. The results of numerical experiments show that the time-series modeling strategy enjoys the training efficiency on this supervised learning task.