Training Deep Neural Networks Research Articles

Synthetic ground motions in heterogeneous geologies from various sources: the HEMEWS-3D database

Abstract. The ever-improving performances of physics-based simulations and the rapid developments of deep learning are offering new perspectives to study earthquake-induced ground motion. Due to the large amount of data required to train deep neural networks, applications have so far been limited to recorded data or two-dimensional (2D) simulations. To bridge the gap between deep learning and high-fidelity numerical simulations, this work introduces a new database of physics-based earthquake simulations. The HEterogeneous Materials and Elastic Waves with Source variability in 3D (HEMEWS-3D) database comprises 30 000 simulations of elastic wave propagation in 3D geological domains. Each domain is parametrized by a different geological model built from a random arrangement of layers augmented by random fields that represent heterogeneities. Elastic waves originate from a randomly located pointwise source parametrized by a random moment tensor. For each simulation, ground motion is synthesized at the surface by a grid of virtual sensors. The high frequency of waveforms (fmax⁡=5 Hz) allows for extensive analyses of surface ground motion. Existing and foreseen applications range from statistical analyses of the ground motion variability and machine learning methods on geological models to deep-learning-based predictions of ground motion that depend on 3D heterogeneous geologies and source properties. Data are available at https://doi.org/10.57745/LAI6YU (Lehmann, 2023).

Recognizing Digital Ink Chinese Characters Written by International Students Using a Residual Network with 1-Dimensional Dilated Convolution

Due to the complex nature of Chinese characters, junior international students often encounter writing problems related to strokes, components, and their combinations when writing Chinese characters. Digital ink Chinese characters (DICCs) are obtained by sampling the writing trajectory of Chinese characters with a pen input device. DICCs contain rich information, such as the time and space of strokes and sampling points. Recognizing DICCs is crucial for evaluating and correcting writing errors and enhancing the quality of Chinese character teaching for international students. Here, the paper first employs a one-dimensional dilated convolution to digital ink Chinese character recognition (DICCR) and proposes a novel residual network with one-dimensional dilated convolution (1-D ResNetDC). The 1-D ResNetDC not only utilizes multi-scale convolution kernels, but also employs different dilation rates on a single-scale convolution kernel to obtain information from various ranges. Additionally, residual connections facilitate the training of deep one-dimensional convolutional neural networks. Moreover, the paper proposes a more expressive ten-dimensional feature representation that includes spatial, temporal, and writing direction information for each sampling point, thereby improving classification accuracy. Because the DICC dataset of international students is small and unbalanced, the 1-D ResNetDC is pre-trained on the published available dataset. The experiments demonstrate that our approach is effective and superior. This model features a compact architecture, a reduced number of parameters, and excellent scalability.

Improved region proposal network for enhanced few-shot object detection

Despite significant success of deep learning in object detection tasks, the standard training of deep neural networks requires access to a substantial quantity of annotated images across all classes. Data annotation is an arduous and time-consuming endeavor, particularly when dealing with infrequent objects. Few-shot object detection (FSOD) methods have emerged as a solution to the limitations of classic object detection approaches based on deep learning. FSOD methods demonstrate remarkable performance by achieving robust object detection using a significantly smaller amount of training data. A challenge for FSOD is that instances from novel classes that do not belong to the fixed set of training classes appear in the background and the base model may pick them up as potential objects. These objects behave similarly to label noise because they are classified as one of the training dataset classes, leading to FSOD performance degradation. We develop a semi-supervised algorithm to detect and then utilize these unlabeled novel objects as positive samples during the FSOD training stage to improve FSOD performance. Specifically, we develop a hierarchical ternary classification region proposal network (HTRPN) to localize the potential unlabeled novel objects and assign them new objectness labels to distinguish these objects from the base training dataset classes. Our improved hierarchical sampling strategy for the region proposal network (RPN) also boosts the perception ability of the object detection model for large objects. We test our approach and COCO and PASCAL VOC baselines that are commonly used in FSOD literature. Our experimental results indicate that our method is effective and outperforms the existing state-of-the-art (SOTA) FSOD methods. Our implementation is provided as a supplement to support reproducibility of the results https://github.com/zshanggu/HTRPN.11Early partial results of this work is presented in the 2023 ICCV Workshop on Visual Continual Learning (Shangguan and Rostami, 2023).

Feature balanced re-enhanced network with multi-factor margin loss for long-tailed visual recognition

Real-world data often exhibits a long-tailed distribution, where the number of training samples for head classes far exceeds that of tail classes. This class imbalance phenomenon poses significant challenges for training deep neural networks. Existing class-aware loss methods typically focus only on the numerical relationship between class samples, blindly favoring the optimization of tail classes during the process, while neglecting the difficulty of samples and the similarity between the current class and other classes. To this end, relying only on the number relationship can easily lead to over-fitting of tail classes, thereby failing to fully utilize the potential information in the data. Therefore, we propose the Multi-Factor Margin Loss (MFMLoss), which consists of positive margin loss and negative margin loss. MFMLoss comprehensively considers three factors at three levels: overall, class, and sample: (1) quantitative relationships, (2) inter-class similarity relationships, and (3) sample recognition difficulty. The combined consideration of these three factors enables the model to pay more attention to confusing classes and difficult samples during the training process, rather than solely on tail classes, thus achieving optimization from coarse-grained to fine-grained. To further mitigate the negative impact of the imbalance between head and tail classes on feature learning, we design a new network architecture, called F-BREN. F-BREN consists of two components: the feature balancing network and the feature re-enhancement network. The former is trained with negative margin loss, which reduces the recognizability of easy samples. The latter is trained with positive margin loss, using positive margin to give more attention to hard samples, thus balancing the model’s attention to all samples. We conducted extensive experiments on four long-tailed benchmark datasets: CIFAR10-LT, CIFAR100-LT, ImageNet-LT and iNaturalist 2018, comparing the recognition accuracy of our method with eight state-of-the-art methods. The experimental results demonstrate that our proposed method outperforms the eight compared methods.

One-Shot Learning from Prototype Stock Keeping Unit Images

This paper highlights the importance of one-shot learning from prototype Stock Keeping Unit (SKU) images for efficient product recognition in retail and inventory management. Traditional methods require large supervised datasets to train deep neural networks, which can be costly and impractical. One-shot learning techniques mitigate this issue by enabling classification from a single prototype image per product class, thus reducing data annotation efforts. We introduce the Variational Prototyping Encoder (VPE), a novel deep neural network for one-shot classification. Utilizing a support set of prototype SKU images, VPE learns to classify query images by capturing image similarity and prototypical concepts. Unlike metric learning-based approaches, VPE pre-learns image translation from real-world object images to prototype images as a meta-task, facilitating efficient one-shot classification with minimal supervision. Our research demonstrates that VPE effectively reduces the need for extensive datasets by utilizing a single image per class while accurately classifying query images into their respective categories, thus providing a practical solution for product classification tasks.

Hybrid two-level protection system for preserving pre-trained DNN models ownership

AbstractRecent advancements in deep neural networks (DNNs) have made them indispensable for numerous commercial applications. These include healthcare systems and self-driving cars. Training DNN models typically demands substantial time, vast datasets and high computational costs. However, these valuable models face significant risks. Attackers can steal and sell pre-trained DNN models for profit. Unauthorised sharing of these models poses a serious threat. Once sold, they can be easily copied and redistributed. Therefore, a well-built pre-trained DNN model is a valuable asset that requires protection. This paper introduces a robust hybrid two-level protection system for safeguarding the ownership of pre-trained DNN models. The first-level employs zero-bit watermarking. The second-level incorporates an adversarial attack as a watermark by using a perturbation technique to embed the watermark. The robustness of the proposed system is evaluated against seven types of attacks. These are Fast Gradient Method Attack, Auto Projected Gradient Descent Attack, Auto Conjugate Gradient Attack, Basic Iterative Method Attack, Momentum Iterative Method Attack, Square Attack and Auto Attack. The proposed two-level protection system withstands all seven attack types. It maintains accuracy and surpasses current state-of-the-art methods.

CMOS-Compatible Single-Gated Memtransistor Having 3-D Bulk Channel Embedded with Quantum Well Memory Nodes for Fast Training of Deep Neural Network

Recently, three-terminal synaptic devices having gate, source, and drain electrodes have been actively researched to compensate for the poor reliability of two-terminal synaptic devices such as resistive random access memory (ReRAM) and phase change random access memory (PCRAM)1. Two-terminal devices inherently suffer from read disturbance problems, where the stored resistance state changes gradually during the read process of the artificial neural network2. In addition, two-terminal devices essentially require selection devices for performing the write (program or erase) and read process, which is limited to commercialization because matching of the electrical characteristics between the selection device and the synaptic device was extremely difficult3. Moreover, the training speed of artificial neural networks would degrade due to the constant on/off ratio of conductance of two-terminal synaptic devices.To solve this problem, the memtransistors that can change the conductance range of the channel with gate bias have been reported. However, most of the reported memtransistors employed 2-D materials such as MoS2, WSe2, and SnS2 as a channel layer, which is a limitation for uniform large-area thin film growth4–6. In other words, the memtransistors with a 2-D channel layer do not have CMOS compatibility.In this study, for the first time, we designed a CMOS-compatible single-gated memtransistor having an indium-gallium-zinc-oxide (IGZO) channel embedded with quantum well memory nodes via Au nanoparticles. The quantum well memory nodes were obtained by precisely designing the diameter of Au nanoparticles in the IGZO 3-D bulk channel and SiO2 gate oxide interfaces. In addition, the quantum well memory nodes using Au nanoparticles were investigated by an energy band diagram obtained from ultraviolet photoelectron spectroscopy (UPS) analysis of Au, IGZO, and Al source/drain electrodes. Thereby, the gate-tunable pinched hysteresis loops in the output curve of the designed memtransistor were achieved ranging from -30 to 30 V, which present channel conductance tuning by controlling the amplitude of gate bias. In addition, the gate-tunable synaptic plasticities (i.e., long-term potentiation (LTP) and long-term depression (LTD)) were obtained by changing the amplitude of gate bias from 10.0 to 13.5 V at an incremental of 0.5 V, exhibiting exponential growth of maximum channel conductance depending on the amplitude of gate bias. Finally, we demonstrated the fast training of the hardware-based deep neural network simulation using the gate-tunable synaptic plasticity of the designed memtransistor. The mechanism of single-gated memtransistor having quantum well memory nodes via Au nanoparticles and its application will be presented in detail. Acknowledgement This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No. RS-2023-00260527) and Institute of Information & communications Technology Planning & Evaluation (IITP) under the artificial intelligence semiconductor support program to nurture the best talents (IITP-(2023)-RS-2023-00253914) grant funded by the Korea government(MSIT)" References Fuller, E. J. et al. Parallel programming of an ionic floating-gate memory array for scalable neuromorphic computing. Science (80-. ). 364, 570–574 (2019).Yoon, J., Chang, M. & Khwa, W. Compute-in-Memory RRAM Macro With Write Verification and Multi-Bit Encoding. Jssc 57, 1–13 (2022).Kim, H. J. et al. Super-Linear-Threshold-Switching Selector with Multiple Jar-Shaped Cu-Filaments in the Amorphous Ge3Se7 Resistive Switching Layer in a Cross-Point Synaptic Memristor Array. Adv. Mater. 2203643, (2022).Lee, H. S. et al. Dual-Gated MoS2 Memtransistor Crossbar Array. Adv. Funct. Mater. 30, 1–12 (2020).Ding, G. et al. Reconfigurable 2D WSe2-Based Memtransistor for Mimicking Homosynaptic and Heterosynaptic Plasticity. Small 17, 1–13 (2021).Rehman, S., Khan, M. F., Kim, H. D. & Kim, S. Analog–digital hybrid computing with SnS2 memtransistor for low-powered sensor fusion. Nat. Commun. 13, 1–8 (2022). Figure 1

Assessing CNN Architectures for Estimating Correct Posture in Cruise Machinists

Cruise machinists operate in dynamic and physically demanding environments where improper posture can lead to musculoskeletal disorders, adversely affecting their health and work efficiency. Current ergonomic assessments in such settings are often generic and not tailored to the unique challenges of maritime operations. This paper presents a novel application of artificial intelligence tools for real-time posture estimation specifically designed for cruise machinists. The primary aim is to enhance occupational health and safety by providing precise, real-time feedback on ergonomic practices. We developed a dataset by capturing video recordings of cruise machinists at work, which were processed to extract skeletal outlines using advanced computer vision techniques. This dataset was used to train deep neural networks, optimizing them for accuracy in diverse and constrained shipboard environments. The networks were tested across various computational platforms to ensure robustness and adaptability. The AI model demonstrated high efficacy in recognizing both correct and incorrect postures under real-world conditions aboard ships. The system significantly outperformed traditional ergonomic assessment tools in terms of speed, accuracy, and the ability to provide instant feedback. The findings suggest that AI-enhanced ergonomic assessments could be a transformative approach for occupational health across various industries.

Randomness Regularization With Simple Consistency Training for Neural Networks.

Randomness is widely introduced in neural network training to simplify model optimization or avoid the over-fitting problem. Among them, dropout and its variations in different aspects (e.g., data, model structure) are prevalent in regularizing the training of deep neural networks. Though effective and performing well, the randomness introduced by these dropout-based methods causes nonnegligible inconsistency between training and inference. In this paper, we introduce a simple consistency training strategy to regularize such randomness, namely R-Drop, which forces two output distributions sampled by each type of randomness to be consistent. Specifically, R-Drop minimizes the bidirectional KL-divergence between two output distributions produced by dropout-based randomness for each training sample. Theoretical analysis reveals that R-Drop can reduce the above inconsistency by reducing the inconsistency among the sampled sub structures and bridging the gap between the loss calculated by the full model and sub structures. Experiments on 7 widely-used deep learning tasks ( 23 datasets in total) demonstrate that R-Drop is universally effective for different types of neural networks (i.e., feed-forward, recurrent, and graph neural networks) and different learning paradigms (supervised, parameter-efficient, and semi-supervised). In particular, it achieves state-of-the-art performances with the vanilla Transformer model on WMT14 English → German translation ( 30.91 BLEU) and WMT14 English → French translation ( 43.95 BLEU), even surpassing models trained with extra large-scale data and expert-designed advanced variants of Transformer models.

Vision-Language Models for Vision Tasks: A Survey.

Most visual recognition studies rely heavily on crowd-labelled data in deep neural networks (DNNs) training, and they usually train a DNN for each single visual recognition task, leading to a laborious and time-consuming visual recognition paradigm. To address the two challenges, Vision-Language Models (VLMs) have been intensively investigated recently, which learns rich vision-language correlation from web-scale image-text pairs that are almost infinitely available on the Internet and enables zero-shot predictions on various visual recognition tasks with a single VLM. This paper provides a systematic review of visual language models for various visual recognition tasks, including: (1) the background that introduces the development of visual recognition paradigms; (2) the foundations of VLM that summarize the widely-adopted network architectures, pre-training objectives, and downstream tasks; (3) the widely-adopted datasets in VLM pre-training and evaluations; (4) the review and categorization of existing VLM pre-training methods, VLM transfer learning methods, and VLM knowledge distillation methods; (5) the benchmarking, analysis and discussion of the reviewed methods; (6) several research challenges and potential research directions that could be pursued in the future VLM studies for visual recognition.

CATIL: Customized Adversarial Training based on Instance Loss

Adversarial training is one of the most effective adversarial defense methods currently recognized. It enhances the robustness of deep neural network (DNN) classifiers by generating adversarial samples. However, current adversarial training methods cannot effectively trade off the robust accuracy and natural accuracy when training DNN classifiers, and are prone to overfit. To solve these problems, we propose Customized Adversarial Training based on Instance Loss (CATIL), aiming to improve robust accuracy and natural accuracy while alleviating overfitting. We first comprehensively consider the influencing factors of adversarial training and propose the comprehensive customization strategy (CCS), which crafts unique attack strategies for each sample, fine-tunes the classifier's decision boundary, and boosts the robustness of the DNN classifier without compromising its natural accuracy. Second, the loss adjustment strategy (LAS) is designed to update the attack strategy according to the loss value. This increases the fitting difficulty of the DNN classifier and alleviates the overfitting problem. Finally, numerous experiments show that CATIL can effectively enhance robust accuracy and alleviate the overfitting problem without significantly compromising natural accuracy. When evaluating CIFAR-10 on Wide ResNet, CATIL achieves the best performance in both natural and robust accuracy compared to all benchmarks.

Cross-Species Prediction of Transcription Factor Binding by Adversarial Training of a Novel Nucleotide-Level Deep Neural Network.

Cross-species prediction of TF binding remains a major challenge due to the rapid evolutionary turnover of individual TF binding sites, resulting in cross-species predictive performance being consistently worse than within-species performance. In this study, a novel Nucleotide-Level Deep Neural Network (NLDNN) is first proposed to predict TF binding within or across species. NLDNN regards the task of TF binding prediction as a nucleotide-level regression task, which takes DNA sequences as input and directly predicts experimental coverage values. Beyond predictive performance, it also assesses model performance by locating potential TF binding regions, discriminating TF-specific single-nucleotide polymorphisms (SNPs), and identifying causal disease-associated SNPs. The experimental results show that NLDNN outperforms the competing methods in these tasks. Then, a dual-path framework is designed for adversarial training of NLDNN to further improve the cross-species prediction performance by pulling the domain space of human and mouse species closer. Through comparison and analysis, it finds that adversarial training not only can improve the cross-species prediction performance between humans and mice but also enhance the ability to locate TF binding regions and discriminate TF-specific SNPs. By visualizing the predictions, it is figured out that the framework corrects some mispredictions by amplifying the coverage values of incorrectly predicted peaks.

Analysis and comparison of two-level KFAC methods for training deep neural networks

As a second-order method, the Natural Gradient Descent (NGD) has the ability to accelerate training of neural networks. However, due to the prohibitive computational and memory costs of computing and inverting the Fisher Information Matrix (FIM), efficient approximations are necessary to make NGD scalable to Deep Neural Networks (DNNs). Many such approximations have been attempted. The most sophisticated of these is KFAC, which approximates the FIM as a block-diagonal matrix, where each block corresponds to a layer of the neural network. By doing so, KFAC ignores the interactions between different layers. In this work, we investigate the interest of restoring some low-frequency interactions between the layers by means of two-level methods. Inspired from domain decomposition, several two-level corrections to KFAC using different coarse spaces are proposed and assessed. The obtained results show that incorporating the layer interactions in this fashion does not really improve the performance of KFAC. This suggests that it is safe to discard the off-diagonal blocks of the FIM, since the block-diagonal approach is sufficiently robust, accurate and economical in computation time.

Uncertainty-aware deep learning for monitoring and fault diagnosis from synthetic data

Deep neural networks (DNNs) are often coupled with physics-based and data-driven models to perform fault detection and health monitoring. The system models serve as digital surrogates that generate large quantities of data for training DNNs which would otherwise be difficult to obtain from the real-life system. In such a scenario, the uncertainty in the system model and in the DNN parameters will influence the predictions of the DNN. Here, we quantify the impact of this uncertainty on the performance of DNNs. The uncertainty from the system model is captured with two methods, namely assumed density filtering and heteroskedastic modelling. In addition to quantification, these methods allow training DNNs in an uncertainty-aware manner. The uncertainty in the DNN parameters is captured with Monte Carlo dropout. The proposed approach is demonstrated for fault diagnosis of electric power lines. Data generated from a physics-based model calibrated with real-life measurements is used to train three neural network architectures for fault diagnosis. The results reveal that uncertainty-aware models can provide 1% to 19% improvement in classification accuracy than their deterministic counterparts. The uncertainty-aware models also exhibit better robustness to uncertainty and, thus, offer more reliable models for deployment. Remarkably, the article provides a system-agnostic framework for uncertainty-aware training of DNN models for fault diagnosis and monitoring that explicitly accounts for the synthetic nature of training data.

Identifying and training deep learning neural networks on biomedical-related datasets.

This manuscript describes the development of a resources module that is part of a learning platform named 'NIGMS Sandbox for Cloud-based Learning' https://github.com/NIGMS/NIGMS-Sandbox. The overall genesis of the Sandbox is described in the editorial NIGMS Sandbox at the beginning of this Supplement. This module delivers learning materials on implementing deep learning algorithms for biomedical image data in an interactive format that uses appropriate cloud resources for data access and analyses. Biomedical-related datasets are widely used in both research and clinical settings, but the ability for professionally trained clinicians and researchers to interpret datasets becomes difficult as the size and breadth of these datasets increases. Artificial intelligence, and specifically deep learning neural networks, have recently become an important tool in novel biomedical research. However, use is limited due to their computational requirements and confusion regarding different neural network architectures. The goal of this learning module is to introduce types of deep learning neural networks and cover practices that are commonly used in biomedical research. This module is subdivided into four submodules that cover classification, augmentation, segmentation and regression. Each complementary submodule was written on the Google Cloud Platform and contains detailed code and explanations, as well as quizzes and challenges to facilitate user training. Overall, the goal of this learning module is to enable users to identify and integrate the correct type of neural network with their data while highlighting the ease-of-use of cloud computing for implementing neural networks. This manuscript describes the development of a resource module that is part of a learning platform named ``NIGMS Sandbox for Cloud-based Learning'' https://github.com/NIGMS/NIGMS-Sandbox. The overall genesis of the Sandbox is described in the editorial NIGMS Sandbox [1] at the beginning of this Supplement. This module delivers learning materials on the analysis of bulk and single-cell ATAC-seq data in an interactive format that uses appropriate cloud resources for data access and analyses.

Simulation-based inference on virtual brain models of disorders

Connectome-based models, also known as virtual brain models (VBMs), have been well established in network neuroscience to investigate pathophysiological causes underlying a large range of brain diseases. The integration of an individual’s brain imaging data in VBMs has improved patient-specific predictivity, although Bayesian estimation of spatially distributed parameters remains challenging even with state-of-the-art Monte Carlo sampling. VBMs imply latent nonlinear state space models driven by noise and network input, necessitating advanced probabilistic machine learning techniques for widely applicable Bayesian estimation. Here we present simulation-based inference on VBMs (SBI-VBMs), and demonstrate that training deep neural networks on both spatio-temporal and functional features allows for accurate estimation of generative parameters in brain disorders. The systematic use of brain stimulation provides an effective remedy for the non-identifiability issue in estimating the degradation limited to smaller subset of connections. By prioritizing model structure over data, we show that the hierarchical structure in SBI-VBMs renders the inference more effective, precise and biologically plausible. This approach could broadly advance precision medicine by enabling fast and reliable prediction of patient-specific brain disorders.

Artificial Neural Network-aided Mathematical Model for Predicting Soil Stress-strain Hysteresis Loop Evolution

This study presents a novel approach to forecasting the evolution of hysteresis stress-strain response of different types of soils under repeated loading-unloading cycles. The forecasting is made solely from the knowledge of soil properties and loading parameters. Our approach combines mathematical modeling, regression analysis, and Deep Neural Networks (DNNs) to overcome the limitations of traditional DNN training. As a novelty, we propose a hysteresis loop evolution equation and design a family of DNNs to determine the parameters of this equation. Knowing the nature of the phenomenon, we can impose certain solution types and narrow the range of values, enabling the use of a very simple and efficient DNN model. The experimental data used to develop and test the model was obtained through Torsional Shear (TS) tests on soil samples. The model demonstrated high accuracy, with an average R² value of 0.9788 for testing and 0.9944 for training.

Convergence of gradient descent for learning linear neural networks

We study the convergence properties of gradient descent for training deep linear neural networks, i.e., deep matrix factorizations, by extending a previous analysis for the related gradient flow. We show that under suitable conditions on the stepsizes gradient descent converges to a critical point of the loss function, i.e., the square loss in this article. Furthermore, we demonstrate that for almost all initializations gradient descent converges to a global minimum in the case of two layers. In the case of three or more layers, we show that gradient descent converges to a global minimum on the manifold matrices of some fixed rank, where the rank cannot be determined a priori.

Radiomics of pituitary adenoma using computer vision: a review.

Pituitary adenomas (PA) represent the most common type of sellar neoplasm. Extracting relevant information from radiological images is essential for decision support in addressing various objectives related to PA. Given the critical need for an accurate assessment of the natural progression of PA, computer vision (CV) and artificial intelligence (AI) play a pivotal role in automatically extracting features from radiological images. The field of "Radiomics" involves the extraction of high-dimensional features, often referred to as "Radiomic features," from digital radiological images. This survey offers an analysis of the current state of research in PA radiomics. Our work comprises a systematic review of 34 publications focused on PA radiomics and other automated information mining pertaining to PA through the analysis of radiological data using computer vision methods. We begin with a theoretical exploration essential for understanding the theoretical background of radionmics, encompassing traditional approaches from computer vision and machine learning, as well as the latest methodologies in deep radiomics utilizing deep learning (DL). Thirty-four research works under examination are comprehensively compared and evaluated. The overall results achieved in the analyzed papers are high, e.g., the best accuracy is up to 96% and the best achieved AUC is up to 0.99, which establishes optimism for the successful use of radiomic features. Methods based on deep learning seem to be the most promising for the future. In relation to this perspective DL methods, several challenges are remarkable: It is important to create high-quality and sufficiently extensive datasets necessary for training deep neural networks. Interpretability of deep radiomics is also a big open challenge. It is necessary to develop and verify methods that will explain to us how deep radiomic features reflect various physics-explainable aspects.

A stable and robust fault diagnosis method for bearing using lightweight batch normalization-free residual network

Abstract The conventional deep learning-based bearing fault diagnosis methods tend to utilize denoising modules to improve the fault diagnosis performance in noisy scenes. However, the addition of denoising modules will increase expensive computational costs, leading to a delayed acquisition of fault diagnosis results. This work proposed a lightweight batch normalization (BN)-free residual network without any denoising modules for bearing fault diagnosis which properly rescaled the weights in a standard initialization instead of BN to avoid the exploding gradient problem and vanishing gradient problem at the beginning of training for deep neural networks. Therefore, it prevents the undesirable properties caused by BN. Compared with other methods, the fault diagnosis performance of the proposed method can maintain a high level with different input sizes and batch sizes. Especially in noisy scenes, the testing accuracy of fault diagnosis on different bearing datasets can be improved by 13.54% and 7.74% using fewer parameters and floating point operations on different bearing datasets.