Interpretation Of Deep Learning Research Articles

Rotational Convolution Design in Convolutional Neural Networks for Direct 3D Electromagnetic Tomography

The three-dimensional (3D) reconstruction of Electromagnetic Tomography (EMT) is an important task for many applications, such as the non-destructive testing of inner defects in rail systems. Additionally, image reconstruction algorithms utilizing deep learning methods have been verified to be useful in recent years. Therefore, the interpretability of deep learning is a question that is relevant to its application in other areas. This paper proposes an innovative rotational convolution pattern, Conv-P, for convolutional neural network (CNN) image reconstruction in a 3D EMT system. This pattern is based on the projection relationships inherent in tomographic imaging, where each convolution is performed on adjacent projections along the excitation rotation direction. The advantage of this pattern is that it can generate the convolution process by utilizing the 3D structural information from real sensors. To verify the effectiveness of this convolution pattern, we constructed a 3D dual-layer 16-coil EMT model and tested its image reconstruction performance. The results demonstrate that, compared with two common convolution patterns, Conv-P achieves a 4.7% and 4.1% increase in the Image Correlation Coefficient (CC), a 19.8% and 13.1% reduction in the Relative Image Error (IE), a 0.67% and 1.59% increase in the Peak Signal-to-Noise Ratio (PSNR), and a 3.24% and 0.74% increase in the Structural Similarity Index Measure (SSIM).

Diagnosing schizophrenia using deep learning: Novel interpretation approaches and multi-site validation

Schizophrenia is a profound and enduring mental disorder that imposes significant negative impacts on individuals, their families, and society at large. The development of more accurate and objective diagnostic tools for schizophrenia can be expedited through the employment of deep learning (DL), that excels at deciphering complex hierarchical non-linear patterns. However, the limited interpretability of deep learning has eroded confidence in the model and restricted its clinical utility. At the same time, if the data source is only derived from a single center, the model's generalizability is difficult to test. To enhance the model's reliability and applicability, leave-one-center-out validation with a large and diverse sample from multiple centers is crucial. In this study, we utilized Nine different global centers to train and test the 3D Resnet model's generalizability, resulting in an 82% classification performance (area under the curve) on all datasets sourced from different countries, employing a leave-one-center-out-validation approach. Per our approximation of the feature significance of each region on the atlas, we identified marked differences in the thalamus, pallidum, and inferior frontal gyrus between individuals with schizophrenia and healthy controls, lending credence to prior research findings. At the same time, in order to translate the model's output into clinically applicable insights, the SHapley Additive exPlanations (SHAP) permutation explainer method with an anatomical atlas have been refined, thereby offering precise neuroanatomical and functional interpretations of different brain regions.

An interpretable deep learning model to map land subsidence hazard.

The main goal of this research is the interpretability of deep learning (DL) model output (e.g., CNN and LSTM) used to map land susceptibility to subsidence hazard by means of different techniques. For this purpose, an inventory map of land subsidence (LS) is prepared based on fieldwork and a record of LS presence points, and 16 features controlling LS were mapped. Thereafter, 11 effective features controlling LS were identified by means of a particle swarm optimization (PSO) algorithm, which was then used as input in the CNN and LSTM predictive models. To address the inherent black box nature of DL models, six interpretation methods (feature interaction, permutation importance plot (PFIM), bar plot, SHapley Additive exPlanations (SHAP) main plot, heatmap plot, and waterfall plot) were used to interpret the predictive model outputs. Both models (CNN and LSTM) had AUC > 90 and therefore provided excellent accuracy for mapping LS hazard. According to the most accurate model-the CNN predictive model-the range from very low to very high hazard classes occupied 20%, 20%, 25%, 16.3%, and 18.7% of the study area, respectively. According to three plots (bar plot, SHAP main plot, and heatmap plot), which were constructed based on the SHAP value, distance from the well, GDR and DEM were identifiedas the three most important features with the highest impact on the DL model output. The results of the waterfall plot indicate two effective features consisting of distance from the well and coarse fragment, and two effective features comprising landuse and DEM, contributed negatively and positively to LS, respectively. Overall, these explanation techniques can address critical concerns with respect to the interpretability of sophisticated DL predictive models.

SHAP values accurately explain the difference in modeling accuracy of convolution neural network between soil full-spectrum and feature-spectrum

Acquiring soil nutrient content quickly and accurately through remote sensing is the key to advance precision agriculture. The development of deep learning has provided new technical means for soil hyperspectral modeling. However, the problem of poor interpretability of deep learning models limits its development. Although SHapley Additive exPlanations (SHAP) values based on game theory have been successfully applied to the interpretation of deep learning soil spectral modeling, whether they can accurately explain the differences in deep learning model accuracy remains to be verified. Based on this, we explored whether SHAP values can accurately explain the differences in convolutional neural network (CNN) modeling accuracy. We collected soil samples from agricultural land in the Liangshui River Basin in the southern mountainous and hilly areas of China, and measured the soil total nitrogen (STN) content and soil spectral data in the laboratory. We compared the effects of full-spectrum and feature-spectrum on the accuracy of deep learning models, and obtained the contribution of wavelengths in the CNN modeling process by calculating SHAP values. The results showed that combining different spectral pre-processing methods can play their respective advantages and help improve modeling accuracy. Among them, the CNN model obtained the highest prediction accuracy under the first-derivative Savitzky-Golay smoothing combination standard normal variate (SG1-SNV) spectral pre-processing in full-spectrum modeling. Compared with the feature-spectrum selected for modeling by Mutual information (MI) and competitive adaptive reweighted sampling (CARS), the CNN model achieved higher accuracy in most pre-processed spectra in full-spectrum modeling, and SHAP values accurately explained this reason. This is because the contribution is usually higher at most wavelengths with a high correlation with STN content. The feature-spectrum selected by CARS is more widely distributed but lacks continuity, and some wavelengths with high correlation and high contribution will also be missed. Meanwhile, some wavelengths with low correlation also have high contributions, which are usually not involved in the feature spectrum modeling of MI, thus affecting the modeling accuracy. Therefore, the deep learning model is more suitable for full-spectrum modeling due to its strong feature extraction and self-learning capabilities, and SHAP can obtain the wavelength contribution of the CNN model in soil spectral modeling, and then explain the differences in modeling accuracy. This study further proves the interpretability of deep learning, provides an important basis for the application of deep learning in soil hyperspectral modeling.

Signs and Supersigns in Deep Learning

Semiotics is the study of signs and sign-using behavior. Computational semiotics is an interdisciplinary field which proposes a new kind of approach to intelligent systems, where an explicit account for the notion of sign is prominent. Our fundamental thesis is that information concentration processes appear in successive layers of deep learning models: each layer aggregates information from the previous layer of the network. In computational semiotics, this information concentration is known as superization, and it is accompanied by a decrease of entropy: signs are aggregated into supersign. Our interdisciplinary approach enables us to depict superization processes within deep learning models. This is a novel semantic interpretation of deep learning. We use concepts from computational semiotics to explain decision processes in deep learning. Semiotic tools can be used to optimize the architecture of deep neural networks. Interpretability/explainability and architecture optimization of neural models are currently among the hottest topics in machine learning. We illustrate our semiotic approach with several applications. Our contribution can be seen as the initial move in establishing a cohesive semiotic framework for deep learning models.

A Weakly Supervised Gradient Attribution Constraint for Interpretable Classification and Anomaly Detection.

The lack of interpretability of deep learning reduces understanding of what happens when a network does not work as expected and hinders its use in critical fields like medicine, which require transparency of decisions. For example, a healthy vs pathological classification model should rely on radiological signs and not on some training dataset biases. Several post-hoc models have been proposed to explain the decision of a trained network. However, they are very seldom used to enforce interpretability during training and none in accordance with the classification. In this paper, we propose a new weakly supervised method for both interpretable healthy vs pathological classification and anomaly detection. A new loss function is added to a standard classification model to constrain each voxel of healthy images to drive the network decision towards the healthy class according to gradient-based attributions. This constraint reveals pathological structures for patient images, allowing their unsupervised segmentation. Moreover, we advocate both theoretically and experimentally, that constrained training with the simple Gradient attribution is similar to constraints with the heavier Expected Gradient, consequently reducing the computational cost. We also propose a combination of attributions during the constrained training making the model robust to the attribution choice at inference. Our proposition was evaluated on two brain pathologies: tumors and multiple sclerosis. This new constraint provides a more relevant classification, with a more pathology-driven decision. For anomaly detection, the proposed method outperforms state-of-the-art especially on difficult multiple sclerosis lesions segmentation task with a 15 points Dice improvement.

Towards self-explainable graph convolutional neural network with frequency adaptive inception

Graph convolutional neural networks (GCNs) have demonstrated powerful representing ability of irregular data, e.g., skeletal data and graph-structured data, providing the effective mechanism to fuse the neighbor nodes. However, inheriting from the deep learning, GCN also lacks interpretability, which hinders its application to scenarios that have high demand for transparency. Although, there have been many efforts on the interpretability of deep learning, they mainly concentrate on i.i.d data that is hard to be deployed to GCNs, which involve not only the node feature, but also the graph structure. There are few works that attempt to explain it with post-hoc manner, which can be biased, resulting in mis-representation of the true explanation. Therefore, in this paper, we propose a framework, namely ExpFiGCN, that reveals explainability of the GCNs from the perspective of graph structure and mathematical analysis. Specifically, ExpFiGCN can find the most intrinsically relevant node to the central node and obtain the informative and discriminative signals while performing denoising. For the graph structure, we find K-nearest nodes; for the mathematical analysis, every channel of a node and its neighborhoods contribute dynamically to the final channel signal, which can capture the inherent difference of different channels and neighbor nodes. Meanwhile, it can enhance the representation ability of nodes and ameliorate the over-smoothing problem. On the other hand, our model can dynamically adjust the importance of neighborhoods to the central vertex. We empirically validate the effectiveness of the proposed framework ExpFiGCN on various benchmark datasets. Experimental results show that our method achieves substantial improvements and outperforms the state-of-the-art performance strikingly.

COSMOS: a platform for real-time morphology-based, label-free cell sorting using deep learning

Cells are the singular building blocks of life, and a comprehensive understanding of morphology, among other properties, is crucial to the assessment of underlying heterogeneity. We developed Computational Sorting and Mapping of Single Cells (COSMOS), a platform based on Artificial Intelligence (AI) and microfluidics to characterize and sort single cells based on real-time deep learning interpretation of high-resolution brightfield images. Supervised deep learning models were applied to characterize and sort cell lines and dissociated primary tissue based on high-dimensional embedding vectors of morphology without the need for biomarker labels and stains/dyes. We demonstrate COSMOS capabilities with multiple human cell lines and tissue samples. These early results suggest that our neural networks embedding space can capture and recapitulate deep visual characteristics and can be used to efficiently purify unlabeled viable cells with desired morphological traits. Our approach resolves a technical gap in the ability to perform real-time deep learning assessment and sorting of cells based on high-resolution brightfield images.

A Novel Physics-Statistical Coupled Paradigm for Retrieving Integrated Water Vapor Content Based on Artificial Intelligence

Retrieval of integrated water vapor content (WVC) from remote sensing data is often ill-posed because of insufficient observational information. There are many factors that cause WVC changes, which yield instability in the accuracy of many traditional algorithms. To overcome this problem, we developed a novel fully-coupled paradigm for the robust retrieval of WVC from thermal infrared remote sensing data. Through the derivation of the physical radiative transfer equation, we determined two conditions that need to be satisfied for the deep learning retrieval paradigm of WVC. The first condition is that the input parameters and output parameters of the deep learning need to be able to build a complete set of solvable equations in theory. The second condition is that, if there is a strong relationship between input parameters and output parameters, it can be directly retrieved. If it is a weak relationship, we need to use prior knowledge to improve the portability and accuracy of the algorithm. The training and test data of deep learning is composed of representative solutions of physical methods and solutions of statistical methods. The representative solutions of the physical methods were obtained from the physical forward model, and the statistical solutions were obtained from multi-source data which can compensate for the defect that the physical model cannot simulate mixed pixels. MODIS L1B data was used for case analysis of paradigm retrieval, and the analysis indicated that four thermal infrared bands were usually needed as the input parameters of deep learning and the integrated water vapor content as the output parameter. When land surface temperature and emissivity were taken as prior knowledge, the root-mean-square error (RMSE) of the retrieved WVC was 0.07 g/cm2. The optimal accuracy RMSE was 0.27 g/cm2. When there was a strong correlation between input parameters and output parameters, i.e., if there were two bands that were very sensitive to WVC in the band combination, high-precision retrieval could also be achieved without prior knowledge. All the analyses show that the paradigm of deep learning coupling physics and statistics can accurately retrieve WVC, which is a significant improvement on the traditional method and solves the problem of lack of physical interpretation of deep learning.

Clinical Interpretability of Deep Learning for Predicting Microvascular Invasion in Hepatocellular Carcinoma by Using Attention Mechanism.

Preoperative prediction of microvascular invasion (MVI) is essential for management decision in hepatocellular carcinoma (HCC). Deep learning-based prediction models of MVI are numerous but lack clinical interpretation due to their "black-box" nature. Consequently, we aimed to use an attention-guided feature fusion network, including intra- and inter-attention modules, to solve this problem. This retrospective study recruited 210 HCC patients who underwent gadoxetate-enhanced MRI examination before surgery. The MRIs on pre-contrast, arterial, portal, and hepatobiliary phases (hepatobiliary phase: HBP) were used to develop single-phase and multi-phase models. Attention weights provided by attention modules were used to obtain visual explanations of predictive decisions. The four-phase fusion model achieved the highest area under the curve (AUC) of 0.92 (95% CI: 0.84-1.00), and the other models proposed AUCs of 0.75-0.91. Attention heatmaps of collaborative-attention layers revealed that tumor margins in all phases and peritumoral areas in the arterial phase and HBP were salient regions for MVI prediction. Heatmaps of weights in fully connected layers showed that the HBP contributed the most to MVI prediction. Our study firstly implemented self-attention and collaborative-attention to reveal the relationship between deep features and MVI, improving the clinical interpretation of prediction models. The clinical interpretability offers radiologists and clinicians more confidence to apply deep learning models in clinical practice, helping HCC patients formulate personalized therapies.

An Alternative to Cognitivism: Computational Phenomenology for Deep Learning

We propose a non-representationalist framework for deep learning relying on a novel method computational phenomenology, a dialogue between the first-person perspective (relying on phenomenology) and the mechanisms of computational models. We thereby propose an alternative to the modern cognitivist interpretation of deep learning, according to which artificial neural networks encode representations of external entities. This interpretation mainly relies on neuro-representationalism, a position that combines a strong ontological commitment towards scientific theoretical entities and the idea that the brain operates on symbolic representations of these entities. We proceed as follows: after offering a review of cognitivism and neuro-representationalism in the field of deep learning, we first elaborate a phenomenological critique of these positions; we then sketch out computational phenomenology and distinguish it from existing alternatives; finally we apply this new method to deep learning models trained on specific tasks, in order to formulate a conceptual framework of deep-learning, that allows one to think of artificial neural networks’ mechanisms in terms of lived experience.

Exploration of dual-attention mechanism-based deep learning for multi-step-ahead flood probabilistic forecasting

In order to improve the flood forecasting accuracy and reflect the forecast uncertainty information in the Three Gorges Reservoir (TGR) interval-basin in China, this study integrates the feature and temporal dual-attention (DA) mechanism and recursive encoder-decoder (RED) structure into the long short-term memory (LSTM) neural network to develop a DA-LSTM-RED model. The feature attention acts on the input variables of the encoder, and the temporal attention mechanism acts on the hidden layer states extracted by the LSTM neural network during encoding process, prompting the proposed model to extract critical input information among different types and moments of input variables to improve the multi-step-ahead flood forecasting accuracy. Second, the copula-based Hydrological Uncertainty Processor (copula-HUP) is used to quantify the forecast uncertainty of the proposed model meanwhile creating multi-step-ahead flood probabilistic forecasts. Combining the long-term 6 h hydrologic data series of the Xiangjiaba-TGR interval-basin and the forecasted precipitation from the European Centre for Medium-Range Weather Forecasts (ECMWF), the effectiveness of the proposed model, the effect of forecast precipitation on multi-step-ahead flood forecasting, and the effect of different copula functions on the probabilistic forecast of copula-HUP are investigated, respectively. The results show that the DA-LSTM-RED model can effectively improve the forecasting accuracy for long forecast horizons (3-7d) compared to the LSTM-RED model, and the average absolute error metrics are reduced by 10%-17%. Meanwhile, the proposed model can identify input variables with a high correlation with the target output variables, which improves the interpretability of deep learning to a certain extent. The Student copula-HUP has the lowest RB and CRPS metrics than the Frank and Gaussian copula-HUP, which can better quantify the DA-LSTM-RED model's forecast uncertainty. Therefore, combining the proposed model with the Student copula-HUP can effectively reduce the forecasting uncertainty, enhance the forecasting reliability and accuracy for future horizons, and provide reliable risk information for the TGR flood control scheduling decision.

DRL: Dynamic rebalance learning for adversarial robustness of UAV with long-tailed distribution

Adversarial robustness has attracted extensive studies in various fields by increasing the interpretability of deep learning and enhancing the understanding of neural network models. In realistic scenarios such as UAV control system, imbalanced datasets are a consensus. Therefore, how to solve the adversarial robustness on imbalanced datasets is a more and more inescapable problem in UAV control system. There have been some works on adversarial robustness on imbalanced datasets, which bring us a deeper understanding of vulnerability of deep neural networks and the generation of adversarial examples. To adjust the classification plane after training, the long-tailed robustness framework is usually designed to be multi-stage, and different classifiers are used in different stages, which can improve the robustness through multi learning. The existing methods are hardly considered to effectively handle the long-tailed robustness problem in UAV control system. To explore the intrinsic features of long-tailed robustness, we propose a one-stage robustness framework. First, we study different classifiers and propose a general cosine classifier. By changing the general cosine classifier adaptively, the model obtains a more robust classification. Then, we analyze the scalability of the focal loss and design a focal-margin loss. Finally, we design a category focus mobile learning strategy, obtained more robust features by changing the learning emphasis with this strategy. From this, we design a simple and efficient one-stage dynamic adversarial robustness method DRL under long-tailed distribution, which consists of an adaptive cosine classifier and a focal-margin loss under long-tailed mobile learning. The extended experiments demonstrate the superiority of our approach over other state-of-the-art methods, and the effectiveness of the designed module. This method can effectively solve the long-tailed robustness problem on UAV control system and other terminals.

Ensemble Image Explainable AI (XAI) Algorithm for Severe Community-Acquired Pneumonia and COVID-19 Respiratory Infections

Since the onset of the COVID-19 pandemic in 2019, many clinical prognostic scoring tools have been proposed or developed to aid clinicians in the disposition and severity assessment of pneumonia. However, there is limited work that focuses on explaining techniques that are best suited for clinicians in their decision making. In this article, we present a new image explainability method named ensemble AI explainability (XAI), which is based on the SHAP and Grad-CAM++ methods. It provides a visual explanation for a deep learning prognostic model that predicts the mortality risk of community-acquired pneumonia and COVID-19 respiratory infected patients. In addition, we surveyed the existing literature and compiled prevailing quantitative and qualitative metrics to systematically review the efficacy of ensemble XAI, and to make comparisons with several state-of-the-art explainability methods (LIME, SHAP, saliency map, Grad-CAM, Grad-CAM++). Our quantitative experimental results have shown that ensemble XAI has a comparable absence impact (decision impact: 0.72, confident impact: 0.24). Our qualitative experiment, in which a panel of three radiologists were involved to evaluate the degree of concordance and trust in the algorithms, has showed that ensemble XAI has localization effectiveness (mean set accordance precision: 0.52, mean set accordance recall: 0.57, mean set <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${{\bm{F}}_1}$</tex-math></inline-formula> : 0.50, mean set IOU: 0.36) and is the most trusted method by the panel of radiologists (mean vote: 70.2%). Finally, the deep learning interpretation dashboard used for the radiologist panel voting will be made available to the community. Our code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/IHIS-HealthInsights/Interpretation-Methods-Voting-dashboard</uri> .

Special issue on deep interpretation of deep learning: prediction, representation, modeling and utilization

Special issue on deep interpretation of deep learning: prediction, representation, modeling and utilization

GLH: From Global to Local Gradient Attacks with High-Frequency Momentum Guidance for Object Detection.

The adversarial attack is crucial to improving the robustness of deep learning models; they help improve the interpretability of deep learning and also increase the security of the models in real-world applications. However, existing attack algorithms mainly focus on image classification tasks, and they lack research targeting object detection. Adversarial attacks against image classification are global-based with no focus on the intrinsic features of the image. In other words, they generate perturbations that cover the whole image, and each added perturbation is quantitative and undifferentiated. In contrast, we propose a global-to-local adversarial attack based on object detection, which destroys important perceptual features of the object. More specifically, we differentially extract gradient features as a proportion of perturbation additions to generate adversarial samples, as the magnitude of the gradient is highly correlated with the model's point of interest. In addition, we reduce unnecessary perturbations by dynamically suppressing excessive perturbations to generate high-quality adversarial samples. After that, we improve the effectiveness of the attack using the high-frequency feature gradient as a motivation to guide the next gradient attack. Numerous experiments and evaluations have demonstrated the effectiveness and superior performance of our from global to Local gradient attacks with high-frequency momentum guidance (GLH), which is more effective than previous attacks. Our generated adversarial samples also have excellent black-box attack ability.

An Attribution Deep Learning Interpretation Model for Landslide Susceptibility Mapping in the Three Gorges Reservoir Area

An Attribution Deep Learning Interpretation Model for Landslide Susceptibility Mapping in the Three Gorges Reservoir Area

A formal validation of a deep learning-based automated workflow for the interpretation of the echocardiogram

This study compares a deep learning interpretation of 23 echocardiographic parameters—including cardiac volumes, ejection fraction, and Doppler measurements—with three repeated measurements by core lab sonographers. The primary outcome metric, the individual equivalence coefficient (IEC), compares the disagreement between deep learning and human readers relative to the disagreement among human readers. The pre-determined non-inferiority criterion is 0.25 for the upper bound of the 95% confidence interval. Among 602 anonymised echocardiographic studies from 600 people (421 with heart failure, 179 controls, 69% women), the point estimates of IEC are all <0 and the upper bound of the 95% confidence intervals below 0.25, indicating that the disagreement between the deep learning and human measures is lower than the disagreement among three core lab readers. These results highlight the potential of deep learning algorithms to improve efficiency and reduce the costs of echocardiography.

Graph Neural Tree: A novel and interpretable deep learning-based framework for accurate molecular property predictions

Determining various properties of molecules is a critical step in drug discovery. Recently, with the improvement of large heterogeneous datasets and the development of deep learning approaches, more and more scientists have turned their attention to neural network-based virtual preliminary screening to reduce the time and monetary cost of drug discovery. However, the poor interpretability of deep learning masks causality, so models' conclusions are often beyond the comprehension of human users, which reduces the credibility of the model and makes it difficult for chemists to further narrow the huge chemical space based on models’ results. Thus, this study develops a novel framework consisting of Graph Neural Networks for feature extraction, Curriculum-Based Learning Strategies for optimization, and a Learning Binary Neural Tree (LBNT) for prediction, to improve the performance of neural networks and reveal their decision-making process to chemists. The framework encodes molecular graph data with graph neural networks (GNNs), then retrains the encoder with curriculum-based learning strategies to reduce uncertainty and improve accuracy, and finally uses LBNT as the predictor, which joint retrains with the encoder after independently training, for prediction and visualization. The framework is validated on the public datasets and compared to single GNNs with normal training strategies as well as GNN encoders with common machine learning predictors instead of the LBNT predictor. The result reveals that the proposed framework enhances the point prediction accuracy of the completely trained GNN and reduces its uncertainty through curriculum-based learning, and further improves the accuracy by combining LBNT. Besides, compared with common machine learning tools, the LBNT predictor generally has the best performance because of joint retraining with the GNN encoder. The decision-making process of LBNT is also better and easier to explain than that of other models.

Deep Transfer Learning With Self-Attention for Industry Sensor Fusion Tasks

Monitoring of complex industrial processes can be achieved by obtaining process data by utilising various sensing modalities. The recent emergence of deep learning provides a new routine for processing multi-sensor information. However, the learning ability of shallow neural networks is insufficient, and the data amount required by deep networks is often too large for industrial scenarios. This paper provides a novel deep transfer learning method as a possible solution that offers an advantage of better learning ability of the deep network without the requirement for a large amount of training data. This paper presents how Transformer with self-attention trained from natural language can be transferred to the sensor fusion task. Our proposed method is tested on 3 datasets: condition monitoring of a hydraulic system, bearing, and gearbox dataset. The results show that the Transformer trained from natural language can effectively reduce the required data amount for using deep learning in industrial sensor fusion with high prediction accuracy. The difficult and uncertain artificial feature engineering which requires a large workload can also be eliminated, as the deep networks are able to extract features automatically. In addition, the self-attention mechanism of Transformer aids in the identification of critical sensors, hence the interpretability of deep learning in industrial sensor fusion can be improved.