Current Convolutional Neural Networks Research Articles

S2Former: Parallel Spectral–Spatial Transformer for Hyperspectral Image Classification

Due to their excellent representation talent in local features, the convolutional neural network (CNN) has achieved favourable performance in hyperspectral image (HSI) classification tasks. Nevertheless, current CNN models exhibit a marked flaw: they are hard to model the dependencies in long-range distanced positions. This flaw becomes more problematic for the HSI classification task, which targets extracting more discriminative features in local and global dimensions from limited samples. In this paper, we introduce a spatial–spectral transformer (S2Former), which explores spatial and spectral feature extraction in a dual-stream framework for HSI Classification. S2Former, which consists of a spatial transformer and a spectral transformer in parallel branches, extracts the discriminative feature in spatial and spectral dimensions. More specifically, we propose multi-head spatial self-attention to capture the long-range spatial dependency of non-adjacent HSI pixels in a spatial transformer. In the spectral transformer, we propose multi-head covariance spectral attention to mine and represent spectral signatures by computing covariance-based channel maps. Meanwhile, the local activation feed-forward network is developed to complement local details. Extensive experiments conducted on four publicly available datasets indicate that our S2Former achieves state-of-the-art performance for the HSI classification task.

Artificial Intelligence's Use in the Diagnosis of Mouth Ulcers: A Systematic Review.

Artificial intelligence (AI) has been cited as being helpful in the diagnosis of diseases, the prediction of prognoses, and the development of patient-specific therapeutic strategies. AI can help dentists, in particular, when they need to make important judgments quickly. It can eliminate human mistakes in making decisions, resulting in superior and consistent medical treatment while lowering the workload on dentists.The existing studies relevant to the study and application of AI in the diagnosis of various forms of mouth ulcers are reviewed in this work. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards were followed in the preparation of the review. There were no rule violations, with the significant exception of the use of a better search method that led to more accurate findings. Using search terms mainly such as AI, oral health, oral ulcers, oral herpes simplex, oral lichen planus, pemphigus vulgaris, recurrent aphthous ulcer (RAU), oral cancer, premalignant and malignant disorders, etc., a comprehensive search was carried out in the reliable sources of literature, namely PubMed, Scopus, Embase, Web of Science, Ovid, Global Health, and PsycINFO. For all papers, exhaustive searches were done using inclusion criteria as well as exclusion criteria between June 28, 2018, and June 28, 2023. An AI framework for the automatic categorization of oral ulcers from oral clinical photographs was developed by the authors, and it performed satisfactorily. The newly designed AI model works better than the current convolutional neural network image categorization techniques and shows a fair level of precision in the classification of oral ulcers. However, despite being useful for identifying oral ulcers, the suggested technique needs a broader set of data for validation and training purposes before being used in clinical settings. Automated OCSCC identification using a deep learning-based technique is a quick, harmless, affordable, and practical approach to evaluating the effectiveness of cancer treatment. The categorization and identification of RAU lesions through the use of non-intrusive oral pictures using the previously developed ResNet50 and YOLOV algorithms demonstrated better accuracy as well as adequate potential for the future, which could be helpful in clinical practice. Moreover, the most reliable projections for the likelihood of the presence or absence of RAU were made by the optimized neural network. The authors also discovered variables associated with RAU that might be used as input information to build artificial neural networks that anticipate RAU.

Towards an adversarially robust convolutional neural network for automated diagnosis of malaria infection from microscopy images

Malaria is a life-threatening mosquito-borne disease of global importance, and it is most prevalent in the low-income countries of the developing world. The diagnosis of malaria infection is conventionally performed by laboratorians through visual examination of blood smear images under a microscope. However, the results of such diagnostic testing depend on the laboratory technicians' experience with staining and image interpretation, which can be severely lacking in low-resource settings. Convolutional neural networks (CNNs) have great potential for automated malaria diagnosis from images without the need of human expert interaction. However, studies have shown that current CNNs are not robust to adversarial noises which are small input perturbations that cause CNNs to incorrectly classify medical images. In this study, we developed a CNN for malaria parasite identification and evaluated it on a dataset of 27,558 cell images through patient-level cross validation. To improve robustness against adversarial noises for this application, the structure of the CNN was specially designed to limit the receptive field of the output, and adversarial training was applied with gradually increasing level of noises. Compared to three baseline networks (ResNet-18, MobileNet, and MnasNet) and two variants of the CNN, our custom-designed CNN achieved a substantially lower computation cost and better adversarial robustness against PGD-generated adversarial noises on the malaria image dataset. Our study holds the promise of developing a robust and affordable solution for automated diagnosis of malarial infection.

Convolutional neural networks for vision neuroscience: significance, developments, and outstanding issues.

Convolutional Neural Networks (CNN) are a class of machine learning models predominately used in computer vision tasks and can achieve human-like performance through learning from experience. Their striking similarities to the structural and functional principles of the primate visual system allow for comparisons between these artificial networks and their biological counterparts, enabling exploration of how visual functions and neural representations may emerge in the real brain from a limited set of computational principles. After considering the basic features of CNNs, we discuss the opportunities and challenges of endorsing CNNs as in silico models of the primate visual system. Specifically, we highlight several emerging notions about the anatomical and physiological properties of the visual system that still need to be systematically integrated into current CNN models. These tenets include the implementation of parallel processing pathways from the early stages of retinal input and the reconsideration of several assumptions concerning the serial progression of information flow. We suggest design choices and architectural constraints that could facilitate a closer alignment with biology provide causal evidence of the predictive link between the artificial and biological visual systems. Adopting this principled perspective could potentially lead to new research questions and applications of CNNs beyond modeling object recognition.

Two-Stream Boundary-Aware Neural Network for Concrete Crack Segmentation and Quantification

Cracks can be important performance indicators for determining damage processes in new and existing concrete structures. In recent years, deep convolutional neural networks (CNNs) have shown great potential in automatic crack detection and segmentation. However, most of the current CNNs tend to lose high-resolution details and, therefore, lead to blurry object boundaries; this results in poor performance for crack images with complex backgrounds in engineering structures. This study proposes a two-stream boundary-aware crack segmentation (BACS) network that combines semantic image segmentation with semantically informed edge detection explicitly. Firstly, a high-resolution network (HRNet) is utilized in the segmentation branch for strong high-resolution representations through repeatedly conducting multi-scale fusions across parallel convolutions. Furthermore, an edge branch is utilized for preserving fine-grained details of elongated thin cracks, which adopts a modified dynamic feature fusion (DFF) network to produce more accurate and sharper edge predictions. The proposed method is evaluated using a dataset of 1,892 images for three different scenarios. The results show that the mean intersection-over-union (mIoU) scores reach 79.26%, 68.74%, and 70.31% for pure crack, complex background, and variable-width scenarios, respectively. In addition, crack width quantification is performed to validate the accuracy in terms of engineering practice. The BACS achieves high accuracy with an average absolute error of 0.0992 mm, which corresponds to approximately two pixels in the images. In conclusion, the study provides an effective solution for the crack segmentation task, especially for the variable-width scenario, providing an accurate data foundation for the digital twin of concrete structures.

Proposing a new approach based on convolutional neural networks and random forest for the diagnosis of Parkinson's disease from speech signals

Parkinson which occurs because of affected motor system by central nervous system is a neurodegenerative disease which is often seen in community. This disease, which is frequently seen especially in the elderly, brings problems such as speech disorders in patients. It is seen that with the rapidly developing deep learning and machine learning methods in recent years, it is possible to distinguish speech disorders in PD patients at a high rate and quickly. In this study, PD diagnosis was performed using datasets containing voice signals of healthy individuals and PD patients (PD_Dataset and PDO_Dataset). Current convolutional neural networks (CNN) and machine learning (ML) algorithms for PD diagnosis have been examined and a comparative performance analysis has been made. In addition, a different method called SkipConNet + RF based on CNN and random forest (RF) has been proposed for PD diagnosis. With the proposed SkipConNet, important features were obtained from the speech signals; then, the estimation process was performed using the RF algorithm. The proposed method provided an improvement between 3% and 17.19% in the performance of RF algorithms. In addition, the SkipConNet + RF method showed the highest success with 99.11% accuracy in the PD_Dataset dataset and 98.30% in the PDO_Dataset dataset.

Seismic random noise attenuation via a two-stage U-net with supervised attention

Random noise, which has a significant impact on subsequent processing and interpretation, easily interferes with seismic data. Current convolutional neural networks (CNN) use a single-stage technique to boost network capacity by exploiting the complicated network structure, but the performance of the network becomes saturated and prone to overfitting at a certain stage. Hence, we propose a two-stage U-Net denoising network with a supervised attention module (UNet-SAM). In this supervised algorithm, the first stage obtains the pre-denoising results, while the second stage achieves more accurate data. The supervised attention module (SAM) block is inserted in the first stage, extracting features with supervised attention to utilise as a priori information and guide the fine denoising in the second stage. The combination of the attention mechanism and two-stage strategy provides prior information that helps to train a network with better denoising performance. Experiments on synthetic and field data illustrate that the proposed UNet-SAM not only has a superior denoising effect but also retains more of the original effective signal.

Suppressing DFB Lasers Relaxation Oscillations by High-Order Continuous Shaping Current With CNN Prediction

A technique of continuous shaping current waveform to suppress relaxation oscillations (ROs) of distributed feedback (DFB) laser for a high-performance optic system is demonstrated. To effectively suppress ROs, expressions for the shaping current waveform are theoretically derived based on the rate equations and different polynomials for the 3 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">rd</sup> , 5 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sup> , and 8 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sup> order Fourier basis functions are introduced. The convolutional neural network (CNN) is employed to predict the multi-parameter values that determine the results of the shaping input current, which exempt from the difficult and time-consuming process of parameter selection. Prior to training, preprocessing of the data obtained from DFB laser forward simulation using min-max normalization aims to improve the training efficiency of the CNN. The shaping current signals obtained from the CNN predicted parameters are put into the equivalent circuit model for the DFB laser to verify the effectiveness of the shaping current technique and CNN parameter optimization. Afterwards, the shaping current waveform is verified in a time division multiplex passive optical network (TDM-PON) utilizing the DFB laser model as a directly modulated source achieving remarkable performance with low cost. The results show that the high-order continuous shaping current modulated technique can successfully suppress the ROs and enhance the performance of the optic system.

Time–frequency Transformer with shifted windows for journal bearing-rotor systems fault diagnosis under multiple working conditions

This paper proposes a new Time–frequency Transformer with shifted windows method (TFSwin-T) for fault diagnosis of journal sliding bearing-rotor systems under multiple working conditions. Current convolutional neural network models struggle with accurate diagnosis in complex and variable working conditions. TFSwin-T uses time–frequency representations of vibration signals to extract features from typical sliding bearing faults, such as scratch, fretting wear, pitting, and fatigue crack. Our model adapts to multiple speeds and reduces computing power consumption through the use of a Transformer encoder with shifted windows. We demonstrate the effectiveness of our optimal fault diagnosis model structure for both journal bearing and rotor fault diagnosis, with higher diagnostic precision and generalizability compared to benchmark models and other advanced methods in complex working conditions.

GMTS: GNN-based multi-scale transformer siamese network for remote sensing building change detection

ABSTRACT With the remarkable success of change detection (CD) in remote sensing images in the context of deep learning, many convolutional neural network (CNN) based methods have been proposed. In the current research, to obtain a better context modeling method for remote sensing images and to capture more spatiotemporal characteristics, several attention-based methods and transformer (TR)-based methods have been proposed. Recent research has also continued to innovate on TR-based methods, and many new methods have been proposed. Most of them require a huge number of calculation to achieve good results. Therefore, using the TR-based mehtod while maintaining the overhead low is a problem to be solved. Here, we propose a GNN-based multi-scale transformer siamese network for remote sensing image change detection (GMTS) that maintains a low network overhead while effectively modeling context in the spatiotemporal domain. We also design a novel hybrid backbone to extract features. Compared with the current CNN backbone, our backbone network has a lower overhead and achieves better results. Further, we use high/low frequency (HiLo) attention to extract more detailed local features and the multi-scale pooling pyramid transformer (MPPT) module to focus on more global features respectively. Finally, we leverage the context modeling capabilities of TR in the spatiotemporal domain to optimize the extracted features. We have a relatively low number of parameters compared to that required by current TR-based methods and achieve a good effect improvement, which provides a good balance between efficiency and performance.

Water spatial distribution in polymer electrolyte membrane fuel cell: Convolutional neural network analysis of neutron radiography

Polymer electrolyte membrane (PEM) fuel cells produce water as byproduct, which may cause electrode “flooding” and reduce cell performance. In operation, water usually builds up downstream in the gas flow channel due to the water production by the oxygen reduction reaction (ORR), leading to a water spatial distribution. In this study, a convolutional neural network (CNN) is presented to analyze neutron radiography images to obtain water spatial variation under various operating conditions. 5 and 10 segments of a fuel cell are analyzed for spatial variations. Image pre-processing treatments are carried out to improve the convolutional neural network accuracy to 96.6%. The results show that liquid water emerges at a position around 55% downstream for 50% relative humidity while the entire cell is subject to two-phase flow for 100% relative humidity under a co-flow configuration. Large water content is present in most of the segments and the near-outlet segment for the counter-flow and co-flow configurations, respectively. In addition, the quad-serpentine cell exhibits more water accumulation than the single serpentine one in most downstream segments. The convolutional neural network results agree well with the data obtained from a pixelation image processing method with an accuracy of 91.8%. Compared with conventional pixelation methods, the convolutional neural network method performs better in speed for high-resolution images. It also shows that the current CNN tool fails to predict local water for small spatial scales, such as 10 segments, which leads to a large error (>27%) in prediction.

Railroad Crossing Surveillance and Foreground Extraction Network: Weakly Supervised Artificial-Intelligence Approach

According to the report to Congress National Strategy to Prevent Trespassing on Railroad Property issued by the Federal Railroad Administration, trespassing is currently the number one cause of all railroad-related deaths. The number of fatalities resulting from trespassing, including both illegally entering and remaining in the railroad right-of-way, is even higher than the number of deaths caused by collisions between vehicles and trains. Accurately and effectively detecting objects at rail crossings is critical for improving railway safety. The convolutional neural network (CNN) is one of the most potent tools in image processing and video surveillance. However, the current CNNs under supervised learning rely on a substantial amount of manually labeled images, which is labor-intensive and time-consuming. This study proposes a weakly supervised learning technique for training foreground-segmentation networks and develops a tailored CNN network, RC-SAFE (railroad crossing surveillance and foreground extraction), to accurately and effectively detect and track both still and moving objects in the railroad crossing area. The weakly supervised learning significantly reduces the load for image labeling. The experimental results on the public CDnet 2014 dataset show that the weakly supervised network outperforms the supervised network and some other state-of-the-art foreground-segmentation methods. Railroad crossing monitoring videos are collected at different railroad crossings to validate the proposed RC-SAFE network. The results confirm the robustness of the proposed RC-SAFE network, and that RC-SAFE can detect any object that does not belong to the railroad crossing, which outperforms object-detection-based models, such as Mask RCNN (a region-based CNN).

Self-progress aggregate learning for weakly supervised salient object detection

The task of salient object detection aims to find the most salient object from the samples. In the field of weakly supervised learning, the existing weakly supervised salient object detection (WSSOD) methods often fail to utilize the limited label information (such as self-affinity or edge features, and scale transform) for learning. Therefore, this paper proposes a self-progress aggregate learning method named SPAL. First, the feature optimization scheme of the edge information module is put forward based on analysis of the problems existing in the current convolutional neural network for detection of the edge information of the object. Obviously, the salient object has a low requirement for high-level information, In particular, in order to improve the utilization rate of the network structure without increasing its complexity, the affinity global context is design in view of the particularity of the structure of a salient object. The structure of a salient object not only depends on the deep-level semantic feature information to a certain extent, but also has a certain guiding effect on the object position and edge information. Second, high-level affinity information is used to complement the slight-level edge information globally, and the scale attention module is adopted to guide the network to adapt the multi-scale reinforcement feature learning ability for the salient object regions. Our method SPAL achieved better experimental results than the other competitive models for comparison on five benchmark data sets (i.e. for DUTS-TEST, compared with CCFNet our method achieved an improvement of 0.6% for mean absolute error (MAE), 5.1% for Fb, and 1.1% for Eℰ ), which demonstrates the effectiveness of our proposed method.

Compact Convolutional Neural Network with Multi-Headed Attention Mechanism for Seizure Prediction.

Epilepsy is a neurological disorder related to frequent seizures. Automatic seizure prediction is crucial for the prevention and treatment of epilepsy. In this paper, we propose a novel model for seizure prediction that incorporates a convolutional neural network (CNN) with multi-head attention mechanism. In this model, the shallow CNN automatically captures the EEG features, and the multi-headed attention focuses on discriminating the effective information among these features for identifying pre-ictal EEG segments. Compared with current CNN models for seizure prediction, the embedded multi-headed attention empowers the shallow CNN to be more flexible, and enables improvement of the training efficiency. Hence, this compact model is more resistant to being trapped in overfitting. The proposed method was evaluated over the scalp EEG data from the two publicly available epileptic EEG databases, and achieved outperforming values of event-level sensitivity, false prediction rate (FPR), and epoch-level F1. Furthermore, our method achieved the stable length of seizure prediction time that was between 14 and 15 min. The experimental comparisons showed that our method outperformed other prediction methods in terms of prediction and generalization performance.

Hierarchical neural network with efficient selection inference

The image classification precision is vastly enhanced with the growing complexity of convolutional neural network (CNN) structures. However, the uneven visual separability between categories leads to various difficulties in classification. The hierarchical structure of categories can be leveraged to deal with it, but a few CNNs pay attention to the character of data. Besides, a network model with a hierarchical structure is promising to extract more specific features from the data than current CNNs, since, for the latter, all categories have the same fixed number of layers for feed-forward computation. In this paper, we propose to use category hierarchies to integrate ResNet-style modules to form a hierarchical network model in a top-down manner. To extract abundant discriminative features and improve the computation efficiency, we adopt residual block selection based on coarse categories to allocate different computation paths. Each residual block works as a switch to determine the JUMP or JOIN mode for an individual coarse category. Interestingly, since some categories need less feed-forward computation than others by jumping layers, the average inference time cost is reduced. Extensive experiments show that our hierarchical network achieves higher prediction accuracy with similar FLOPs on CIFAR-10 and CIFAR-100, SVHM, and Tiny-ImageNet datasets compared to original residual networks and other existing selection inference methods.

Fourier-inspired neural module for real-time and high-fidelity computer-generated holography.

Learning-based computer-generated holography (CGH) algorithms appear as novel alternatives to generate phase-only holograms. However, most existing learning-based approaches underperform their iterative peers regarding display quality. Here, we recognize that current convolutional neural networks have difficulty learning cross-domain tasks due to the limited receptive field. In order to overcome this limitation, we propose a Fourier-inspired neural module, which can be easily integrated into various CGH frameworks and significantly enhance the quality of reconstructed images. By explicitly leveraging Fourier transforms within the neural network architecture, the mesoscopic information within the phase-only hologram can be more handily extracted. Both simulation and experiment were performed to showcase its capability. By incorporating it into U-Net and HoloNet, the peak signal-to-noise ratio of reconstructed images is measured at 29.16 dB and 33.50 dB during the simulation, which is 4.97 dB and 1.52 dB higher than those by the baseline U-Net and HoloNet, respectively. Similar trends are observed in the experimental results. We also experimentally demonstrated that U-Net and HoloNet with the proposed module can generate a monochromatic 1080p hologram in 0.015 s and 0.020 s, respectively.

Multi-label classification of chest X-ray images with pre-trained vision Transformer model

目的 基于计算机的胸腔X线影像疾病检测和分类目前存在误诊率高，准确率低的问题。本文在视觉Transformer（vision Transformer，ViT）预训练模型的基础上，通过迁移学习方法，实现胸腔X线影像辅助诊断，提高诊断准确率和效率。方法 选用带有卷积神经网络（convolutional neural network，CNN）的ViT模型，其在超大规模自然图像数据集中进行了预训练；通过微调模型结构，使用预训练的ViT模型参数初始化主干网络，并迁移至胸腔X线影像数据集中再次训练，实现疾病多标签分类。结果 在IU X-Ray数据集中对ViT迁移学习前、后模型平均AUC（area under ROC curve）得分进行对比分析实验。结果表明，预训练ViT模型平均AUC得分为0.774，与不使用迁移学习相比提升了0.208。并针对模型结构和数据预处理进行了消融实验，对ViT中的注意力机制进行可视化，进一步验证了模型有效性。最后使用Chest X-Ray14和CheXpert数据集训练微调后的ViT模型，平均AUC得分为0.839和0.806，与对比方法相比分别有0.014～0.031的提升。结论 与其他方法相比，ViT模型胸腔X线影像的多标签分类精确度更高，且迁移学习可以在降低训练成本的同时提升ViT模型的分类性能和泛化性。消融实验与模型可视化表明，包含CNN结构的ViT模型能重点关注有意义的区域，高效获取胸腔X线影像的视觉特征。;Objective The chest X-ray-relevant screening and diagnostic method is essential for radiology nowadays. Most of chest X-ray images interpretation is still restricted by clinical experience and challenged for misdiagnose and missed diagnoses. To detect and identify one or more potential diseases in images automatically，it is beneficial for improving diagnostic efficiency and accuracy using computer-based technique. Compared to natural images，multiple lesions are challenged to be detected and distinguished accurately in a single image because abnormal areas have a small proportion and complex representations in chest X-ray images. Current convolutional neural network（CNN）based deep learning models have been widely used in the context of medical imaging. The structure of the CNN convolution kernel has sensitive to local detail information，and it is possible to extract richer image features. However，the convolution kernel cannot be used to get global information，and the features-extracted are restricted of redundant information like its relevance of background， muscles，and bones. The model’s performance in multi-label classification tasks are affected to a certain extent. At present，the vision Transformer（ViT）model has achieved its priorities in computer vision-related tasks. The ViT can be used to capture information simultaneously and effectively for multiple regions of the entire image. However，it is required to use large-scale dataset training to achieve good performance. Due to some factors like patient privacy and manual annotate costs，the size of the chest X-ray image data set has been limited. To reduce the model's dependence on data scale and improve the performance of multi-label classification，we develop the CNN-based ViT pre-training model in terms of the transfer learning method for diagnosis-assisted of chest X-ray image and multi-label classification. Method The CNN-based ViT model is pre-trained on a huge scale ground truth dataset，and it is used to obtain the initial parameters of the model. The model structure is fine-tuned according to the features of chest X-ray dataset. A 1×1 convolution layer is used to convert the chest X-ray images channels between 1 to 3. The number of output nodes of the linear layer in the classifier is balanced from 1 000 to the number of chest X-ray classification labels，and the Sigmoid is used as an activation function. The parameters of the backbone network are initialized in terms of the pre-trained ViT model parameters，and it is trained in the chest X-ray dataset after that to complete multi-label classification. The experiment is configured of Python3. 7 and PyTorch1. 8 to construct the model and RTX3090 GPU for training. Stochastic gradient descent（SGD）optimizer，binary cross-entropy（BCE）loss function，an initial learning rate of 1E-3，the cosine annealing learning rate decay are used. For training，each image is scaled to a size of 512×512 pixels，and a 224×224 pixels area and it is then cropped in random as the model input，and data augmentation is performed randomly by some of the flipping，perspective transformation， shearing，translation，zooming，and changing brightness. For testing，the chest X-ray image is scaled to 256×256 pixels and center crop a 224×224 area to input the trained model. Result The experiment is performed on the IU X-Ray，which is a small-scale chest X-ray dataset. This model is evaluated in quantitative using the average of area under ROC curve （AUC）scores across all classification labels. The results show that the average AUC score of the pre-trained ViT model is 0. 774. The accuracy and training efficiency of the non-pre-trained ViT model is dropped significantly. The average AUC score is reached to 0. 566 only，which is 0. 208 lower. In addition，the attention mechanism heat map is generated based on the ViT model，which can strengthen the interpretability of the model. A series of ablation experiments are carried out for data augmentation，model structure，and batch size design. The fine-tuned ViT model is trained on the Chest-Ray14 and CheXpert dataset as well. The average AUC score is reached to 0. 839 and 0. 806，which is optimized by 0. 014 and 0. 031. Conclusion A pre-trained ViT model is used for the multi-label classification of chest X-ray images via transfer learning. The experimental results illustrate that the ViT has its stronger multi-label classification performance in chest Xray images，and its attention mechanism is beneficial for lesions precision-focused like the interior of the chest cavity and the heart. Transfer learning is potential to improve the classification performance and model generalization of the ViT in small-scale datasets，and the training cost is reduced greatly. Ablation experiments demonstrate that the incorporated model of CNN and Transformer has its priority beyond single-structure model. Data enhancement and the batch size cutting can improve the performance of the model，but smaller scale of batch is still interlinked to longer training span. To improve the model's ability，we predict that future research direction can be focused on the extraction for complex disease and highlevel semantic information，such as their small lesions，disease location，and severity.

Understanding How CNNs Recognize Facial Expressions: A Case Study with LIME and CEM.

Recognizing facial expressions has been a persistent goal in the scientific community. Since the rise of artificial intelligence, convolutional neural networks (CNN) have become popular to recognize facial expressions, as images can be directly used as input. Current CNN models can achieve high recognition rates, but they give no clue about their reasoning process. Explainable artificial intelligence (XAI) has been developed as a means to help to interpret the results obtained by machine learning models. When dealing with images, one of the most-used XAI techniques is LIME. LIME highlights the areas of the image that contribute to a classification. As an alternative to LIME, the CEM method appeared, providing explanations in a way that is natural for human classification: besides highlighting what is sufficient to justify a classification, it also identifies what should be absent to maintain it and to distinguish it from another classification. This study presents the results of comparing LIME and CEM applied over complex images such as facial expression images. While CEM could be used to explain the results on images described with a reduced number of features, LIME would be the method of choice when dealing with images described with a huge number of features.

Landslide Susceptibility Mapping by Fusing Convolutional Neural Networks and Vision Transformer

Landslide susceptibility mapping (LSM) is an important decision basis for regional landslide hazard risk management, territorial spatial planning and landslide decision making. The current convolutional neural network (CNN)-based landslide susceptibility mapping models do not adequately take into account the spatial nature of texture features, and vision transformer (ViT)-based LSM models have high requirements for the amount of training data. In this study, we overcome the shortcomings of CNN and ViT by fusing these two deep learning models (bottleneck transformer network (BoTNet) and convolutional vision transformer network (ConViT)), and the fused model was used to predict the probability of landslide occurrence. First, we integrated historical landslide data and landslide evaluation factors and analysed whether there was covariance in the landslide evaluation factors. Then, the testing accuracy and generalisation ability of the CNN, ViT, BoTNet and ConViT models were compared and analysed. Finally, four landslide susceptibility mapping models were used to predict the probability of landslide occurrence in Pingwu County, Sichuan Province, China. Among them, BoTNet and ConViT had the highest accuracy, both at 87.78%, an improvement of 1.11% compared to a single model, while ConViT had the highest F1-socre at 87.64%, an improvement of 1.28% compared to a single model. The results indicate that the fusion model of CNN and ViT has better LSM performance than the single model. Meanwhile, the evaluation results of this study can be used as one of the basic tools for landslide hazard risk quantification and disaster prevention in Pingwu County.

Irregularly sampled seismic data interpolation via wavelet-based convolutional block attention deep learning

Seismic data interpolation, especially irregularly sampled data interpolation, is a critical task for seismic processing and subsequent interpretation. Recently, with the development of machine learning and deep learning, convolutional neural networks (CNNs) are applied for interpolating irregularly sampled seismic data. CNN based approaches can address the apparent defects of traditional interpolation methods, such as the low computational efficiency and the difficulty on parameters selection. However, current CNN based methods only consider the temporal and spatial features of irregularly sampled seismic data, which fail to consider the frequency features of seismic data, i.e., the multi-scale features. To overcome these drawbacks, we propose a wavelet-based convolutional block attention deep learning (W-CBADL) network for irregularly sampled seismic data reconstruction. We firstly introduce the discrete wavelet transform (DWT) and the inverse wavelet transform (IWT) to the commonly used U-Net by considering the multi-scale features of irregularly sampled seismic data. Moreover, we propose to adopt the convolutional block attention module (CBAM) to precisely restore sampled seismic traces, which could apply the attention to both channel and spatial dimensions. Finally, we adopt the proposed W-CBADL model to synthetic and pre-stack field data to evaluate its validity and effectiveness. The results demonstrate that the proposed W-CBADL model could reconstruct irregularly sampled seismic data more effectively and more efficiently than the state-of-the-art contrastive CNN based models.