CNN Module Research Articles

Scale Variant Vehicle Object Recognition by CNN Module of Multi-Pooling-PCA Process

Scale Variant Vehicle Object Recognition by CNN Module of Multi-Pooling-PCA Process

EPT-Net: Edge Perception Transformer for 3D Medical Image Segmentation.

The convolutional neural network has achieved remarkable results in most medical image seg- mentation applications. However, the intrinsic locality of convolution operation has limitations in modeling the long-range dependency. Although the Transformer designed for sequence-to-sequence global prediction was born to solve this problem, it may lead to limited positioning capability due to insufficient low-level detail features. Moreover, low-level features have rich fine-grained information, which greatly impacts edge segmentation decisions of different organs. However, a simple CNN module is difficult to capture the edge information in fine-grained features, and the computational power and memory consumed in processing high-resolution 3D features are costly. This paper proposes an encoder-decoder network that effectively combines edge perception and Transformer structure to segment medical images accurately, called EPT-Net. Under this framework, this paper proposes a Dual Position Transformer to enhance the 3D spatial positioning ability effectively. In addition, as low-level features contain detailed information, we conduct an Edge Weight Guidance module to extract edge information by minimizing the edge information function without adding network parameters. Furthermore, we verified the effectiveness of the proposed method on three datasets, including SegTHOR 2019, Multi-Atlas Labeling Beyond the Cranial Vault and the re-labeled KiTS19 dataset called KiTS19-M by us. The experimental results show that EPT-Net has significantly improved compared with the state-of-the-art medical image segmentation method.

Object-based crop classification in Hetao irrigation zone by using deep learning and region merging optimization

crop classification is conducive to precision agriculture. Due to the cost of high-resolution image collection, it is uneasy to conduct crop classification in remotely sensed scenes using deep networks, which have become increasingly popular in remote sensing. This work combines geographical-based image analysis (GEOBIA) with deep learning for crop classification in a small area. An image classifier network is designed by using multi-scale CNN and transformer modules. The network input is an image transformed from a segment obtained using multi-resolution segmentation (MRS). An iterative optimization framework is developed to correct the segments with under-segmentation errors (USE). Two scenes of high-resolution images are employed for the experiment. The proposed optimization algorithm leads to superior performance to competitors. By using the proposed classifier network as a baseline, the optimization approach can improve overall accuracy (OA) by 4.33 % and 1.29 % respectively for the first and second dataset.

Fully automated sinogram-based deep learning model for detection and classification of intracranial hemorrhage.

To propose an automated approach for detecting and classifying Intracranial Hemorrhages (ICH) directly from sinograms using a deep learning framework. This method is proposed to overcome the limitations of the conventional diagnosis by eliminating the time-consuming reconstruction step and minimizing the potential noise and artifacts that can occur during the Computed Tomography (CT) reconstructionprocess. This study proposes a two-stage automated approach for detecting and classifying ICH from sinograms using a deep learning framework. The first stage of the framework is Intensity Transformed Sinogram Sythesizer, which synthesizes sinograms that are equivalent to the intensity-transformed CT images. The second stage comprises of a cascaded Convolutional Neural Network-Recurrent Neural Network (CNN-RNN) model that detects and classifies hemorrhages from the synthesized sinograms. The CNN module extracts high-level features from each input sinogram, while the RNN module provides spatial correlation of the neighborhood regions in the sinograms. The proposed method was evaluated on a publicly available RSNA dataset consisting of a large sample size of 8652patients. The results showed that the proposed method had a notable improvement as high as 27% in patient-wise accuracies when compared to state-of-the-art methods like ResNext-101, Inception-v3 and Vision Transformer. Furthermore, the sinogram-based approach was found to be more robust to noise and offset errors in comparison to CT image-based approaches. The proposed model was also subjected to a multi-label classification analysis to determine the hemorrhage type from a given sinogram. The learning patterns of the proposed model were also examined for explainability using the activationmaps. The proposed sinogram-based approach can provide an accurate and efficient diagnosis of ICH without the need for the time-consuming reconstruction step and can potentially overcome the limitations of CT image-based approaches. The results show promising outcomes for the use of sinogram-based approaches in detecting hemorrhages, and further research can explore the potential of this approach in clinicalsettings.

OptionNet: A multiscale residual deep learning model with confidence interval to predict option price

Option is an important financial derivative. Accurate option pricing is essential to the development of financial markets. For option pricing, existing time series models and neural networks are difficult to extract multi-scale temporal features from option data, which greatly limits their performance. To solve this problem, we propose a novel deep learning model named as MRC-LSTM-CI. It contains three modules, including Multi-scale Residual CNN module (MRC), Long Short-Term Memory neural network module (LSTM) and confidence interval output module (CI). The proposed model can effectively extract multi-scale features from real market option data, and make interval prediction to provide more information to the decision maker. In addition, the proposed model is further improved using the residual prediction strategy, where the output value is chosen as the residual value between BS theory price and actual market price. Experimental results show that our model has better prediction accuracy than other deep learning models and achieves the state-of-the-art performance.

A combined wind-PV power prediction method based on clustering prescreening

Due to the volatility caused by the high uncertainty of renewable energy, the large-scale integration of renewable energy into the electric power grid brings huge challenges to the scheduling and operation of the electric power system. Effective power prediction of power source can not only relieve the pressure of power system peaking and frequency regulation, but also improve the accuracy of decision-making. So as to make the most of the information between wind and photovoltaic power source, this paper puts forward a wind-PV joint power forecasting model. First, the raw power data is pre-screened by clustering to select the closer connection between wind and PV power; next, the screened data are passed through the upper CNN structure respectively after pre-processed. Then, the outputs from CNN modules are fed together into the prediction layer, which include LSTM layers and fully connected layers. Finally, the proposed wind-PV joint prediction model is verified by simulation experiments.

CTA-UNet: CNN-transformer architecture UNet for dental CBCT images segmentation

In view of the limitations of current deep learning models in segmenting dental cone-beam computed tomography (CBCT) images, specifically dealing with complex root morphological features, fuzzy boundaries between tooth roots and alveolar bone, and the need for costly annotation of dental CBCT images. We collected dental CBCT data from 200 patients and annotated 45 of them for network training, and proposed a CNN-Transformer Architecture UNet network, which combines the advantages of CNN and Transformer. The CNN component effectively extracts local features, while the Transformer captures long-range dependencies. Multiple spatial attention modules were included to enhance the network’s ability to extract and represent spatial information. Additionally, we introduced a novel Masked image modeling method to pre-train the CNN and Transformer modules simultaneously, mitigating limitations due to a smaller amount of labeled training data. Experimental results demonstrate that the proposed method achieved superior performance (DSC of 87.12%, IoU of 78.90%, HD95 of 0.525 mm, ASSD of 0.199 mm), and provides a more efficient and effective approach to automatically and accurately segment dental CBCT images, has real-world applicability in orthodontics and dental implants.

Machinery fault diagnosis using multi-scale feature focused network and adaptive cost-sensitive loss towards imbalanced dataset

The current deep learning based machinery fault diagnosis models still face challenges in effectively capturing rich multi-scale feature information and dynamically compensating training loss when dealing with imbalanced dataset. This paper presents a novel approach for machinery fault diagnosis using multi-scale feature focused network and adaptive cost-sensitive loss. Firstly, a multi-scale feature focused network is constructed with improved multi-scale CNN and point-wise attention mechanism module, in which the former can synthetically fuse the features at different scales to expand the coverage of the equivalent receptive field, and the latter can further refine fine-grained features and filter out irrelevant feature interference. Then, an adaptive cost-sensitive loss function is designed to adjust the cost matrix in the training process, dynamically assigning more loss weights for small samples that are difficult to distinguish. The experimental results of planetary gearbox fault diagnosis demonstrate that the proposed approach exhibits superior diagnostic performance compared to other existing methods.

HyFormer: Hybrid Grouping-Aggregation Transformer and Wide-Spanning CNN for Hyperspectral Image Super-Resolution

Hyperspectral image (HSI) super-resolution is a practical and challenging task as it requires the reconstruction of a large number of spectral bands. Achieving excellent reconstruction results can greatly benefit subsequent downstream tasks. The current mainstream hyperspectral super-resolution methods mainly utilize 3D convolutional neural networks (3D CNN) for design. However, the commonly used small kernel size in 3D CNN limits the model’s receptive field, preventing it from considering a wider range of contextual information. Though the receptive field could be expanded by enlarging the kernel size, it results in a dramatic increase in model parameters. Furthermore, the popular vision transformers designed for natural images are not suitable for processing HSI. This is because HSI exhibits sparsity in the spatial domain, which can lead to significant computational resource waste when using self-attention. In this paper, we design a hybrid architecture called HyFormer, which combines the strengths of CNN and transformer for hyperspectral super-resolution. The transformer branch enables intra-spectra interaction to capture fine-grained contextual details at each specific wavelength. Meanwhile, the CNN branch facilitates efficient inter-spectra feature extraction among different wavelengths while maintaining a large receptive field. Specifically, in the transformer branch, we propose a novel Grouping-Aggregation transformer (GAT), comprising grouping self-attention (GSA) and aggregation self-attention (ASA). The GSA is employed to extract diverse fine-grained features of targets, while the ASA facilitates interaction among heterogeneous textures allocated to different channels. In the CNN branch, we propose a Wide-Spanning Separable 3D Attention (WSSA) to enlarge the receptive field while keeping a low parameter number. Building upon WSSA, we construct a wide-spanning CNN module to efficiently extract inter-spectra features. Extensive experiments demonstrate the superior performance of our HyFormer.

An adaptive multi-modal hybrid model for classifying thyroid nodules by combining ultrasound and infrared thermal images

BackgroundTwo types of non-invasive, radiation-free, and inexpensive imaging technologies that are widely employed in medical applications are ultrasound (US) and infrared thermography (IRT). The ultrasound image obtained by ultrasound imaging primarily expresses the size, shape, contour boundary, echo, and other morphological information of the lesion, while the infrared thermal image obtained by infrared thermography imaging primarily describes its thermodynamic function information. Although distinguishing between benign and malignant thyroid nodules requires both morphological and functional information, present deep learning models are only based on US images, making it possible that some malignant nodules with insignificant morphological changes but significant functional changes will go undetected.ResultsGiven the US and IRT images present thyroid nodules through distinct modalities, we proposed an Adaptive multi-modal Hybrid (AmmH) classification model that can leverage the amalgamation of these two image types to achieve superior classification performance. The AmmH approach involves the construction of a hybrid single-modal encoder module for each modal data, which facilitates the extraction of both local and global features by integrating a CNN module and a Transformer module. The extracted features from the two modalities are then weighted adaptively using an adaptive modality-weight generation network and fused using an adaptive cross-modal encoder module. The fused features are subsequently utilized for the classification of thyroid nodules through the use of MLP. On the collected dataset, our AmmH model respectively achieved 97.17% and 97.38% of F1 and F2 scores, which significantly outperformed the single-modal models. The results of four ablation experiments further show the superiority of our proposed method.ConclusionsThe proposed multi-modal model extracts features from various modal images, thereby enhancing the comprehensiveness of thyroid nodules descriptions. The adaptive modality-weight generation network enables adaptive attention to different modalities, facilitating the fusion of features using adaptive weights through the adaptive cross-modal encoder. Consequently, the model has demonstrated promising classification performance, indicating its potential as a non-invasive, radiation-free, and cost-effective screening tool for distinguishing between benign and malignant thyroid nodules. The source code is available at https://github.com/wuliZN2020/AmmH.

A physics-informed CNN-TSE hybrid network for micro-EDM process monitoring and control

Despite being a widespread non-conventional processing technique, electrical discharge machining (EDM) has been complicated due to its removal mechanisms and weak process stability for decades. Process monitoring is an enabling technology to offer an indirect insight into the discharge phenomena, understanding the probabilistic anomaly events such as arcs and short circuits. However, the transient high-frequency pulse train poses a great challenge for statistical monitoring and control. Traditional approaches are limited to single pulse analysis, which hinders a deep understanding of the process. To address the challenge, this paper proposes a novel physics-informed deep learning architecture for real-time micro-EDM process monitoring. To prepare condition labels, a series of signal-processing techniques including sensory fusion, pulse-shape filtering and pulse segmentation are employed, followed by observation diagrams on two critical indicators, i.e. discharge energy and discharge time interval. A decision logic that allows embedding of domain knowledge is formulated, yielding physics-informed labels on each pulse train. For online monitoring applications, a hybrid hierarchical structure that combines a 1D convolutional neural network (1D-CNN) and a Transformer encoder (TSE) is designed to simultaneously extract local pulse features and temporal dependencies between pulse sequences. This deep learning model achieves a classification accuracy of 96% on the validation dataset and reaches precision and recall scores of more than 92% on the test dataset with varying processing parameters. Two model interpretation approaches, i.e. gradient-weighted class activation mapping (Grad-CAM) and attention visualization, are adopted. Consequently, it is found that the CNN module assigns high feature importance to the breakdown and maintaining stages of one pulse and the TSE module improves the feature salience and reliance on the condition of distinct events. This monitoring network is further demonstrated on a control application for adaptive regulation of the micro-EDM drilling process. The results of deep-hole drilling suggest that the CNN-TSE model can highly improve the process stability and hence the machining quality.

Conv-Swinformer: Integration of CNN and shift window attention for Alzheimer’s disease classification

Deep learning (DL) algorithms based on brain MRI images have achieved great success in the prediction of Alzheimer’s disease (AD), with classification accuracy exceeding even that of the most experienced clinical experts. As a novel feature fusion method, Transformer has achieved excellent performance in many computer vision tasks, which also greatly promotes the application of Transformer in medical images. However, when Transformer is used for 3D MRI image feature fusion, existing DL models treat the input local features equally, which is inconsistent with the fact that adjacent voxels have stronger semantic connections than spatially distant voxels. In addition, due to the relatively small size of the dataset for medical images, it is difficult to capture local lesion features in limited iterative training by treating all input features equally. This paper proposes a deep learning model Conv-Swinformer that focuses on extracting and integrating local fine-grained features. Conv-Swinformer consists of a CNN module and a Transformer encoder module. The CNN module summarizes the planar features of the MRI slices, and the Transformer module establishes semantic connections in 3D space for these planar features. By introducing the shift window attention mechanism in the Transformer encoder, the attention is focused on a small spatial area of the MRI image, which effectively reduces unnecessary background semantic information and enables the model to capture local features more accurately. In addition, the layer-by-layer enlarged attention window can further integrate local fine-grained features, thus enhancing the model’s attention ability. Compared with DL algorithms that indiscriminately fuse local features of MRI images, Conv-Swinformer can fine-grained extract local lesion features, thus achieving better classification results.

Video Surveillance Object Forgery Detection using PDCL Network with Residual-based Steganalysis Feature

Video surveillance has various applications in various fields and industries. However, the rapid development of video processing technology has made video surveillance information susceptible to multiple malicious attacks. At present, the state-of-the-art methods, including the latest deep learning techniques, cannot get satisfactory results when addressing video surveillance object forgery detection (VSOFD) due to the following limitations: (i) lack of VSOFD-specific features for effective processing and (ii) lack of effective deep network architecture designed explicitly for VSOFD. This paper proposes a new detection scheme to alleviate these limitations. The proposed approach first extracted VSOFD-specific features via residual-based steganalysis feature (RSF) from the spatial-temporal-frequent domain. Key clues of video frames can be more effectively learned from RSF, instead of raw frame images. Then, the RSF feature is used to form the residual-based steganography feature vector group (RSFVG), which serves as the input of our following network. Finally, a new VSOFD-specific deep network architecture called parallel-DenseNet-concatenated-LSTM (PDCL) network is designed, which includes two improved CNN and RNN modules. The improved CNN module fuses and processes the coarse-to-fine feature extraction and simultaneously preserves the frame independence in video frames. The improved RNN module learns the correlation features between the adjacent frames to identify forgery frames. Experimental results show that the proposed scheme using the PDCL network with RSF can achieve high performance in test error, precision, recall, and F1 scores in our newly constructed dataset (SYSU-OBJFORG + newly generated forgery video clips). Compared to existing SOTA methods, our framework achieves the best F1 score of 90.33%, which is greatly improved by nearly 8%.

SRT: Improved transformer-based model for classification of 2D heartbeat images

Electrocardiography (ECG) is a crucial tool for diagnosing cardiovascular diseases. In particular, combining clinical ECG with computer technology for automatic ECG analysis can improve ECG discrimination accuracy. In this study, to tackle the existing problems of 2D classification models (CNN and Transformer) of ECG heartbeat images, we proposed a 2D deep learning classification network SRT comprising a CNN and Transformer-encoder modules to improve the model structure. Furthermore, to identify the global and local differences between different types of ECG heartbeats, we propose a novel attention module (SC-RGA module) to assist the multi-headed attention mechanism in improving the feature extraction of ECG images and the Dilated Stem structure to expand the receptive field at the input of the network. We also applied a series of strategies to deal with data imbalances in public ECG datasets. Finally, we evaluated the performance of SRT using the MIT-BIH arrhythmia database, with a classification accuracy of 0.957, a sensitivity of 0.881, and an average F1 index of 0.826. Comparative experiments demonstrate that the proposed model outperforms several advanced methods.

Frequency-based CNN and attention module for acoustic scene classification

Acoustic scene classification (ASC) is an audio classification task that identifies the environment in which sounds are recorded. Audio-related machine learning algorithms suffer from the device mismatch problem; that is, when trained from audio data recorded from one device, the algorithms cannot generalize to audio samples recorded using another device. In this study, a novel convolutional neural network, called a frequency-aware convolutional neural network (FACNN), is introduced to solve the device mismatch problem by focusing on the frequency information of the audio samples. Furthermore, an attention module, called the frequency attention network (FANet), is introduced to generate an attention map based on the frequency information of the input feature maps. FANet helps the FACNN to focus on the important frequency information, thus improving performance. The proposed method is trained on the TAU Urban Acoustic Scenes 2019 Mobile development dataset and TAU Urban Acoustic Scenes 2020 Mobile development dataset. The proposed method achieves a state-of-the-art accuracy of 75.99% in the TAU Urban Acoustic Scenes 2019 Mobile development dataset and a competitive result of 72.6% in the TAU Urban Acoustic Scenes 2020 Mobile development dataset. In addition, a comparison of FANet with the convolutional block attention module (CBAM) and the squeeze-and-excitation network (SENet) was performed. The results show that FANet can mitigate the device mismatch problem by improving the performance of the unseen devices.

Deep Learning Network-Based Evaluation method of Online teaching quality of International Chinese Education

The development of vocational education in the information age requires us to think about the path and strategy of active change. Course teaching quality evaluation should also shift from passive evaluation of online teaching development to active construction of a mixed teaching quality evaluation system. In the information age, the development of teaching resources is dizzying. From paper to digital, from single to diverse, from offline to online, from scarcity to mass—various changes impact the traditional teaching model. Aiming at the online teaching quality evaluation of international Chinese education on the Internet, this paper proposes a method based on deep learning. Firstly, this paper proposes an index system construction and evaluation index weighting for online teaching of international Chinese education, and collects online data as a corpus at the same time. Then construct the CNN_BiLSTM_Att model, which is composed of the CNN module, the BiLSTM module and the Att module. Finally, compare with other model experiments. The experimental results show that CNN_BiLSTM_Att has achieved the best results in the evaluation index results, with P and F1 reaching 97.89% and 97.85%. Compared with other models, the overall effect is improved by 2%~5%. From this, the superiority of the model in the online teaching quality evaluation standard task of this paper can be obtained.

PhenoBERT: A Combined Deep Learning Method for Automated Recognition of Human Phenotype Ontology.

Automated recognition of Human Phenotype Ontology (HPO) terms from clinical texts is of significant interest to the field of clinical data mining. In this study, we develop a combined deep learning method named PhenoBERT for this purpose. PhenoBERT uses BERT, currently the state-of-the-art NLP model, as its core model for evaluating whether a clinically relevant text segment (CTS) could be represented by an HPO term. However, to avoid unnecessary comparison of a CTS with each of ∼14,000 HPO terms using BERT, we introduce a two-levels CNN module consisting of a series of CNN models organized at two levels in PhenoBERT. For a given CTS, the CNN module produces only a short list of candidate HPO terms for BERT to evaluate, significantly improving the computational efficiency. In addition, BERT is able to assign an ancestor HPO term to a CTS when recognition of the direct HPO term is not successful, mimicking the process of HPO term assignment by human. In two benchmarks, PhenoBERT outperforms four traditional dictionary-based methods and two recently developed deep learning-based methods in two benchmark tests, and its advantage is more obvious when the recognition task is more challenging. As such, PhenoBERT is of great use for assisting in the mining of clinical text data.

DeepmRNALoc: A Novel Predictor of Eukaryotic mRNA Subcellular Localization Based on Deep Learning.

The subcellular localization of messenger RNA (mRNA) precisely controls where protein products are synthesized and where they function. However, obtaining an mRNA's subcellular localization through wet-lab experiments is time-consuming and expensive, and many existing mRNA subcellular localization prediction algorithms need to be improved. In this study, a deep neural network-based eukaryotic mRNA subcellular location prediction method, DeepmRNALoc, was proposed, utilizing a two-stage feature extraction strategy that featured bimodal information splitting and fusing for the first stage and a VGGNet-like CNN module for the second stage. The five-fold cross-validation accuracies of DeepmRNALoc in the cytoplasm, endoplasmic reticulum, extracellular region, mitochondria, and nucleus were 0.895, 0.594, 0.308, 0.944, and 0.865, respectively, demonstrating that it outperforms existing models and techniques.

Sign Language Translator and Teaching System

Abstract: Sign Language is a non-verbal form of communication that is used to interact with the specially abled. These people have communication barriers when it comes to daily interactions with others who don’t understand sign language. Technological advancements and progress in Machine Learning algorithms have made gesture recognition possible. Sign language is one of the oldest and most natural forms of language for communication, but since most people do not know sign language and interpreters are very difficult to come by, we have come up with a real-time method using neural networks for fingerspelling based on American sign language. In our method, the hand is first passed through a filter and after the filter is applied, the hand is passed through a CNN module which predicts the class of the hand gestures. This literature survey analyses studies based on sign language translator and teaching applications. Conclusions drawn from this literature review could help in creating the perfect application that can make daily communication easy for the specially aided. We plan on providing two modules, one wherein the sign language gestures are converted into text and displayed. The second module is made wherein the text entered is converted into sign language and displayed on the User interface.

An improved 3D quantitative structure-activity relationships (QSAR) of molecules with CNN-based partial least squares model

Ligand-based virtual screening plays an important role for cases in which protein structures are not available. Among ligand-based methods, accurate and fast prediction of protein-ligand binding affinity is crucial for reducing computational cost and exploring the chemical search space efficiently. Here we proposed a CNN-based method, termed as L3D-PLS for building the quantitative structure-activity relationships without target structures. In L3D-PLS, a CNN module was designed for extracting the key interaction features from the grids around aligned ligands, and a partial least square (PLS) model fits the binding affinity with the extracted features of the pre-trained CNN module. In 30 publicly available pre-aligned molecular datasets, L3D-PLS outperformed the traditional CoMFA method. This results highlight that L3D-PLS can be useful for lead optimization based on small datasets which is often true in drug discovery compaign.