Branch Convolutional Neural Network Research Articles

Lightweight Classification Network for Pulmonary Tuberculosis Based on CT Images

With the continuous development of medical informatics and digital diagnosis, the classification of tuberculosis cases from computed tomography (CT) images of the lung based on deep learning is an important guiding aid in clinical diagnosis and treatment. Due to its potential application in medical image classification, this task has received extensive research attention. Existing related neural network techniques are still challenging in terms of feature extraction of global contextual information of images and network complexity in achieving image classification. To address these issues, this paper proposes a lightweight medical image classification network based on a combination of Transformer and convolutional neural network (CNN) for the classification of tuberculosis cases from lung CT. The method mainly consists of a fusion of the CNN module and the Transformer module, exploiting the advantages of both in order to accomplish a more accurate classification task. On the one hand, the CNN branch supplements the Transformer branch with basic local feature information in the low level; on the other hand, in the middle and high levels of the model, the CNN branch can also provide the Transformer architecture with different On the other hand, in the middle and high levels of the model, the CNN branches can also provide different local and global feature information to the Transformer architecture to enhance the ability of the model to obtain feature information and improve the accuracy of image classification. A shortcut is used in each module of the network to solve the problem of poor model results due to gradient divergence and to optimise the effectiveness of TB classification. The proposed lightweight model can well solve the problem of long training time in the process of TB classification of lung CT and improve the speed of classification. The proposed method was validated on a computed tomography (CT) image dataset provided by the First Hospital of Lanzhou University. The experimental results show that the proposed lightweight classification network for tuberculosis based on CT medical images of lungs can fully extract the feature information of the input images and obtain high accuracy classification results.

Hierarchical multi-label taxonomic classification of carbonate skeletal grains with deep learning

Recent advances in Artificial Intelligence (AI), particularly the rise of deep learning, are revolutionizing data collection and analysis in many aspects of the Earth Sciences, including paleontology. Rapid digital transformation of paleontological information enables automation of various paleontological tasks using AI technology. One such task is identifying and classifying skeletal grains at different levels of the Linnaean taxonomic hierarchy, both macro- and microscopically. Unfortunately, fossil classification remains largely untouched by AI due to the expertise and time required to generate high-quality, large-scale labeled training data. This task is particularly challenging when part of the compositional analysis of limestone, where three-dimensional objects (i.e., skeletal grains) are observed in a two-dimensional cross-section. It is compounded by the imbalanced data and limited number of images available, approximately four orders smaller than the number necessary to train deep neural networks. Recent efforts to classify such fossil images using deep learning returned a single output/label classification, limiting the information output from the model, and hindering its further applications to replicate paleontologist-level identification. Here, we couple a multi-head deep convolutional neural network architecture called TaxonNet that adopts the Branch Convolutional Neural Networks and advanced image augmentation to overcome these barriers. This model uses prior knowledge of hierarchical category relationships to output multiple hierarchical taxonomic predictions from a single petrographic image. In this study, we developed and evaluated four different strategies of hierarchical classification: (i) forward (coarse to fine); (ii) reverse (fine to coarse); (iii) parallel (coarse and fine simultaneously), and (iv) hybrid. The forward and reverse strategies yield the most accurate prediction of ~95 % accuracy for coarse-level and fine-level recognitions across major fossil types in carbonate petrographic images, including algae, mollusks, and corals. These results represent an important step toward applying AI to advance fossil grain classification and exhibit a promising real-world geological application of a hierarchical network in paleontology, sedimentary petrography, and other image-based geological analysis requiring hierarchical, multilabel output.

Multimodal detection of epilepsy with deep neural networks

Epilepsy constitutes a chronic noncommunicable disease of the brain affecting approximately 50 million people around the world. Most of the existing research initiatives propose methods for detecting and predicting epilepsy, which rely on the extraction of handcrafted features and the train of traditional machine learning classifiers. In this paper, we present two new methods to distinguish healthy, interictal, and ictal cases without the time-consuming procedure of feature extraction. Firstly, we apply the short-time fourier transform (STFT) to the single-channel electroencephalogram (EEG) signals and construct an image consisting of three channels. This image is passed through pretrained models, including AlexNet, DenseNet201, EfficientNet, ResNet18, etc. Secondly, we introduce a multimodal deep neural network. Specifically, we pass each single-channel EEG signal through two branches of convolutional neural networks (CNNs), which can extract low and high frequency features. Also, we apply the short-time fourier transform (STFT) to the EEG signals and create an image consisting of three channels. The image is passed through a pretrained EfficientNet-B7 model. Finally, we employ a gated multimodal unit to control the importance of each modality. We evaluate the performance of the proposed model on five different cases on the EEG database of the University of Bonn and show that our introduced model achieves comparable performance to state-of-the-art approaches.

A Local and Nonlocal Feature Interaction Network for Pansharpening

Pansharpening based on deep learning (DL) has shown great advantages. Most convolutional neural network (CNN)-based methods focus on obtaining local features from multispectral (MS) and panchromatic (PAN) images, but ignore the nonlocal dependence on images. Therefore, Transformer-based methods are introduced to obtain long-range information on images. However, the representational capabilities of features extracted by CNN or Transformer alone are weak. To solve this problem, a local and nonlocal feature interaction network (LNFIN) is proposed in this paper for pansharpening. It comprises Transformer and CNN branches. Furthermore, a feature interaction module (FIM) is proposed to fuse different features and return to the two branches to enhance the representational capability of features. Specifically, a CNN branch consisting of multiscale dense modules (MDMs) is proposed for acquiring local features of the image, and a Transformer branch consisting of pansharpening Transformer modules (PTMs) is introduced for acquiring nonlocal features of the image. In addition, inspired by the PTM, a shift pansharpening Transformer module (SPTM) is proposed for the learning of texture features to further enhance the spatial representation of features. The LNFIN outperforms the state-of-the-art method experimentally on three datasets.

HOB-CNN: Hallucination of occluded branches with a convolutional neural network for 2D fruit trees

Orchard automation has attracted the attention of researchers recently due to the shortage of global labor force. To automate tasks in orchards such as pruning, thinning, and harvesting, a detailed understanding of the tree structure is required. However, occlusions from foliage and fruits can make it challenging to predict the position of occluded trunks and branches. This work proposes a regression-based deep learning model, Hallucination of Occluded Branch Convolutional Neural Network (HOB-CNN), for tree branch position prediction in varying occluded conditions. We formulate tree branch position prediction as a regression problem towards the horizontal locations of the branch along the vertical direction or vice versa. We present comparative experiments on Y-shaped trees with two state-of-the-art baselines, representing common approaches to the problem. Experiments show that HOB-CNN outperform the baselines at predicting branch position and shows robustness against varying levels of occlusion. We further validated HOB-CNN against two different types of 2D trees, and HOB-CNN shows generalization across different trees and robustness under different occluded conditions.

Hierarchical classification based on coarse- to fine-grained knowledge transfer

Hierarchical classification has become one of the most popular research topics because the scale of data has increased exponentially. Top-down hierarchical classification is an effective classification method using the hierarchical class structure as important side information. However, inter-level error propagation is a crucial problem in the top-down strategy. In this paper, we propose a hierarchical classifier based on deep branch convolutional neural networks, which achieves hierarchical classification based on coarse- to fine-grained knowledge transfer. Specifically, we use a deep convolutional neural network to extract image rough and detailed features from shallow and deep networks. We then embed the branch network at different depths of the convolutional network for hierarchical classification. We splice features from the previous branch network to the current branch. It alleviates inter-level error propagation by knowledge transfer from coarse- to fine-grained. Experimental results on four datasets show that the proposed method outperforms nine popular classifiers for hierarchical classification. Especially on the CIFAR10 dataset, our method is about 5% better than the second-best method.

Deep learning feature representation for image matching under large viewpoint and viewing direction change

Feature based image matching has been a research focus in photogrammetry and computer vision for decades, as it is the basis for many applications where multi-view geometry is needed. A typical feature based image matching algorithm contains five steps: feature detection, affine shape estimation, orientation assignment, description and descriptor matching. This paper contains innovative work in different steps of feature matching based on convolutional neural networks (CNN). For the affine shape estimation and orientation assignment, the main contribution of this paper is twofold. First, we define a canonical shape and orientation for each feature. As a consequence, instead of the usual Siamese CNN, only single branch CNNs needs to be employed to learn the affine shape and orientation parameters, which turns the related tasks from supervised to self supervised learning problems, removing the need for known matching relationships between features. Second, the affine shape and orientation are solved simultaneously. To the best of our knowledge, this is the first time these two modules are reported to have been successfully trained together. In addition, for the descriptor learning part, a new weak match finder is suggested to better explore the intra-variance of the appearance of matched features. For any input feature patch, a transformed patch that lies far from the input feature patch in descriptor space is defined as a weak match feature. A weak match finder network is proposed to actively find these weak match features; they are subsequently used in the standard descriptor learning framework. The proposed modules are integrated into an inference pipeline to form the proposed feature matching algorithm. The algorithm is evaluated on standard benchmarks and is used to solve for the parameters of image orientation of aerial oblique images. It is shown that deep learning feature based image matching leads to more registered images, more reconstructed 3D points and a more stable block geometry than conventional methods. The code is available at https://github.com/Childhoo/Chen_Matcher.git.

Directionally separable dilated CNN with hierarchical attention feature fusion for hyperspectral image classification

ABSTRACT In recent years, the convolutional neural network (CNN) plays a vital role in hyperspectral image classification and performs more competitively than many other methods. However, in order to pursue better performance, most of existing CNN-based methods just simply stack rather deep convolutional layers. Although they improve the classification accuracy to a certain extent, they result in plenty of network parameters. In this paper, a light-weighted directionally separable dilated CNN with hierarchical attention feature fusion (DSD-HAFF) is proposed to solve these problems. First, two global dense dilated CNN branches that focus on two spatial directions separately are constructed to extract and reuse spatial information as much as possible. Second, a hierarchical attention feature fusion branch that consists of several coordinate attention blocks (CABs) is constructed. Hierarchical features from two directionally separable dilated CNN branches are adopted as inputs of CABs. In this way, the structure can not only fully incorporate hierarchical features, but also significantly reduce the network parameters. Meanwhile, the hierarchical attention feature fusion branch incorporates features from high-level to low-level in the kernel-number pyramid strategy. Experimental results on three popular benchmark datasets demonstrate that the DSD-HAFF achieves better performance and has a much smaller number of network parameters than the other state-of-the-art methods.

The Efficient Implementation of Face Mask Detection Using MobileNet

Under the condition of the global pandemic, Wearing the mask in the public area is the direct and ordered action to prevent the spreading of the virus. Therefore, developing the machine learning model to detect whether the mask is appropriately wearing is practical and meaningful. This paper unfolded the mask recognition based on the MobileNet model, which belongs to one of the branches of Convolutional Neural Network, CNN for short, in the deep learning field. With the background and tremendous needs of face mask detection, in the beginning, we go through the review of the MobileNet’s history and the practicability of it on demanding functions. After that, we explained in a more detailed way to help gain a better understanding of MobileNet’s structure and how those differences make it outstanding. During the experiment, approximately 9000 images were used as inputs by the model for training and optimizing. The result turns to the less weighted model presenting an acceptable accuracy of 87.96% and 93.5% for testing whether a person wears a mask and whether the mask is covered correctly correspondingly. We also discovered several shortcomings and limitations during the testing process. Those problems and some of the possible further directions of MobileNet will be discussed. As a result, the MobileNet shows the huge probability for further improvement to be equipped into the small-scale but sophisticated utensil.

Spectral and Spatial Feature Fusion for Hyperspectral Image Classification

Compared with traditional images, hyperspectral images (HSI) not only have spatial information, but also have rich spectral information. However, the mainstream hyperspectral image classification (HIC) methods are all based on Convolutional Neural Network (CNN), which has great advantages in extracting spatial features, but it has certain limitations in dealing with spectral continuous sequence information. Therefore Transformer which is good at processing sequences, has also been gradually applied to HIC. Besides, Since HSI are typical three-dimensional structures, we believe that the correlation of the three dimensions is also an important information. So in order to fully extract the spectral spatial information, as well as the correlation of the three dimensions. we propose a spectral and spatial feature fusion module ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., TransCNN) for HIC. TransCNN consists of CNNs and a Transformer. The former is in charge of mining the spatial and spectral information from different dimensions, while the latter not only undertakes the most critical fusion but also captures the deeper relationship characteristics. We transpose the data to extract features and their correlation through three CNNs branches. we believe that these feature maps still have deep spectral information. Therefore, we have embedded them into one-dimensional vectors and use Transformer’s Encoder to extract features. However, some information will be lost when embedding into one-dimensional vectors. Therefore we use Decoder, which has been ignored in the field of vision, to fuse the features before passing Encoder and the features after extracted by Encoders. Two kinds of features are fused by Decoder, and the obtained information is finally input into the classifier for classification. Experimental results on real HSIs show that the proposed architecture can achieve competitive performance compared with the state-of-the-art methods.

Multibranch 1-D CNN Based on Attention Mechanism for the DAB Converter Fault Diagnosis

This article considers the fault diagnosis problem for the dual active bridge (DAB) converter. Given that information from multiple diagnostic signals needs to be combined effectively for fault diagnosis of the DAB converter and diagnostic signals have noise, a multi-branch 1-D convolutional neural network (CNN) based on attention mechanism is proposed. First, the proposed network processes multiple raw 1-D diagnostic signals without extracting features manually. Then, soft thresholding and attention mechanism are combined to acquire denoising thresholds, which adaptively removes channel noise from feature maps. Besides, the attention mechanism is utilized to adaptively obtain importance weights of the feature maps learned by different 1-D CNN branches, which helps to learn the most useful features and fuse the multi-branch features effectively. Finally, the diagnostic results are obtained by inputting the fused features into the Softmax layer. The experimental results indicate that the proposed method with denoise module and feature fusion module improves the diagnostic accuracy effectively. Contrastive experiments show that the proposed method is superior to other fault diagnosis methods.

Convolution Transformer Mixer for Hyperspectral Image Classification

Hyperspectral image (HSI) can provide rich spectral information which can be helpful for accurate classification in many applications. Yet, incorporating spatial information in the classification process can improve the classification accuracy even further. Existing convolutional neural network (CNN) usually only focuses on local features in hyperspectral cubes, whereas the burgeoning vision transformer (ViT) is interested in global features in HSIs. In this letter, we propose a deep aggregated framework for HSI classification called convolution transformer mixer (CTMixer) to combine the advantages of the above two paradigms effectively. A group parallel residual block is firstly applied to capture local spectral-spatial features in the HSI patches. Secondly, a double-branch structure, consisting of the CNN and transformer branches, is developed to capture local-global hyperspectral features. Finally, to achieve an elegant combination of CNN and ViT, a novel local-global multi-head self-attention mechanism is proposed by introducing convolution operations in the multi-head self-attention mechanism to further improve the classification accuracy. Extensive experiments demonstrate that the CTMixer achieves competitive classification results on several common HSI datasets compared with other state-of-the-art networks. The source code for this work will be available at https://github.com/ZJier/CTMixer.

Sensor-Based Human Activity Recognition Using Adaptive Class Hierarchy.

In sensor-based human activity recognition, many methods based on convolutional neural networks (CNNs) have been proposed. In the typical CNN-based activity recognition model, each class is treated independently of others. However, actual activity classes often have hierarchical relationships. It is important to consider an activity recognition model that uses the hierarchical relationship among classes to improve recognition performance. In image recognition, branch CNNs (B-CNNs) have been proposed for classification using class hierarchies. B-CNNs can easily perform classification using hand-crafted class hierarchies, but it is difficult to manually design an appropriate class hierarchy when the number of classes is large or there is little prior knowledge. Therefore, in our study, we propose a class hierarchy-adaptive B-CNN, which adds a method to the B-CNN for automatically constructing class hierarchies. Our method constructs the class hierarchy from training data automatically to effectively train the B-CNN without prior knowledge. We evaluated our method on several benchmark datasets for activity recognition. As a result, our method outperformed standard CNN models without considering the hierarchical relationship among classes. In addition, we confirmed that our method has performance comparable to a B-CNN model with a class hierarchy based on human prior knowledge.

Hierarchical diagnosis of bearing faults using branch convolutional neural network considering noise interference and variable working conditions

Rolling bearings are prone to malfunction due to the complexity of rotating equipment and the serious operational environment. Intelligent fault diagnosis based on convolutional neural networks (CNNs) has been an effective and efficient tool to ensure reliable and safe operation of rolling bearings. However, environmental noise and variable working conditions affect the data. To accurately detect, locate, and identify bearing faults, we propose a hierarchical branch CNN (HB-CNN) scheme that considers polluted data and variations in the working environment conditions. In our approach, the hierarchical structure of the bearing fault is constructed initially, with three-level labels indicating fault detection, fault isolation, and fault identification. Then, a one-dimensional (1-D) CNN is introduced as a basic building block by stacking small convolutional kernels. Finally, the hierarchical structure is combined with the natural stratification of the network, and the HB-CNN is generated by incorporating the branch structure into the 1-D CNN. To demonstrate the superiority of the HB-CNN, we compare the performance of our HB-CNN scheme with several salient deep learning models on two benchmark datasets. The results demonstrate that the proposed model successfully achieved hierarchical diagnosis of bearing faults and exhibited superior robustness against noise interference and changing working conditions.

BCNN-Methylpred: Feature-Based Prediction of RNA Sequence Modification Using Branch Convolutional Neural Network.

RNA modification is vital to various cellular and biological processes. Among the existing RNA modifications, N6-methyladenosine (m6A) is considered the most important modification owing to its involvement in many biological processes. The prediction of m6A sites is crucial because it can provide a better understanding of their functional mechanisms. In this regard, although experimental methods are useful, they are time consuming. Previously, researchers have attempted to predict m6A sites using computational methods to overcome the limitations of experimental methods. Some of these approaches are based on classical machine-learning techniques that rely on handcrafted features and require domain knowledge, whereas other methods are based on deep learning. However, both methods lack robustness and yield low accuracy. Hence, we develop a branch-based convolutional neural network and a novel RNA sequence representation. The proposed network automatically extracts features from each branch of the designated inputs. Subsequently, these features are concatenated in the feature space to predict the m6A sites. Finally, we conduct experiments using four different species. The proposed approach outperforms existing state-of-the-art methods, achieving accuracies of 94.91%, 94.28%, 88.46%, and 94.8% for the H. sapiens, M. musculus, S. cerevisiae, and A. thaliana datasets, respectively.

Hyperspectral Remote Sensing Images Deep Feature Extraction Based on Mixed Feature and Convolutional Neural Networks

To achieve effective deep fusion features for improving the classification accuracy of hyperspectral remote sensing images (HRSIs), a pixel frequency spectrum feature is presented and introduced to convolutional neural networks (CNNs). Firstly, the fast Fourier transform is performed on each spectral pixel to obtain the amplitude spectrum, i.e., the pixel frequency spectrum feature. Then, the obtained pixel frequency spectrum is combined with the spectral pixel to form a mixed feature, i.e., spectral and frequency spectrum mixed feature (SFMF). Several multi-branch CNNs fed with pixel frequency spectrum, SFMF, spectral pixel, and spatial features are designed for extracting deep fusion features. A pre-learning strategy, i.e., basic single branch CNNs are used to pre-learn the weights of a multi-branch CNN, is also presented for improving the network convergence speed and avoiding the network from getting into a locally optimal solution to a certain extent. And after reducing the dimensionality of SFMF by principal component analysis (PCA), a 3-dimensionality (3-D) CNN is also designed to further extract the joint spatial-SFMF feature. The experimental results of three real HRSIs show that adding the presented frequency spectrum feature into CNNs can achieve better recognition results, which in turn proves that the presented multi-branch CNNs can obtain the deep fusion features with more discriminant information.

Real-time Semantic Segmentation with Context Aggregation Network

With the increasing demand of autonomous systems, pixelwise semantic segmentation for visual scene understanding needs to be not only accurate but also efficient for potential real-time applications. In this paper, we propose Context Aggregation Network, a dual branch convolutional neural network, with significantly lower computational costs as compared to the state-of-the-art, while maintaining a competitive prediction accuracy. Building upon the existing dual branch architectures for high-speed semantic segmentation, we design a high resolution branch for effective spatial detailing and a context branch with light-weight versions of global aggregation and local distribution blocks, potent to capture both long-range and local contextual dependencies required for accurate semantic segmentation, with low computational overheads. We evaluate our method on two semantic segmentation datasets, namely Cityscapes dataset and UAVid dataset. For Cityscapes test set, our model achieves state-of-the-art results with mIOU of 75.9%, at 76 FPS on an NVIDIA RTX 2080Ti and 8 FPS on a Jetson Xavier NX. With regards to UAVid dataset, our proposed network achieves mIOU score of 63.5% with high execution speed (15 FPS).

4D deep image prior: dynamic PET image denoising using an unsupervised four-dimensional branch convolutional neural network

Although convolutional neural networks (CNNs) demonstrate the superior performance in denoising positron emission tomography (PET) images, a supervised training of the CNN requires a pair of large, high-quality PET image datasets. As an unsupervised learning method, a deep image prior (DIP) has recently been proposed; it can perform denoising with only the target image. In this study, we propose an innovative procedure for the DIP approach with a four-dimensional (4D) branch CNN architecture in end-to-end training to denoise dynamic PET images. Our proposed 4D CNN architecture can be applied to end-to-end dynamic PET image denoising by introducing a feature extractor and a reconstruction branch for each time frame of the dynamic PET image. In the proposed DIP method, it is not necessary to prepare high-quality and large patient-related PET images. Instead, a subject’s own static PET image is used as additional information, dynamic PET images are treated as training labels, and denoised dynamic PET images are obtained from the CNN outputs. Both simulation with [18F]fluoro-2-deoxy-D-glucose (FDG) and preclinical data with [18F]FDG and [11C]raclopride were used to evaluate the proposed framework. The results showed that our 4D DIP framework quantitatively and qualitatively outperformed 3D DIP and other unsupervised denoising methods. The proposed 4D DIP framework thus provides a promising procedure for dynamic PET image denoising.

Pneumonia Recognition Based on Convolutional Neural Network Feature Map Fusion

Pneumonia recognition has important research significance in computer-aided diagnosis, and there is a problem of low accuracy for pneumonia recognition. In this paper, an improved network is based on the convolutional neural network AlexNet, the AlexNet_Branch network. The AlexNet_Branch network adds a parallel branch convolutional neural network to AlexNet, and it connects AlexNet and the branch convolutional neural network at the fully connected layer. During training, the same image is simultaneously obtained by AlexNet and the branch convolutional neural network to obtain different feature maps, and then the feature maps are merged together at the fully connected layer to improve the accuracy of recognition. Through design experiments, different AlexNet_Branch networks composed of different layers of branch convolutional neural networks were built, and the network was trained and tested on the chest X-ray image set respectively. The results show that the addition of a branch convolutional neural network greatly improves the accuracy of pneumonia recognition, and the AlexNet_Branch network test accuracy consisting of a 16-layer branch convolutional neural network is 98.01%.

Dual branch convolutional neural network for copy move forgery detection

Dual branch convolutional neural network for copy move forgery detection