Fully Convolutional Neural Network Research Articles

Fully Convolutional Neural Network Structure and Its Loss Function for Image Classification

The overall structure of a convolutional neural network classifier includes multiple convolutional layers and one or more linear layers. Due to the fully connected characteristics of linear layer networks, there are usually many parameters, which may easily lead to local optimization and over-fitting of the model. Therefore, its indispensability in Convolutional Neural Networks (CNNs) classifier is questionable. At the same time, the excessive number of output features of the convolutional layer will also lead to the curse of dimensionality. After verifying the redundancy characteristics of the final activation function, linear layer, and the number of channels by eigen-values analysis of latent features and experiments, we propose a Fully Convolutional Neural Network (FCNN) classifier architecture, which removes the linear layers and the corresponding activation functions from the conventional CNN classifiers. By modifying the number of output channels of the last convolutional layer, the network can be trained directly through Softmax Loss. Furthermore, a softmax-free loss (POD Loss) based on Predefined Optimal-Distribution of latent features is adopted instead of Softmax Loss to obtain better recognition performance. Experiments on multiple commonly used datasets and typical networks have proved that the network structure not only reduces the amount of parameters and calculation, but also improves the recognition rate. In the meanwhile, the adoption of POD Loss further improves the classification accuracy and robustness of the model, making them play a better synergy. Code is available in <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/TianYuZu/Fully-Convolutional-Network</uri> .

A Novel Activation Function for Brain Tumor Segmentation using V-NET Approach

This work emphasizes on automatic brain tumor segmentation from three dimensional magnetic resonance imaging (3D-MRI) scan images. We have used fully convolutional neural networks (FCNN) for extracting whole tumor. The proposed architecture is based on V-Net architecture. We have developed a new activation function for training the network. ReLU is used and in experiment 2, proposed activation function is used. We have conducted two experiments for brain tumor segmentation by varying the activation functions. In experiment 1, we have also used dice loss as loss function. The proposed method is trained and tested using BraTs 2021 training dataset which contains 2000 volumes of 3D-MRI scans. The proposed method’s performance is closer to the manually segmented images by experienced neurologists available with BraTs 2021 datasets. We have obtained a mean dice score of whole tumor as 99% while using proposed activation function which is higher when compared to the existing methods. We conclude that our novel activation function proposed activation function produced enhanced accuracy and dice score when compared with ReLU activation function.

A Bayesian Deep Image Prior Downscaling Approach for High-Resolution Soil Moisture Estimation

Soil moisture (SM) estimation is a critical part of environmental and agricultural monitoring, with satellite-based microwave remote sensing being the main SM source. However, the limited spatial resolution of most current remote sensing SM products reduces their utility for many applications such as evapotranspiration modeling and agriculture management. To address this issue, we propose a Bayesian deep image prior (BDIP) downscaling approach for producing high-resolution satellite SM estimates. More specifically, the high-resolution soil moisture estimation problem is formulated as a maximum a posteriori (MAP) problem, and solved via an encoder-decoder neural network architecture comprising of a deep fully convolutional neural network (FCNN) encoder for modeling the prior spatial correlation distribution of the underlying high-resolution SM variables, and a forward model characterizing the SM degeneration process for modeling the data likelihood. As such, the proposed BDIP approach provides a statistical framework that integrates deep learning with forward modelling in a coherent manner for combining different sources of information, i.e., the knowledge in forward model, the spatial correlation prior, and the remote sensing data and products. Experiments on the downscaling of Soil Moisture Active Passive (SMAP) SM products using the Moderate Resolution Imaging Spectroradiometer (MODIS) products show that SM maps estimated using the proposed method provide greater spatial detail information than other existing downscaling methods, with the SM estimates very close to in-situ measurements.

Training Deep Filters for Physical-Layer Frame Synchronization

In this paper we demonstrate the application of Fully Convolutional Neural Network (FCN) for Frame Synchronization (FS) in bursty single carrier transmissions, commonly used in wireless sensor networks and Internet of Things (IoT) applications. Our approach shows greatly improved performance compared to noncoherent correlation-based methods under carrier phase and frequency offsets, especially for shorter preambles. Using a fully convolutional architecture allows the training of a deep filter, which we believe is more suited to signal processing tasks than more commonly used deep learning architectures with fully connected layers. In terms of deployment within a wider communications system, it could be treated similarly to a typical signal processing filter, which means it can be deployed to inputs of arbitrary length. Additionally, because the proposed model is composed only of convolutional layers, the entire model benefits from the weight sharing property of convolutional filters, and results in a greatly reduced memory footprint compared to that of similar models containing fully connected layers.

Non-Rigid Registration Algorithm of Lung Computed Tomography Image Based on Multi-Scale Parallel Fully Convolutional Neural Network

Non-Rigid Registration Algorithm of Lung Computed Tomography Image Based on Multi-Scale Parallel Fully Convolutional Neural Network

Conv-MLP: A Convolution and MLP Mixed Model for Multimodal Face Anti-Spoofing

Local features contain crucial clues for face anti-spoofing. Convolutional neural networks (CNNs) are powerful in extracting local features, but the intrinsic inductive bias of CNNs limits the ability to capture long-range dependencies. This paper aims to develop a simple yet effective framework that is versatile in extracting both local information and long-range dependencies for face anti-spoofing. To this end, we propose a novel architecture, namely Conv-MLP, which incorporates local patch convolution with global multi-layer perceptrons (MLP). Conv-MLP breaks the inductive bias limitation of traditional full CNNs and can be expected to better exploit long-range dependencies. Furthermore, we design a new loss specifically for the face anti-spoofing task, namely moat loss. The moat loss benefits discriminative representations learning and can improve the generalization capability on unseen presentation attacks. In this work, multi-modal data are directly fused at the signal level to extract complementary features. Extensive experiments on single and multi-modal datasets demonstrate that Conv-MLP outperforms existing state-of-the-art methods while being more computationally efficient. The code is available at https://github.com/WeihangWANG/Conv-MLP.

Siamese Network Object Tracking Algorithm Combining Attention Mechanism and Correlation Filter Theory

Aiming to solve the problem of tracking drift during movement, which was caused by the lack of discriminability of the feature information and the failure of a fixed template to adapt to the change of object appearance, the paper proposes an object tracking algorithm combining attention mechanism and correlation filter theory based on the framework of full convolutional Siamese neural networks. Firstly, the apparent information is processed by using the attention mechanism thought, where the object and search area features are optimized according to the spatial attention and channel attention module. At the same time, the cross-attention module is introduced to process the template branch and search area branch, respectively, which makes full use of the diversified context information of the search area. Then, the background perception correlation filter model with scale adaptation and learning rate adjustment is adopted into the model construction, using as a layer in the network model to realize the object template update. Finally, the optimal object location is determined according to the confidence map with similarity calculation. Experimental results show that the designed method in the paper can promote the object tracking performance under various challenging environments effectively; the success rate increases by 16.2%, and the accuracy rate increases by 16%.

Predicting individual task contrasts from resting‐state functional connectivity using a surface‐based convolutional network

Task-based and resting-state represent the two most common experimental paradigms of functional neuroimaging. While resting-state offers a flexible and scalable approach for characterizing brain function, task-based techniques provide superior localization. In this paper, we build on recent deep learning methods to create a model that predicts task-based contrast maps from resting-state fMRI scans. Specifically, we propose BrainSurfCNN, a surface-based fully-convolutional neural network model that works with a representation of the brain’s cortical sheet. BrainSurfCNN achieves exceptional predictive accuracy on independent test data from the Human Connectome Project, which is on par with the repeat reliability of the measured subject-level contrast maps. Conversely, our analyses reveal that a previously published benchmark is no better than group-average contrast maps. Finally, we demonstrate that BrainSurfCNN can generalize remarkably well to novel domains with limited training data.

Multi-scale brain tumor segmentation combined with deep supervision.

Fully convolutional neural networks (FCNNs) have achieved good performance in the field of medical image segmentation. FCNNs that use multimodal images and multi-scale feature extraction have higher accuracy for brain tumor segmentation. Therefore, we have made some improvements to U-Net for fully automated segmentation of gliomas using multimodal images. And we named it multi-scale dilate network with deep supervision (MSD-Net). MSD-Net is a symmetrical structure composed of a down-sampling process and an up-sampling process. In the down-sampling process, we use the multi-scale feature extraction block (ME) to extract multi-scale features and focus on primary features. Unlike other methods, ME consists of dilate convolution and standard convolution. Dilate convolution extracts multi-scale informations and standard convolution merges features of different scales. Hence, the output of the ME contains local information and global information. During the up-sampling process, we add a deep supervision block (DSB), which can shorten the length of back-propagation. In this paper, we pay more attention to the importance of shallow features for feature restoration. Our network validated in the BraTS17's validation dataset. The DSC scores of MSD-Net for complete tumor, tumor core and enhancing tumor were 0.88, 0.81 and 0.78, respectively, which outperforms most networks. This study shows that ME enhances the feature extraction ability of the network and improves the accuracy of segmentation results. DSB speeds up the convergence of the network. In addition, we should also pay attention to the contribution of shallow features to feature restoration.

Development of a Fully Convolutional Neural Network to Derive Surf-Zone Bathymetry from Close-Range Imagery of Waves in Duck, NC

Timely observations of nearshore water depths are important for a variety of coastal research and management topics, yet this information is expensive to collect using in situ survey methods. Remote methods to estimate bathymetry from imagery include using either ratios of multi-spectral reflectance bands or inversions from wave processes. Multi-spectral methods work best in waters with low turbidity, and wave-speed-based methods work best when wave breaking is minimal. In this work, we build on the wave-based inversion approaches, by exploring the use of a fully convolutional neural network (FCNN) to infer nearshore bathymetry from imagery of the sea surface and local wave statistics. We apply transfer learning to adapt a CNN originally trained on synthetic imagery generated from a Boussinesq numerical wave model to utilize tower-based imagery collected in Duck, North Carolina, at the U.S. Army Engineer Research and Development Center’s Field Research Facility. We train the model on sea-surface imagery, wave conditions, and associated surveyed bathymetry using three years of observations, including times with significant wave breaking in the surf zone. This is the first time, to the authors’ knowledge, an FCNN has been successfully applied to infer bathymetry from surf-zone sea-surface imagery. Model results from a separate one-year test period generally show good agreement with survey-derived bathymetry (0.37 m root-mean-squared error, with a max depth of 6.7 m) under diverse wave conditions with wave heights up to 3.5 m. Bathymetry results quantify nearshore bathymetric evolution including bar migration and transitions between single- and double-barred morphologies. We observe that bathymetry estimates are most accurate when time-averaged input images feature visible wave breaking and/or individual images display wave crests. An investigation of activation maps, which show neuron activity on a layer-by-layer basis, suggests that the model is responsive to visible coherent wave structures in the input images.

Identification of Barrett's esophagus in endoscopic images using deep learning

BackgroundDevelopment of a deep learning method to identify Barrett's esophagus (BE) scopes in endoscopic images.Methods443 endoscopic images from 187 patients of BE were included in this study. The gastroesophageal junction (GEJ) and squamous-columnar junction (SCJ) of BE were manually annotated in endoscopic images by experts. Fully convolutional neural networks (FCN) were developed to automatically identify the BE scopes in endoscopic images. The networks were trained and evaluated in two separate image sets. The performance of segmentation was evaluated by intersection over union (IOU).ResultsThe deep learning method was proved to be satisfying in the automated identification of BE in endoscopic images. The values of the IOU were 0.56 (GEJ) and 0.82 (SCJ), respectively.ConclusionsDeep learning algorithm is promising with accuracies of concordance with manual human assessment in segmentation of the BE scope in endoscopic images. This automated recognition method helps clinicians to locate and recognize the scopes of BE in endoscopic examinations.

Automatic location scheme of anatomical landmarks in 3D head MRI based on the scale attention hourglass network

Background and ObjectiveAn anatomical landmark is biologically meaningful point in medical images and often used for medical image registration. The purpose of this study is to automatically locate anatomical landmarks from 3D medical images. MethodsA two-step automatic location scheme of anatomical landmarks in 3D medical image was designed in this study. In the first step, the full convolutional neural network was used for slice detection from a 3D medical image. In the second step, the scale attention hourglass network was used for landmark location in the detected slice and could overcome the difficulty of similar anatomical structures and different image parameters. This method was implemented and tested on four stable anatomical landmarks in 3D head MRI. ResultsA total of 500 and 300 3D head volumes were used for training and testing, respectively. Results showed that the slice detection accuracy reached 85.7% and that the maximum location error was less than one slice. The average accuracy of the four anatomical landmarks in the detected slice reached 87.2%, and the spatial distance was 2.4 ± 2.4, which obtained better performance compared with hourglass network and feature pyramid networks. ConclusionsThis method can be useful for locating anatomical landmarks in 3D head MRI and provides technical support for medical image registration and big data analysis.

A Cognitive Rail Track Breakage Detection System Using Artificial Neural Network

Abstract Rail track breakages represent broken structures consisting of rail track on the railroad. The traditional methods for detecting this problem have proven unproductive. The safe operation of rail transportation needs to be frequently monitored because of the level of trust people have in it and to ensure adequate maintenance strategy and protection of human lives and properties. This paper presents an automatic deep learning method using an improved fully Convolutional Neural Network (FCN) model based on U-Net architecture to detect and segment cracks on rail track images. An approach to evaluating the extent of damage on rail tracks is also proposed to aid efficient rail track maintenance. The model performance is evaluated using precision, recall, F1-Score, and Mean Intersection over Union (MIoU). The results obtained from the extensive analysis show U-Net capability to extract meaningful features for accurate crack detection and segmentation.

Characterizing brain age in the Alzheimer's disease connectome project using a deep neural network pre‐trained on the UK Biobank

AbstractBackgroundOverall age of an individual is commonly measured in chronological time. With the advent of in vivo imaging and reliable predictive modeling, a finer gradation of the age into many internal biological scales such as cellular, tissue, organ and cognitive aging etc. can be estimated. Accurate estimation of the age of the in vivo brain can offer insights into its differential aging patterns and accelerated excess aging due to the burden of disease processes eventually leading to dementia. Modern machine learning algorithms have allowed for accurate estimation of “brain age” using MRI data. The purpose of this study was to evaluate the applicability of a 3D fully‐convolutional neural network (FCNN), pre‐trained on a large dataset of over ten thousand T1‐weighted (T1‐w) MR images from the UK Biobank to data from the Alzheimer’s disease connectome project.MethodT1‐w data acquired from 152 participants were minimally pre‐processed (brain masking, B1 bias reduction, and linear registration to MNI) and input to the pre‐trained FCNN to predict their brain age. A linear regression between calendar age and brain age gap was used to correct the bias in each of the predicted brain age.ResultThe brain age predictions before and after bias correction, mean and spread of the calendar ages, mean absolute error of prediction for each group and sex are shown in Fig. 1. The quantiles of the excess brain aging are shown in Fig. 2.ConclusionWith a minimal pre‐processing of T1w‐MRI, the UK Biobank pre‐trained deep neural network (DNN), along with a linear bias correction, provided an estimate of the age of the brain in ADCP with excellent approximation. The analysis revealed that the median gap between the brain age and calendar age is about a year in the AD group, and about half‐a‐year in the MCI group and none in the CU group, when averaged over the sexes. This provides preliminary evidence of accelerated aging of the brain in the cognitively impaired groups (MCI/AD). Future analysis involves examining excess brain and cognitive aging due to amyloid and tau burden. Future work entails developing DNNs to improve generalization, standardization, and individualized precision of such predictive models.

A whole-process interpretable and multi-modal deep reinforcement learning for diagnosis and analysis of Alzheimer’s disease ∗ ∗ Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or

Objective. Alzheimer’s disease (AD), a common disease of the elderly with unknown etiology, has been adversely affecting many people, especially with the aging of the population and the younger trend of this disease. Current artificial intelligence (AI) methods based on individual information or magnetic resonance imaging (MRI) can solve the problem of diagnostic sensitivity and specificity, but still face the challenges of interpretability and clinical feasibility. In this study, we propose an interpretable multimodal deep reinforcement learning model for inferring pathological features and the diagnosis of AD. Approach. First, for better clinical feasibility, the compressed-sensing MRI image is reconstructed using an interpretable deep reinforcement learning model. Then, the reconstructed MRI is input into the full convolution neural network to generate a pixel-level disease probability risk map (DPM) of the whole brain for AD. The DPM of important brain regions and individual information are then input into the attention-based fully deep neural network to obtain the diagnosis results and analyze the biomarkers. We used 1349 multi-center samples to construct and test the model. Main results. Finally, the model obtained 99.6% ± 0.2%, 97.9% ± 0.2%, and 96.1% ± 0.3% area under curve in ADNI, AIBL and NACC, respectively. The model also provides an effective analysis of multimodal pathology, predicts the imaging biomarkers in MRI and the weight of each individual item of information. In this study, a deep reinforcement learning model was designed, which can not only accurately diagnose AD, but analyze potential biomarkers. Significance. In this study, a deep reinforcement learning model was designed. The model builds a bridge between clinical practice and AI diagnosis and provides a viewpoint for the interpretability of AI technology.

Automated Detection of Brain Tumor through Magnetic Resonance Images Using Convolutional Neural Network.

Brain tumor is a fatal disease, caused by the growth of abnormal cells in the brain tissues. Therefore, early and accurate detection of this disease can save patient's life. This paper proposes a novel framework for the detection of brain tumor using magnetic resonance (MR) images. The framework is based on the fully convolutional neural network (FCNN) and transfer learning techniques. The proposed framework has five stages which are preprocessing, skull stripping, CNN-based tumor segmentation, postprocessing, and transfer learning-based brain tumor binary classification. In preprocessing, the MR images are filtered to eliminate the noise and are improve the contrast. For segmentation of brain tumor images, the proposed CNN architecture is used, and for postprocessing, the global threshold technique is utilized to eliminate small nontumor regions that enhanced segmentation results. In classification, GoogleNet model is employed on three publicly available datasets. The experimental results depict that the proposed method is achieved average accuracies of 96.50%, 97.50%, and 98% for segmentation and 96.49%, 97.31%, and 98.79% for classification of brain tumor on BRATS2018, BRATS2019, and BRATS2020 datasets, respectively. The outcomes demonstrate that the proposed framework is effective and efficient that attained high performance on BRATS2020 dataset than the other two datasets. According to the experimentation results, the proposed framework outperforms other recent studies in the literature. In addition, this research will uphold doctors and clinicians for automatic diagnosis of brain tumor disease.

Random Subspace Ensembles of Fully Convolutional Network for Time Series Classification

Time series classification (TSC) task is one of the most significant topics in data mining. Among all methods for this issue, the deep-learning-based shows superior performance for its good adaption to raw series data and automatic extraction of features. However, rare eyes are kept on composing ensembles of these superior individual classifiers to achieve further breakthroughs. The existing deep learning ensembles NNE did a heavy work of combining 60 individuals but did not maximize the deserving improvement, since it merely pays attention to the diversity of individuals but ignores their accuracy. In this paper, we propose to construct an ensemble of Full Convolutional Neural Networks (FCN) by Random Subspace Method (RSM), named RSM-FCN. FCN is a simple but outstanding individual classifier and RSM is suitable for high dimensional data such as time series, but there are few instances. Thus, the combination of these strengths, RSM-FCN provides a highly cost-effective approach to yield promising results. Experiments on the UCR dataset demonstrate the effectiveness and reasonability of the proposed method.

Monitoring hand hygiene with commercial gas sensors: A pattern recognition approach

Insufficient hand hygiene compliance of Health care workers is a widespread problem in medicine leading to several preventable infections each year. Hence, World Health Organization (WHO) presumes, that up to 1 in 10 patients acquires a Health care associated infection (HCAI) while receiving treatment in health care facilities. Apart from hand hygiene adherence and compliance, a correct hand disinfection technique regarding the amount of hand rub and exposure time is crucial for hand hygiene efficacy. Since alcohol-based hand rub is prevalent in the medical sector, the present study presents a new method for anonymous, electronic hand hygiene monitoring with commercial Ethanol gas sensors. In contrast to previous attempts of electronic monitoring, which mostly merely count the amount of hand hygiene actions, the present approach exploits different algorithms of Machine Learning to assess and categorize the hand disinfection technique into correct and incorrect. To demonstrate functionality of the mentioned approach, labeled hand disinfection data was collected in a pilot study over the period of one year with two different sensor setups. In detail, on the one hand a commercial portable gas sensor tag is used, which can be directly attached to the surgical gown. On the other hand, the performance of several commercial gas sensors is investigated, which are stationarily mounted next to the hand rub dispenser station. The present study follows three subgoals: Firstly, suitability of the stationary vs. portable setup shall be assessed. Secondly, the adequacy of different commercial gas sensor types is evaluated and thirdly the performance of possible algorithms for time series classification of the given sensor response is investigated, i.e. Support Vector Machines (SVM) as well as different Deep Learning algorithms, such as Multilayer Perceptron (MLP), Fully Convolutional Neural Networks (FCN) and a Bidirectional Long-Short-Term Memory network (Bi-LSTM). It could be found that the stationary setup is significantly superior compared to the portable setup. For most of the sensors within this stationary setup an accuracy score between 85% and 87% could be obtained for classification of correctly and incorrectly performed hand disinfections.

3D spatial priors for semi-supervised organ segmentation with deep convolutional neural networks

Fully Convolutional neural Networks (FCNs) are the most popular models for medical image segmentation. However, they do not explicitly integrate spatial organ positions, which can be crucial for proper labeling in challenging contexts. In this work, we propose a method that combines a model representing prior probabilities of an organ position in 3D with visual FCN predictions by means of a generalized prior-driven prediction function. The prior is also used in a self-labeling process to handle low-data regimes, in order to improve the quality of the pseudo-label selection. Experiments carried out on CT scans from the public TCIA pancreas segmentation dataset reveal that the resulting STIPPLE model can significantly increase performances compared to the FCN baseline, especially with few training images. We also show that STIPPLE outperforms state-of-the-art semi-supervised segmentation methods by leveraging the spatial prior information. STIPPLE provides a segmentation method effective with few labeled examples, which is crucial in the medical domain. It offers an intuitive way to incorporate absolute position information by mimicking expert annotators.

Semantic Segmentation of Remote Sensing Image Based on GAN and FCN Network Model

Accurate remote sensing image segmentation can guide human activities well, but current image semantic segmentation methods cannot meet the high-precision semantic recognition requirements of complex images. In order to further improve the accuracy of remote sensing image semantic segmentation, this paper proposes a new image semantic segmentation method based on Generative Adversarial Network (GAN) and Fully Convolutional Neural Network (FCN). This method constructs a deep semantic segmentation network based on FCN, which can enhance the receptive field of the model. GAN is integrated into FCN semantic segmentation network to synthesize the global image feature information and then accurately segment the complex remote sensing image. Through experiments on a variety of datasets, it can be seen that the proposed method can meet the high-efficiency requirements of complex image semantic segmentation and has good semantic segmentation capabilities.