Fully Convolutional Network Architecture Research Articles

Stochastic gradient descent optimisation for convolutional neural network for medical image segmentation.

In accordance with the inability of various hair artefacts subjected to dermoscopic medical images, undergoing illumination challenges that include chest-Xray featuring conditions of imaging acquisi-tion situations built with clinical segmentation. The study proposed a novel deep-convolutional neural network (CNN)-integrated methodology for applying medical image segmentation upon chest-Xray and dermoscopic clinical images. The study develops a novel technique of segmenting medical images merged with CNNs with an architectural comparison that incorporates neural networks of U-net and fully convolutional networks (FCN) schemas with loss functions associated with Jaccard distance and Binary-cross entropy under optimised stochastic gradient descent + Nesterov practices. Digital image over clinical approach significantly built the diagnosis and determination of the best treatment for a patient's condition. Even though medical digital images are subjected to varied components clarified with the effect of noise, quality, disturbance, and precision depending on the enhanced version of images segmented with the optimised process. Ultimately, the threshold technique has been employed for the output reached under the pre- and post-processing stages to contrast the image technically being developed. The data source applied is well-known in PH2 Database for Melanoma lesion segmentation and chest X-ray images since it has variations in hair artefacts and illumination. Experiment outcomes outperform other U-net and FCN architectures of CNNs. The predictions produced from the model on test images were post-processed using the threshold technique to remove the blurry boundaries around the predicted lesions. Experimental results proved that the present model has better efficiency than the existing one, such as U-net and FCN, based on the image segmented in terms of sensitivity = 0.9913, accuracy = 0.9883, and dice coefficient = 0.0246.

Abdominal fat quantification using convolutional networks

ObjectivesTo present software for automated adipose tissue quantification of abdominal magnetic resonance imaging (MRI) data using fully convolutional networks (FCN) and to evaluate its overall performance—accuracy, reliability, processing effort, and time—in comparison with an interactive reference method.Materials and methodsSingle-center data of patients with obesity were analyzed retrospectively with institutional review board approval. Ground truth for subcutaneous (SAT) and visceral adipose tissue (VAT) segmentation was provided by semiautomated region-of-interest (ROI) histogram thresholding of 331 full abdominal image series. Automated analyses were implemented using UNet-based FCN architectures and data augmentation techniques. Cross-validation was performed on hold-out data using standard similarity and error measures.ResultsThe FCN models reached Dice coefficients of up to 0.954 for SAT and 0.889 for VAT segmentation during cross-validation. Volumetric SAT (VAT) assessment resulted in a Pearson correlation coefficient of 0.999 (0.997), relative bias of 0.7% (0.8%), and standard deviation of 1.2% (3.1%). Intraclass correlation (coefficient of variation) within the same cohort was 0.999 (1.4%) for SAT and 0.996 (3.1%) for VAT.ConclusionThe presented methods for automated adipose-tissue quantification showed substantial improvements over common semiautomated approaches (no reader dependence, less effort) and thus provide a promising option for adipose tissue quantification.Clinical relevance statementDeep learning techniques will likely enable image-based body composition analyses on a routine basis. The presented fully convolutional network models are well suited for full abdominopelvic adipose tissue quantification in patients with obesity.Key Points• This work compared the performance of different deep-learning approaches for adipose tissue quantification in patients with obesity.• Supervised deep learning–based methods using fully convolutional networks were suited best.• Measures of accuracy were equal to or better than the operator-driven approach.

Multi-Scale Fully Convolutional Network-Based Semantic Segmentation for Mobile Robot Navigation

In computer vision and mobile robotics, autonomous navigation is crucial. It enables the robot to navigate its environment, which consists primarily of obstacles and moving objects. Robot navigation employing impediment detections, such as walls and pillars, is not only essential but also challenging due to real-world complications. This study provides a real-time solution to the problem of obtaining hallway scenes from an exclusive image. The authors predict a dense scene using a multi-scale fully convolutional network (FCN). The output is an image with pixel-by-pixel predictions that can be used for various navigation strategies. In addition, a method for comparing the computational cost and precision of various FCN architectures using VGG-16 is introduced. The binary semantic segmentation and optimal obstacle avoidance navigation of autonomous mobile robots are two areas in which our method outperforms the methods of competing works. The authors successfully apply perspective correction to the segmented image in order to construct the frontal view of the general area, which identifies the available moving area. The optimal obstacle avoidance strategy is comprised primarily of collision-free path planning, reasonable processing time, and smooth steering with low steering angle changes.

A Novel Transformer Based Semantic Segmentation Scheme for Fine-Resolution Remote Sensing Images

The fully convolutional network (FCN) with an encoder-decoder architecture has been the standard paradigm for semantic segmentation. The encoder-decoder architecture utilizes an encoder to capture multilevel feature maps, which are incorporated into the final prediction by a decoder. As the context is crucial for precise segmentation, tremendous effort has been made to extract such information in an intelligent fashion, including employing dilated/atrous convolutions or inserting attention modules. However, these endeavors are all based on the FCN architecture with ResNet or other backbones, which cannot fully exploit the context from the theoretical concept. By contrast, we introduce the Swin Transformer as the backbone to extract the context information and design a novel decoder of densely connected feature aggregation module (DCFAM) to restore the resolution and produce the segmentation map. The experimental results on two remotely sensed semantic segmentation datasets demonstrate the effectiveness of the proposed scheme.

Medical Image Segmentation Using Transformer Networks.

Deep learning models represent the state of the art in medical image segmentation. Most of these models are fully-convolutional networks (FCNs), namely each layer processes the output of the preceding layer with convolution operations. The convolution operation enjoys several important properties such as sparse interactions, parameter sharing, and translation equivariance. Because of these properties, FCNs possess a strong and useful inductive bias for image modeling and analysis. However, they also have certain important shortcomings, such as performing a fixed and pre-determined operation on a test image regardless of its content and difficulty in modeling long-range interactions. In this work we show that a different deep neural network architecture, based entirely on self-attention between neighboring image patches and without any convolution operations, can achieve more accurate segmentations than FCNs. Our proposed model is based directly on the transformer network architecture. Given a 3D image block, our network divides it into non-overlapping 3D patches and computes a 1D embedding for each patch. The network predicts the segmentation map for the block based on the self-attention between these patch embeddings. Furthermore, in order to address the common problem of scarcity of labeled medical images, we propose methods for pre-training this model on large corpora of unlabeled images. Our experiments show that the proposed model can achieve segmentation accuracies that are better than several state of the art FCN architectures on two datasets. Our proposed network can be trained using only tens of labeled images. Moreover, with the proposed pre-training strategies, our network outperforms FCNs when labeled training data is small.

Performance improvement of weakly supervised fully convolutional networks by skip connections for brain structure segmentation.

For the planning and navigation of neurosurgery, we have developed a fully convolutional network (FCN)-based method for brain structure segmentation on magnetic resonance (MR) images. The capability of an FCN depends on the quality of the training data (i.e., raw data and annotation data) and network architectures. The improvement of annotation quality is a significant concern because it requires much labor for labeling organ regions. To address this problem, we focus on skip connection architectures and reveal which skip connections are effective for training FCNs using sparsely annotated brain images. We tested 2D FCN architectures with four different types of skip connections. The first was a U-Net architecture with horizontal skip connections that transfer feature maps at the same scale from the encoder to the decoder. The second was a U-Net++ architecture with dense convolution layers and dense horizontal skip connections. The third was a full-resolution residual network (FRRN) architecture with vertical skip connections that pass feature maps between each downsampled scale path and the full-resolution scale path. The last one was a hybrid architecture with a combination of horizontal and vertical skip connections. We validated the effect of skip connections on medical image segmentation from sparse annotation based on these four FCN architectures, which were trained under the same conditions. For multiclass segmentation of the cerebrum, cerebellum, brainstem, and blood vessels from sparsely annotated MR images, we performed a comparative evaluation of segmentation performance among the above four FCN approaches: U-Net, U-Net++, FRRN, and hybrid architectures. The experimental results show that the horizontal skip connections in the U-Net architectures were effective for the segmentation of larger sized objects, whereas the vertical skip connections in the FRRN architecture improved the segmentation of smaller sized objects. The hybrid architecture with both horizontal and vertical skip connections achieved the best results of the four FCN architectures. We then performed an ablation study to explore which skip connections in the FRRN architecture contributed to the improved segmentation of blood vessels. In the ablation study, we compared the segmentation performance between architectures with a horizontal path (HP), an HP and vertical up paths (HP+VUPs), an HP and vertical down paths (HP+VDPs), and an HP and vertical up and down paths (FRRN). We found that the vertical up paths were effective in improving the segmentation of smaller sized objects. This paper investigated which skip connection architectures were effective for multiclass brain segmentation from sparse annotation. Consequently, using vertical skip connections with horizontal skip connections allowed FCNs to improve segmentation performance.

Glioblastoma multiforme prognosis: MRI missing modality generation, segmentation and radiogenomic survival prediction.

The accurate prognosis of glioblastoma multiforme (GBM) plays an essential role in planning correlated surgeries and treatments. The conventional models of survival prediction rely on radiomic features using magnetic resonance imaging (MRI). In this paper, we propose a radiogenomic overall survival (OS) prediction approach by incorporating gene expression data with radiomic features such as shape, geometry, and clinical information. We exploit TCGA (The Cancer Genomic Atlas) dataset and synthesize the missing MRI modalities using a fully convolutional network (FCN) in a conditional generative adversarial network (cGAN). Meanwhile, the same FCN architecture enables the tumor segmentation from the available and the synthesized MRI modalities. The proposed FCN architecture comprises octave convolution (OctConv) and a novel decoder, with skip connections in spatial and channel squeeze & excitation (skip-scSE) block. The OctConv can process low and high-frequency features individually and improve model efficiency by reducing channel-wise redundancy. Skip-scSE applies spatial and channel-wise excitation to signify the essential features and reduces the sparsity in deeper layers learning parameters using skip connections. The proposed approaches are evaluated by comparative experiments with state-of-the-art models in synthesis, segmentation, and overall survival (OS) prediction. We observe that adding missing MRI modality improves the segmentation prediction, and expression levels of gene markers have a high contribution in the GBM prognosis prediction, and fused radiogenomic features boost the OS estimation.

Vibration-based damage detection of rail fastener using fully convolutional networks

The rail fastener plays an important role in supporting the rail and isolating the train-track system vibrations. Its damage detection is therefore an inevitable part in railway maintenance to ensure the system performance. Recently, deep learning techniques are widely applied to structural health monitoring, including damage detection of the rail fastener based on visions. However, the vision-based detection is mainly limited to visible damages. To address this limitation, this work presents a vibration-based detection method by introducing the fully convolutional network (FCN) to identify invisible damages of fasteners. Firstly, three damage categories are defined and five damage degrees are equivalently represented by modifying the stiffness and damping coefficients of target fasteners. Then, a vehicle-track vertically coupled dynamics model with variable vehicle speeds is established to obtain axle box accelerations (ABAs) under excitations of fastener damage and track irregularity. Finally, a fastener damage detection network is designed based on the FCN architecture to predict damage degrees by inputting the ABA, the track irregularity and the vehicle speed simultaneously. The detection performance is estimated and the network robustness to noise is analysed. The results show that the proposed method is capable of achieving accurate, real-time and robust identification of the fastener damage.

Prediction of Categorized Sea Ice Concentration From Sentinel-1 SAR Images Based on a Fully Convolutional Network

The consistent and long-term spaceborne synthetic aperture radar (SAR) missions such as Sentinel-1 (S-1) provide high-quality dual-polarized <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$C$</tex-math></inline-formula> -band images particularly suited to sea ice monitoring. SAR data are currently the primary source of information for sea ice charting made by human ice experts as a result of its integration with multiple information sources at different scales (mainly radiometers). The rise of deep learning now opens the prospect of automatic sea ice mapping. In this article, we investigate the potential of a fully convolutional network (FCN) for the automatic estimation of sea ice concentration (SIC). With input data down-sampled at 200 m and an FCN architecture (depth and receptive field) duly parameterized, our approach is to mimic the work of an analyst who considers general context and does not necessarily use the highest possible resolution but speckle-noise polluted data. A comprehensive database is generated with 1320 dual-polarized S-1 scenes collocated with ice charts produced by MET Norway. A dedicated attention is paid to seasonal representativeness to ensure the adequate performance for all sea ice types. Even if the FCN output is modeled as a categorical problem, the proposed architecture accounts for the semantic distances between SIC classes by introducing an auxiliary loss. A comparative benchmark with Ocean and Sea Ice Satellite Application Facility (OSISAF) and MET Norway SIC products is carried out, showing an overall accuracy of 78.2% for our 6-class classification approach. The FCN model is shown to be evenly robust to sea ice seasonal variability and incidence angle.

CondNet: Conditional Classifier for Scene Segmentation

The fully convolutional network (FCN) has achieved tremendous success in dense visual recognition tasks, such as scene segmentation. The last layer of FCN is typically a global classifier (1x1 convolution) to recognize each pixel to a semantic label. We empirically show that this global classifier, ignoring the intra-class distinction, may lead to sub-optimal results. In this work, we present a conditional classifier to replace the traditional global classifier, where the kernels of the classifier are generated dynamically conditioned on the input. The main advantages of the new classifier consist of: (i) it attends on the intra-class distinction, leading to stronger dense recognition capability; (ii) the conditional classifier is simple and flexible to be integrated into almost arbitrary FCN architectures to improve the prediction. Extensive experiments demonstrate that the proposed classifier performs favourably against the traditional classifier on the FCN architecture. The framework equipped with the conditional classifier (called CondNet) achieves new state-of-the-art performances on two datasets. The code and models are available at https://git.io/CondNet.

Fully convolutional recurrent networks for multidate crop recognition from multitemporal image sequences

Crop recognition in tropical regions is a challenging task because of the highly complex crop dynamics, with multiple crops per year. Nevertheless, most automatic methods proposed thus far are devoted to temperate areas where normally a single crop is cultivated along the crop year. This paper introduces convolutional recurrent networks for crop recognition in areas characterized by complex spatiotemporal dynamics typical of tropical agriculture, where a per date classification is required. The proposed networks consist of two sequential steps. First, a deep network simultaneously models spatial and temporal contexts. Second, a post-processing algorithm enforces prior knowledge about the crop dynamics in the target area based on the posterior probabilities computed in the first step. The paper proposes deep network architectures that join a fully convolutional network (FCN) for modeling spatial context at multiple levels and a bidirectional recurrent neural network to explore the temporal context. The recurrent network is configured as N-to-N, where N is the sequence length. This allows it to produce classification outcomes for the entire sequence of multi-temporal images using a single network. Different network designs are proposed based on three FCN architectures: U-Net, dense network, and Atrous Spatial Pyramid Pooling. A convolutional Long-Short-Term-Memory (ConvLSTM) accounts for sequence modeling, whereas the Most Likely Class Sequence (MLCS) algorithm is adopted for enforcing prior knowledge. The paper finally reports experiments conducted on Sentinel-1 data of two publicly available datasets from different tropical regions. The experimental results indicated that the proposed architectures outperformed state-of-the-art methods based on recurrent networks in terms of Overall Accuracy and per-class F1 score.

Feature extraction from satellite images using segnet and fully convolutional networks (FCN)

Object detection and classification are among the most popular topics in Photogrammetry and Remote Sensing studies. With technological developments, a large number of high-resolution satellite images have been obtained and it has become possible to distinguish many different objects. Despite all these developments, the need for human intervention in object detection and classification is seen as one of the major problems. Machine learning has been used as a priority option to this day to reduce this need. Although success has been achieved with this method, human intervention is still needed. Deep learning provides a great convenience by eliminating this problem. Deep learning methods carry out the learning process on raw data unlike traditional machine learning methods. Although deep learning has a long history, the main reasons for its increased popularity in recent years are; the availability of sufficient data for the training process and the availability of hardware to process the data. In this study, a performance comparison was made between two different convolutional neural network architectures (SegNet and Fully Convolutional Networks (FCN)) which are used for object segmentation and classification on images. These two different models were trained using the same training dataset and their performances have been evaluated using the same test dataset. The results show that, for building segmentation, there is not much significant difference between these two architectures in terms of accuracy, but FCN architecture is more successful than SegNet by 1%. However, this situation may vary according to the dataset used during the training of the system.

Comparing Fully Deep Convolutional Neural Networks for Land Cover Classification with High-Spatial-Resolution Gaofen-2 Images

Land cover is an important variable of the terrestrial ecosystem that provides information for natural resources management, urban sprawl detection, and environment research. To classify land cover with high-spatial-resolution multispectral remote sensing imagery is a difficult problem due to heterogeneous spectral values of the same object on the ground. Fully convolutional networks (FCNs) are a state-of-the-art method that has been increasingly used in image segmentation and classification. However, a systematic quantitative comparison of FCNs on high-spatial-multispectral remote imagery was not yet performed. In this paper, we adopted the three FCNs (FCN-8s, Segnet, and Unet) for Gaofen-2 (GF2) satellite imagery classification. Two scenes of GF2 with a total of 3329 polygon samples were used in the study area and a systematic quantitative comparison of FCNs was conducted with red, green, blue (RGB) and RGB+near infrared (NIR) inputs for GF2 satellite imagery. The results showed that: (1) The FCN methods perform well in land cover classification with GF2 imagery, and yet, different FCNs architectures exhibited different results in mapping accuracy. The FCN-8s model performed best among the Segnet and Unet architectures due to the multiscale feature channels in the upsampling stage. Averaged across the models, the overall accuracy (OA) and Kappa coefficient (Kappa) were 5% and 0.06 higher, respectively, in FCN-8s when compared with the other two models. (2) High-spatial-resolution remote sensing imagery with RGB+NIR bands performed better than RGB input at mapping land cover, and yet the advantage was limited; the OA and Kappa only increased an average of 0.4% and 0.01 in the RGB+NIR bands. (3) The GF2 imagery provided an encouraging result in estimating land cover based on the FCN-8s method, which can be exploited for large-scale land cover mapping in the future.

RefineU-Net: Improved U-Net with progressive global feedbacks and residual attention guided local refinement for medical image segmentation

Abstract Motivated by the recent advances in medical image segmentation using a fully convolutional network (FCN) called U-Net and its modified variants, we propose a novel improved FCN architecture called RefineU-Net. The proposed RefineU-Net consists of three modules: encoding module (EM), global refinement module (GRM) and local refinement module (LRM). EM is backboned by pretrained VGG-16 using ImageNet. GRM is proposed to generate intermediate layers in the skip connections in U-Net. It progressively upsamples the top side output of EM and fuses the resulted upsampled features with the side outputs of EM at each resolution level. Such fused features combine the global context information in shallow layers and the semantic information in deep layers for global refinement. Subsequently, to facilitate local refinement, LRM is proposed using residual attention gate (RAG) to generate discriminative attentive features to be concatenated with the decoded features in the expansive path of U-Net. Three modules are trained jointly in an end-to-end manner thereby both global and local refinement are performed complementarily. Extensive experiments conducted on four public datasets of polyp and skin lesion segmentation show the superiority of the proposed RefineU-Net to multiple state-of-the-art related methods.

Skin lesion segmentation using fully convolutional networks: A comparative experimental study

Because the most dangerous type of skin cancer, melanoma, is very difficult for dermatologists to detect because of the low contrast between the lesion and the adjacent skin, the automatic application of skin lesion segmentation is regarded as very challenging. This paper proposes the implementation of a medical image segmentation that will accelerate a melanoma diagnosis by dermatologists. In the implementation, Fully Convolutional Network (FCN) architectures generated by modifying Convolutional Neural Network (CNN) architectures are used. The proposed algorithm for an automatic semantic segmentation of skin lesions utilizes four different FCN architectures, FCN-AlexNet, FCN-8s, FCN-16s, and FCN-32s. The experimental studies in this paper are constructed on the ISIC 2017 dataset, and the evaluations of these architectures on the dataset are carried out for the first time with this study. In the experimental studies, once the images in the dataset are preprocessed, the FCNs are first trained separately. Secondly, the accuracies and Dice coefficients on the validation dataset are calculated by using these trained FCN architectures. Thirdly, the obtained results are compared. Finally, the inferences of lesion segmentation are visualized in order to exhibit how exactly the FCN architectures can segment the lesions. The experimental results show that the FCNs in the proposed algorithm are suitable for skin lesion segmentation. In addition, it is thought that the experimental results will contribute to the scientific literature and assist the researchers who are working on medical image segmentation.

Skin Lesion Segmentation with Improved Convolutional Neural Network.

Recently, the incidence of skin cancer has increased considerably and is seriously threatening human health. Automatic detection of this disease, where early detection is critical to human life, is quite challenging. Factors such as undesirable residues (hair, ruler markers), indistinct boundaries, variable contrast, shape differences, and color differences in the skin lesion images make automatic analysis quite difficult. To overcome these challenges, a highly effective segmentation method based on a fully convolutional network (FCN) is presented in this paper. The proposed improved FCN (iFCN) architecture is used for the segmentation of full-resolution skin lesion images without any pre- or post-processing. It is to support the residual structure of the FCN architecture with spatial information. This situation, which creates a more advanced residual system, enables more precise detection of details on the edges of the lesion, and an analysis independent of skin color can be performed. It offers two contributions: determining the center of the lesion and clarifying the edge details despite the undesirable effects. Two publicly available datasets, the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 Challenge and PH2 datasets, are used to evaluate the performance of the iFCN method. The mean Jaccard index is 78.34%, the mean Dice score is 88.64%, and the mean accuracy value is 95.30% for the proposed method for the ISBI 2017 test dataset. Furthermore, the mean Jaccard index is 87.1%, the mean Dice score is 93.02%, and the mean accuracy value is 96.92% for the proposed method for the PH2 test dataset.

End-Face Localization and Segmentation of Steel Bar Based on Convolution Neural Network

Both number manually-counting method and traditional Machine-Vision (MV) number counting strategy are laborious and very time-consuming (sometimes several hours). Thus a new deep learning (DL) fusion model is proposed, which includes object detection and semantic segmentation. It can solve the problems of end-face localization and segmentation of steel bars at the same time. In this fusion model, firstly, an improved data augmentation method namely, Sliding Window Data Augmentation (SWDA) is adopted to compensate less training data concerning object detection, based on which a new object-detection architecture, Inception-RFB-FPN is presented to improve the accuracy and inference time. Secondly, a novel AI labeling method, Fibonacci-incremental mask labeling method (FIMLM) is introduced to accelerate the generation of annotation mask. Furthermore, by contrast, three FCN (Fully Convolutional Networks) architectures of data segmentation, namely, VGG16-FCN, ResNet18-FCN, and ResNet34-FCN are used to conduct the end-face segmentations of steel bars separately. Finally, a series of experiments show that the proposed Inception-RFB-FPN model can reach 98.17% in F1 score (harmonic mean value of precision and recall) with respect to object detection, and its inference time only needs 0.0306 seconds, much faster than some related reports. In addition, the FIMLM-based ResNet34-FCN model can reach 97.47% in mean Intersection-Over-Union (mIOU) with respect to semantic segmentation, higher than both VGG16-FCN and ResNet18-FCN.

Delineation of Agricultural Field Boundaries from Sentinel-2 Images Using a Novel Super-Resolution Contour Detector Based on Fully Convolutional Networks

Boundaries of agricultural fields are important features necessary for defining the location, shape, and spatial extent of agricultural units. They are commonly used to summarize production statistics at the field level. In this study, we investigate the delineation of agricultural field boundaries (AFB) from Sentinel-2 satellite images acquired over the Flevoland province, the Netherlands, using a deep learning technique based on fully convolutional networks (FCNs). We designed a multiple dilation fully convolutional network (MD-FCN) for AFB detection from Sentinel-2 images at 10 m resolution. Furthermore, we developed a novel super-resolution semantic contour detection network (named SRC-Net) using a transposed convolutional layer in the FCN architecture to enhance the spatial resolution of the AFB output from 10 m to 5 m resolution. The SRC-Net also improves the AFB maps at 5 m resolution by exploiting the spatial-contextual information in the label space. The results of the proposed SRC-Net outperform alternative upsampling techniques and are only slightly inferior to the results of the MD-FCN for AFB detection from RapidEye images acquired at 5 m resolution.

Synthesizing diffusion tensor imaging from functional MRI using fully convolutional networks

PurposeMedical image synthesis can simulate a target modality of interest based on existing modalities and has the potential to save scanning time while contributing to efficient data collection. This study proposed a three-dimensional (3D) deep learning architecture based on a fully convolutional network (FCN) to synthesize diffusion-tensor imaging (DTI) from resting-state functional magnetic resonance imaging (fMRI).Methods: fMRI signals derived from white matter (WM) exist and can be used for assessing WM alterations. We constructed an initial functional correlation tensor image using the correlation patterns of adjacent fMRI voxels as one input to the FCN. We considered T1-weighted images as an additional input to provide an algorithm with the structural information needed to synthesize DTI. Our architecture was trained and tested using a large-scale open database dataset (training n = 648; testing n = 293). ResultsThe average correlation value between synthesized and actual diffusion tensors for 38 WM regions was 0.808, which significantly improves upon an existing study (r = 0.480). We also validated our approach using two open databases. Our proposed method showed a higher correlation with the actual diffusion tensor than the conventional machine-learning method for many WM regions. ConclusionsOur method synthesized DTI images from fMRI images using a 3D FCN architecture. We hope to expand our method of synthesizing various other imaging modalities from a single image source.

A Resource-Efficient Embedded Iris Recognition System Using Fully Convolutional Networks

Applications of fully convolutional networks (FCN) in iris segmentation have shown promising advances. For mobile and embedded systems, a significant challenge is that the proposed FCN architectures are extremely computationally demanding. In this article, we propose a resource-efficient, end-to-end iris recognition flow, which consists of FCN-based segmentation and a contour fitting module, followed by Daugman normalization and encoding. To attain accurate and efficient FCN models, we propose a three-step SW/HW co-design methodology consisting of FCN architectural exploration, precision quantization, and hardware acceleration. In our exploration, we propose multiple FCN models, and in comparison to previous works, our best-performing model requires 50× fewer floating-point operations per inference while achieving a new state-of-the-art segmentation accuracy. Next, we select the most efficient set of models and further reduce their computational complexity through weights and activations quantization using an 8-bit dynamic fixed-point format. Each model is then incorporated into an end-to-end flow for true recognition performance evaluation. A few of our end-to-end pipelines outperform the previous state of the art on two datasets evaluated. Finally, we propose a novel dynamic fixed-point accelerator and fully demonstrate the SW/HW co-design realization of our flow on an embedded FPGA platform. In comparison with the embedded CPU, our hardware acceleration achieves up to 8.3× speedup for the overall pipeline while using less than 15% of the available FPGA resources. We also provide comparisons between the FPGA system and an embedded GPU showing different benefits and drawbacks for the two platforms.