Recurrent Convolutional Layers Research Articles

Video frame interpolation via spatial multi‐scale modelling

AbstractVideo frame interpolation (VFI) is a technique that synthesises intermediate frames between adjacent original video frames to enhance the temporal super‐resolution of the video. However, existing methods usually rely on heavy model architectures with a large number of parameters. The authors introduce an efficient VFI network based on multiple lightweight convolutional units and a Local three‐scale encoding (LTSE) structure. In particular, the authors introduce a LTSE structure with two‐level attention cascades. This design is tailored to enhance the efficient capture of details and contextual information across diverse scales in images. Secondly, the authors introduce recurrent convolutional layers (RCL) and residual operations, designing the recurrent residual convolutional unit to optimise the LTSE structure. Additionally, a lightweight convolutional unit named separable recurrent residual convolutional unit is introduced to reduce the model parameters. Finally, the authors obtain the three‐scale decoding features from the decoder and warp them for a set of three‐scale pre‐warped maps. The authors fuse them into the synthesis network to generate high‐quality interpolated frames. The experimental results indicate that the proposed approach achieves superior performance with fewer model parameters.

Exploiting radar polarimetry for nowcasting thunderstorm hazards using deep learning

Abstract. This work presents the importance of polarimetric variables as an additional data source for nowcasting thunderstorm hazards using an existing neural network architecture with recurrent-convolutional layers. The model can be trained to predict different target variables, which enables nowcasting of hail, lightning, and heavy rainfall for lead times up to 60 min with a 5 min resolution, in particular. The exceedance probabilities of Swiss thunderstorm warning thresholds are predicted. This study is based on observations from the Swiss operational radar network, which consists of five operational polarimetric C-band radars. The study area of the Alpine region is topographically complex and has a comparatively very high thunderstorm activity. Different model runs using combinations of single- and dual-polarimetric radar observations and radar quality indices are compared to the reference run using only single-polarimetric observations. Two case studies illustrate the performance difference when using all predictors compared to the reference model. The importance of the predictors is quantified by investigating the final training loss of the model, with skill scores such as critical success index (CSI), precision, recall, precision–recall area under the curve, and the Shapley value. Results indicate that single-polarization radar data are the most important data source. Adding polarimetric observations improves the model performance compared to reference model in term of the training loss for all three target variables. Adding quality indices does so, too. Including both polarimetric variables and quality indices at the same time improves the accuracy of nowcasting heavy precipitation and lightning, with the largest improvement found for heavy precipitation. No improvement could be achieved for nowcasting of the probability of hail in this way.

Deep Learning in the Recognition of Activities of Daily Living Using Smartwatch Data.

The recognition of human activities (HAR) using wearable device data, such as smartwatches, has gained significant attention in the field of computer science due to its potential to provide insights into individuals' daily activities. This article aims to conduct a comparative study of deep learning techniques for recognizing activities of daily living (ADL). A mapping of HAR techniques was performed, and three techniques were selected for evaluation, along with a dataset. Experiments were conducted using the selected techniques to assess their performance in ADL recognition, employing standardized evaluation metrics, such as accuracy, precision, recall, and F1-score. Among the evaluated techniques, the DeepConvLSTM architecture, consisting of recurrent convolutional layers and a single LSTM layer, achieved the most promising results. These findings suggest that software applications utilizing this architecture can assist smartwatch users in understanding their movement routines more quickly and accurately.

SellaMorph-Net: A Novel Machine Learning Approach for Precise Segmentation of Sella Turcica Complex Structures in Full Lateral Cephalometric Images

Background: The Sella Turcica is a critical structure from an orthodontic perspective, and its morphological characteristics can help in understanding craniofacial deformities. However, accurately extracting Sella Turcica shapes can be challenging due to the indistinct edges and indefinite boundaries present in X-ray images. This study aimed to develop and validate an automated Sella Morphology Network (SellaMorph-Net), a hybrid deep learning pipeline for segmenting Sella Turcica structure and extracting different morphological types; Methods: The SellaMorph-Net model proposed in this study combined attention-gating and recurrent residual convolutional layers (AGM and RrCL) to enhance the encoder’s abilities. The model’s output was then passed through a squeeze-and-excitation (SE) module to improve the network’s robustness. In addition, dropout layers were added to the end of each convolution block to prevent overfitting. A Zero-shot classifier was employed for multiple classifications, and the model’s output layer used five colour codes to represent different morphological types. The model’s performance was evaluated using various quantitative metrics, such as global accuracy and mean pixel-wise Intersection over Union (IoU) and dice coefficient, based on qualitative results; Results: The study collected 1653 radiographic images and categorised them into four classes based on the predefined shape of Sella Turcica. These classes were further divided into three subgroups based on the complexity of the Sella structures. The proposed SellaMorph-Net model achieved a global accuracy of 97.570, mean pixel-wise IoU scores of 0.7129, and a dice coefficient of 0.7324, significantly outperforming the VGG-19 and InceptionV3 models. The publicly available IEEE ISBI 2015 challenge dataset and our dataset were used to evaluate the test performance between the state-of-the-art and proposed models. The proposed model provided higher testing results, which were 0.7314 IoU and 0.7768 dice for our dataset and 0.7864 IoU and 0.8313 dice for the challenge dataset; Conclusions: The proposed hybrid SellaMorph-Net model provides an accurate and reliable pipeline for detecting morphological types of Sella Turcica using full lateral cephalometric images. Future work will focus on further improvement and utilisation of the developed model as a prognostic tool for predicting anomalies related to Sella structures.

DA-CBGRU-Seq2Seq Based Soft Sensor for Mechanical Properties of Hot Rolling Process

High-precision prediction of material properties is an advanced direction of hot rolled product technology in manufacturing processes. However, complex disturbance usually produces wrong or missing data in the acquisition and entry stages. The long process and the diverse operating conditions bring great challenges to constructing soft sensors for the hot rolling process. To address this problem, a new time-series soft sensor model framework is proposed for mechanical properties prediction. Firstly, in order to process the raw dataset, the isolation forest algorithm is used to remove the outliers, and the random forest algorithm is used to fill in missing values. Secondly, the auxiliary variables are extracted by the combination of the XGBoost algorithm and prior mechanistic knowledge. Then, a DA-CBGRU-Seq2Seq time-series model based on a two-stage attention mechanism is designed for soft sensor modeling. In the encoder stage, external factors are introduced by an entity embedding layer. A combination structure of bidirectional gated recurrent units and convolutional layers is designed for extracting and learning the input features adaptively with the input attention mechanism. In the decoder stage, the long-term time dependence of the time series is learned by the temporal attention mechanism. Finally, the reliability of the proposed sensor model is compared and verified by the hot rolling process. The results indicate that the proposed method has an accuracy of over 95% in predicting the mechanical properties of a real hot-rolling process dataset, and has better performance compared to other comparative soft sensor prediction methods.

Optimization of painting efficiency applying unique techniques of high-voltage conductors and nitrotherm spray: Developing deep learning models using computational fluid dynamics dataset

The impetus of the current three-dimensional Eulerian–Lagrangian work is to analyze the impact of simultaneously using the inventive high-voltage conductors and Nitrotherm spraying technique for maximizing the industrial painting process efficiency. This investigation employs high-fidelity computational fluid dynamics (CFD) results in deep learning models as an input dataset. The novel conductors are called high-voltage retractable blades (HVRB) and high-voltage adjustable control-ring (HVACR) mounted on the head of the electrostatic rotating bell sprayer. The influence of dominant operational parameters, such as temperature and velocity of injected nitrogen or air, droplets' electric charge values, and their size ranges, and electric field density are examined in the considered database for the Nitrotherm spraying methodology. This broad range of parametric investigation illustrates that the inclusion of shaping nitrogen flow, manipulated electric field density, and droplet charging weights significantly affect the spraying deposition rate. The pressurized clean heated nitrogen flow, which is injected from the nozzles of the atomizers, positively redirects and harmonizes the charged droplets that construct an optimized spray plume pattern with a smaller diameter. Using innovative HVRB and HVACR conductors is manipulated the electric fields and leads to denser distribution, intensifying the acting electric force on the droplets, resulting in higher spraying transfer efficiency (TE) and thicker film formation. Based on the results, employing the introduced conductors in combination with the heated nitrogen instead of air leads to higher TE, rare overspray occurrence, formation of an esthetic paint film, lower paint consumption, and application time. Also, the collected complete database is employed for machine learning investigation to predict flow with high accuracy, aiming to reduce computational time/cost. A convolutional auto-encoder is used to reduce the computational cost with just 10% of the initial CFD computations, with a mean error of 1% on the prediction of the deposited droplet areas of the spray. The analysis revealed that by employing recurrent convolutional layers, superior capturing of the input pattern is obtained, which significantly aids the final prediction.

EEG channel selection-based binary particle swarm optimization with recurrent convolutional autoencoder for emotion recognition

Electroencephalography (EEG) signals can demonstrate the activities of the human brain and recognize different emotional states. Emotion recognition based on full EEG channels leads to the use of redundant data and increases the hardware complexity. This paper presents a channel selection method based on a new Binary Many-Objective Particle Swarm Optimization with Cooperative Agents (BMaOPSO-CA). More specifically, we perform unsupervised feature learning with recurrent convolutional layers based on autoencoder architecture directly from clean EEG signals. Extensive validation on three public effective benchmarks, i.e. DASPS, DEAP, and SEED which are different in channel number used stimuli, and participant ratings, was carried out with subject independent scheme. As result, the experimental study highlights the optimal electrode locations related to emotions, leading to analyze the relationship between specific brain regions and emotions.

Fully automated segmentation of prostatic urethra for MR-guided radiation therapy.

Accurate delineation of the urethra is a prerequisite for urethral dose reduction in prostate radiotherapy. However, even in magnetic resonance-guided radiation therapy (MRgRT), consistent delineation of the urethra is challenging, particularly in online adaptive radiotherapy. This paper presented a fully automatic MRgRT-based prostatic urethra segmentation framework. Twenty-eight prostate cancer patients were included in this study. In-house 3D half fourier single-shot turbo spin-echo (HASTE) and turbo spin echo (TSE) sequences were used to image the Foley-free urethra on a 0.35T MRgRT system. The segmentation pipeline uses 3D nnU-Net as the base and innovatively combines ground truth and its corresponding radial distance (RD) map during training supervision. Additionally, we evaluate the benefit of incorporating a convolutional long short term memory (LSTM-Conv) layer and spatial recurrent convolution layer (RCL) into nnU-Net. A novel slice-by-slice simple exponential smoothing (SEPS) method specifically for tubular structures was used to post-process the segmentation results. The experimental results show that nnU-Net trained using a combination of Dice, cross-entropy and RD achieved a Dice score of 77.1±2.3% in the testing dataset. With SEPS, Hausdorff distance (HD) and 95% HD were reduced to 2.95±0.17mm and 1.84±0.11mm, respectively. LSTM-Conv and RCL layers only minimally improved the segmentation precision. We present the first Foley-free MRgRT-based automated urethra segmentation study. Our method is built on a data-driven neural network with novel cost functions and a post-processing step designed for tubular structures. The performance is consistent with the need for online and offline urethra dose reduction in prostate radiotherapy.

Estimating the danger of snow avalanches with a machine learning approach using a comprehensive snow cover model

The avalanche warning services use the regional avalanche danger level and activity as key metrics. In this study a dataset of a ski resort from 2012 to 2020 was used. The local avalanche danger level, the success of artificial avalanche release by snow groomer or blasting, and the decision to try an avalanche release by blasting are target variables. Five meteorological recordings and twenty five variables of the modeled snowpack with an hourly resolution were used as input for the machine learning approach. An artificial neural network consists of recurrent layers and convolutional layers for merging the temporal and spatial data. Support vector machines were adapted to calculate the probabilities for the target variables. A logistic regression was used as ensemble method as stacked generalization.The accuracy of the models, i.e., the precision for predicting the local avalanche danger level was 0.73 and 0.52 for the train and test data, respectively. The precisions for artificial avalanche release by snow groomer were 0.94 and 0.85, for artificial avalanche release by blasting 0.75 and 0.72, and for the decision to try an avalanche release by blasting 0.75 and 0.70.Unlike previous studies, which used regional avalanche danger levels as target variable, we used local warning levels and additional target variables. The comparison shows that despite differences similar accuracy can be achieved. We further identified opportunities to optimize the model by examining the relevance of the input variables and performing a sensitivity analysis.

AI-driven spatio-temporal engine for finding gravitationally lensed type Ia supernovae

ABSTRACT We present a spatio-temporal AI framework that concurrently exploits both the spatial and time-variable features of gravitationally lensed supernovae in optical images to ultimately aid in future discoveries of such exotic transients in wide-field surveys. Our spatio-temporal engine is designed using recurrent convolutional layers, while drawing from recent advances in variational inference to quantify approximate Bayesian uncertainties via a confidence score. Using simulated Young Supernova Experiment (YSE) images of lensed and non-lensed supernovae as a showcase, we find that the use of time-series images adds relevant information from time variability of spatial light distribution of partially blended images of lensed supernova, yielding a substantial gain of around 20 per cent in classification accuracy over single-epoch observations. Preliminary application of our network to mock observations from the Legacy Survey of Space and Time (LSST) results in detections with accuracy reaching around 99 per cent. Our innovative deep learning machinery is versatile and can be employed to search for any class of sources that exhibit variability both in flux and spatial distribution of light.

SC-RoadDeepNet: A New Shape and Connectivity-Preserving Road Extraction Deep Learning-Based Network From Remote Sensing Data

Existing automated road extraction approaches concentrate on regional accuracy rather than road shape and connectivity quality. Most of these techniques produce discontinuous outputs caused by obstacles, such as shadows, buildings, and vehicles. This study proposes a shape and connectivity-preserving road identification deep learning-based architecture called SC-RoadDeepNet to overcome the discontinuous results and the quality of road shape and connectivity. The proposed model comprises a state-of-the-art deep learning-based network, namely, the recurrent residual convolutional neural network, boundary learning (BL), and a new measure based on the intersection of segmentation masks and their (morphological) skeleton called connectivity-preserving centerline Dice (CP_clDice). The recurrent residual convolutional layers accumulate low-level features for segmentation tasks, thus allowing for better feature representation. Such representation enables us to construct a UNet network with the same number of network parameters but improved segmentation effectiveness. BL also aids the model in improving the road’s boundaries by penalizing boundary misclassification and fine-tuning the road form. Furthermore, the CP_clDice method aids the model in maintaining road connectivity and obtaining accurate segmentations. We demonstrate that CP_clDice ensures connection preservation for binary segmentation, thereby allowing for efficient road network extraction at the end. The proposed model improves F1 score accuracy to 5.49%, 4.03%, 3.42%, and 2.27% compared with other comparative models, such as LinkNet, ResUNet, UNet, and VNet, respectively. Furthermore, qualitative and quantitative assessments demonstrate that the proposed SC-RoadDeepNet can improve road extraction by tackling shadow and occlusion-related interruptions. These assessments can also produce high-resolution results, particularly in the area of road network completeness.

Bearing Life Prediction Method Based on Parallel Multichannel Recurrent Convolutional Neural Network

To extract the time-series characteristics of the original bearing signals and predict the remaining useful life (RUL) more effectively, a parallel multichannel recurrent convolutional neural network (PMCRCNN) is proposed for the prediction of RUL. Firstly, the time domain, frequency domain, and time-frequency domain features are extracted from the original signal. Then, the PMCRCNN model is constructed. The front of the model is the parallel multichannel convolution unit to learn and integrate the global and local features from the time-series data. The back of the model is the recurrent convolution layer to model the temporal dependence relationship under different degradation features. Normalized life values are used as labels to train the prediction model. Finally, the RUL was predicted by the trained neural network. The proposed method is verified by full life tests of bearing. The comparison with the existing prognostics approaches of convolutional neural network (CNN) and the recurrent convolutional neural network (RCNN) models proves that the proposed method (PMCRCNN) is effective and superior in improving the accuracy of RUL prediction.

Cardiac Segmentation from MRI images using Recurrent & Residual Convolutional Neural Network based on SegNet and Level Set methods

In recent years, semantic segmentation with Deep Learning (DL) is popularly found in many real-world applications. It is also specific, mentioned that these techniques are regularly applied to various segmentation and classification techniques in the medical field. The most popular deep learning techniques, especially SegNet and U-Net, are the most used for this type of medical application. Generally, in case of U-Net architecture which can be used with a skip connection and is capable of retrieving fine data during training. However, such a network consumes a lot of computation time compared to other networks. But SegNet is another network used to retrieve the desired information with a computing efficient. Inspired by the work, the skip connection is introduced into SegNet using the residual neural network (ResNet). ResNet consists of a layer and it has taken inputs involving multiple layers of the neural network, giving precise performance. This article offers first a recurrent convolutional neural network (RecNet) based on Seg-Net called R-SegNet and also a recurrent residual convolutional neural network (R2Net) based on SegNet models called R2-SegNet respectively. The strength of SegNet, RecNet and ResNet are used and have produced the architectures to perform the segmentation of cardiac MRI images. There are a number of advantages derived from the proposed architecture. First of all, using a residual unit, proposed architectures were used to carry out training in deep architecture. Second, the use of recurrent residual convolutional layers ensures that the relevant features retrieved to perform the segmentation tasks. Third, the proposed architecture has designed a good SegNet with a limited network parameters and also produce better performance for performing the task of segmentation in cardiac images. In addition, then, applied the level definition method, to extract the contours or surfaces of the cardiac MRI images. The results proves that the hybrid proposed deep learning methods successfully segments the images and also achieves better accuracy compared to standard architectures.

Cardiac Segmentation from MRI images using Recurrent &Residual Convolutional Neural Network based on SegNet and Level Set methods

In recent years, semantic segmentation with Deep Learning (DL) is popularly foundin many real-world applications. It is also specific, mentioned that these techniques are regularly applied to various segmentation and classification techniques in the medical field. The most popular deep learning techniques, especially SegNet and U-Net, are the most used for this type of medical application.Generally, in case of U-Net architecture which can be used with a skip connection and is capable of retrieving fine data during training. However, such a network consumes a lot of computation time compared to other networks. But SegNet is another network used to retrieve the desired information with a computing efficient. Inspired by the work, the skip connection is introduced into SegNet using the residual neural network (ResNet). ResNet consists of a layer and it has taken inputs involving multiple layers of the neural network, giving precise performance. This article offers first a recurrent convolutional neural network (RecNet) based on Seg-Net called R-SegNet and also a recurrent residual convolutional neural network (R2Net) based on SegNet models called R2-SegNet respectively. The strength of SegNet, RecNet and ResNet are used and have produced the architectures to perform the segmentation of cardiac MRI images. There are a number of advantages derived from the proposed architecture. First of all, using a residual unit, proposed architectures were used to carry out training in deep architecture. Second, the use of recurrent residual convolutional layers ensures that the relevant features retrieved to perform the segmentation tasks. Third, the proposed architecture has designed a good SegNet with a limited network parameters and also produce better performance for performing the task of segmentation in cardiac images. In addition, then, applied the level definition method, to extract the contours or surfaces of the cardiac MRI images.The results proves that the hybrid proposed deep learning methods successfully segments the images and also achieves better accuracy compared to standard architectures.

Recurrent Multi-Fiber Network for 3D MRI Brain Tumor Segmentation

Automated brain tumor segmentation based on 3D magnetic resonance imaging (MRI) is critical to disease diagnosis. Moreover, robust and accurate achieving automatic extraction of brain tumor is a big challenge because of the inherent heterogeneity of the tumor structure. In this paper, we present an efficient semantic segmentation 3D recurrent multi-fiber network (RMFNet), which is based on encoder–decoder architecture to segment the brain tumor accurately. 3D RMFNet is applied in our paper to solve the problem of brain tumor segmentation, including a 3D recurrent unit and 3D multi-fiber unit. First of all, we propose that recurrent units segment brain tumors by connecting recurrent units and convolutional layers. This quality enhances the model’s ability to integrate contextual information and is of great significance to enhance the contextual information. Then, a 3D multi-fiber unit is added to the overall network to solve the high computational cost caused by the use of a 3D network architecture to capture local features. 3D RMFNet combines both advantages from a 3D recurrent unit and 3D multi-fiber unit. Extensive experiments on the Brain Tumor Segmentation (BraTS) 2018 challenge dataset show that our RMFNet remarkably outperforms state-of-the-art methods, and achieves average Dice scores of 89.62%, 83.65% and 78.72% for the whole tumor, tumor core and enhancing tumor, respectively. The experimental results prove our architecture to be an efficient tool for brain tumor segmentation accurately.

Inception recurrent convolutional neural network for object recognition

Deep convolutional neural network (DCNN) is an influential tool for solving various problems in machine learning and computer vision. Recurrent connectivity is a very important component of visual information processing within the human brain. The idea of recurrent connectivity is rarely applied within convolutional layers, the exceptions being a couple of DCNN architectures including recurrent convolutional neural network (RCNN) in Liang and Hu (in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015) and Pinheiro and Collobert (in: ICML, 2014). On the other hand, the Inception network architecture has become popular among the computer vision community (Szegedy et al. in Inception-v4, Inception-ResNet and the impact of Residual connections on learning, 2016. arXiv:1602.07261 ). In this paper, we introduce a deep learning architecture called the Inception Recurrent Convolutional Neural Network (IRCNN), which utilizes the power of an Inception network combined with recurrent convolutional layers. Although the inputs are static, the recurrent property plays a huge role in modeling the contextual information for object recognition tasks and thus improves overall training and testing accuracy. In addition, this proposed architecture generalizes both Inception and RCNN models. We have empirically evaluated the recognition performance of the proposed IRCNN model using different benchmark datasets such as MNIST, CIFAR-10, CIFAR-100, and SVHN. The experimental results show higher recognition accuracy when compared to most of the popular DCNNs including the RCNN. Furthermore, we have investigated IRCNN performance against equivalent Inception networks (EIN) and equivalent Inception–Residual networks (EIRN) using the CIFAR-100 dataset. When using the augmented CIFAR-100 dataset, we achieved about 3.5%, 3.47% and 2.54% improvement in classification accuracy compared to the RCNN, EIN, and EIRN respectively. We have also conducted experiment on Tiny ImageNet-200 dataset with IRCNN, EIN, EIRN, RCNN, DenseNet in Huang et al. (Densely connected convolutional networks, 2016. arXiv:1608.06993 ), and DenseNet with Recurrent Convolution Layer, where the proposed model shows significantly better performance against baseline models.

Convolutional Neural Networks With Gated Recurrent Connections.

The convolutional neural network (CNN) has become a basic model for solving many computer vision problems. In recent years, a new class of CNNs, recurrent convolution neural network (RCNN), inspired by abundant recurrent connections in the visual systems of animals, was proposed. The critical element of RCNN is the recurrent convolutional layer (RCL), which incorporates recurrent connections between neurons in the standard convolutional layer. With increasing number of recurrent computations, the receptive fields (RFs) of neurons in RCL expand unboundedly, which is inconsistent with biological facts. We propose to modulate the RFs of neurons by introducing gates to the recurrent connections. The gates control the amount of context information inputting to the neurons and the neurons' RFs therefore become adaptive. The resulting layer is called gated recurrent convolution layer (GRCL). Multiple GRCLs constitute a deep model called gated RCNN (GRCNN). The GRCNN was evaluated on several computer vision tasks including object recognition, scene text recognition and object detection, and obtained much better results than the RCNN. In addition, when combined with other adaptive RF techniques, the GRCNN demonstrated competitive performance to the state-of-the-art models on benchmark datasets for these tasks.

DSFMA: deeply supervised fully convolutional neural networks based on multi-level aggregation for saliency detection

In recent years, the emergence of fully convolutional neural networks (FCNs) has delivered significant success in the field of saliency detection. Although the different levels of FCNs layers can hold different types of information for salient object detection, it is still a challenging issue to find a generic method while integrating all relevant information synthetically with multi-level aggregation. In this paper, we present a novel multi-level aggregation method by following a U-shaped architecture of the VGG-16 network. As the shallower layers of FCNs contain the low-level integrated features which are capable of capturing the more details of salient objects, while the more profound layers that hold the high-level integrated features have more contextual information. To exploit all the relevant information, we extend the last four side-outputs of U-Net at the encoder and decoder sides and then utilize the concept of skip and short-connections to incorporate the high-level contextual knowledge with low-level details. Besides, we also integrate the recurrent convolutional layers (RCLs) into our model, which provide more deepness and enhance the capability to integrate more contextual knowledge. At last, we combine all the side-outputs into a final saliency map together for salient object detection. We evaluate the performance of the proposed model on six broadly used saliency detection benchmarks by comparing it with the other 11 state-of-the-art approaches. Experimental outcomes determine that our method achieves a favorable performance for all compared evaluation measures.

Joint disc and cup segmentation based on recurrent fully convolutional network.

The optic disc(OD) and the optic cup(OC) segmentation is an key step in fundus medical image analysis. Previously, FCN-based methods have been proposed for medical image segmentation tasks. However, the consecutive convolution and pooling operations usually hinder dense prediction tasks which require detailed spatial information, such as image segmentation. In this paper, we propose a network called Recurrent Fully Convolution Network(RFC-Net) for automatic joint segmentation of the OD and the OC, which can captures more high-level information and subtle edge information. The RFC-Net can minimize the loss of spatial information. It is mainly composed of multi-scale input layer, recurrent fully convolutional network, multiple output layer and polar transformation. In RFC-Net, the multi-scale input layer constructs an image pyramid. We propose four recurrent units, which are respectively applied to RFC-Net. Recurrent convolution layer effectively ensures feature representation for OD and OC segmentation tasks through feature accumulation. For each multiple output image, the multiple output cross entropy loss function is applied. To better balance the cup ratio of the segmented image, the polar transformation is used to transform the fundus image from the cartesian coordinate system to the polar coordinate system. We evaluate the effectiveness and generalization of the proposed method on the DRISHTI-GS1 dataset. Compared with the original FCN method and other state-of-the-art methods, the proposed method achieves better segmentation performance.

A multi-scale recurrent fully convolution neural network for laryngeal leukoplakia segmentation

Laryngeal leukoplakia is one kind of precancerous lesions in the larynx. Precise detection and segmentation of leukoplakia in laryngoscopic images is important for laryngeal disease diagnosis and treatment. In this paper, we proposed a multi-scale recurrent fully convolution neural network named boldface-M-Net (BM-Net) to identify and segment laryngeal leukoplakia lesions. The proposed BM-Net was composed of a multi-scale input layer, a double U-shaped convolution network, and a side-output layer. First, we augmented the image to produce six channels and then constructed image pyramids for the multi-scale input layer. For the U-shaped convolution network, we constructed a new U-Net using multi-scale convolution and a recurrent convolution layer (RCL) instead of the original convolution layer. We then employed skip connections to connect the double U-shaped convolution network, one with three 2 × 2 max pooling layers and the other with four, thus forming the main structure of BM-Net. We added the output for the three-layered U-Net to the side-output layer to produce a companion local prediction map for each scale layer. Image pyramids and multi-scale convolution can generate multiple level-receptive fields, while the RCL allows for the greater perception of context with parameter t increases. Finally, we compared the performance of the proposed BM-Net with the popular networks, including FCN-8s, Seg-Net, U-Net, M-Net, and three other modified networks for segmenting laryngeal leukoplakia in laryngoscopic images. According to the experimental results, BM-Net, which inherited the advantages of U-Net, M-Net, and RCL, exhibited overall better performance in laryngeal leukoplakia segmentation than the other networks.