Fully Convolutional Neural Network Research Articles

Wearable Sensors-Based Hand Gesture Recognition for Human–Robot Collaboration in Construction

The development of robotic machines has shown the potential to promote automation in construction. One of the critical enablers of human–robot collaboration is a user-friendly interface to support their interactions. Compared with other interfaces, hand gesture is an effective communication channel on construction sites. This article proposes a system for recognition of construction workers’ hand gestures using wearable sensors on fingers. The system starts with synchronizing, normalizing, and smoothing finger motions. Then, the motion data are extracted through a sliding window and fed into an enhanced fully convolutional neural network (FCN) for the hand gesture recognition. The system was tested through a system validation test and achieved the precision and recall of 85.7% and 93.8%, respectively. A pilot study demonstrated the use of the proposed system to interact with a robotic dump truck. The system was further compared with vision-based recognition methods to quantitatively and qualitatively assess their relative benefits and limitations.

Prediction of Uncertainty Estimation and Confidence Calibration Using Fully Convolutional Neural Network

Convolution neural networks (CNNs) have proven to be effective clinical imaging methods. This study highlighted some of the key issues within these systems. It is difficult to train these systems in a limited clinical image databases, and many publications present strategies including such learning algorithm. Furthermore, these patterns are known for making a highly reliable prognosis. In addition, normalization of volume and losses of dice have been used effectively to accelerate and stabilize the training. Furthermore, these systems are improperly regulated, resulting in more confident ratings for correct and incorrect classification, which are inaccurate and difficult to understand. This study examines the risk assessment of Fully Convolutional Neural Networks (FCNNs) for clinical image segmentation. Essential contributions have been made to this planned work: 1) dice loss and cross-entropy loss are compared on the basis of segment quality and uncertain assessment of FCNNs; 2) proposal for a group model for assurance measurement of full convolutional neural networks trained with dice loss and group normalization; And 3) the ability of the measured FCNs to evaluate the segment quality of the structures and to identify test examples outside the distribution. To evaluate the study’s contributions, it conducted a series of tests in three clinical image division applications such as heart, brain and prostate. The findings of the study provide significant insights into the predictive ambiguity assessment and a practical strategies for outside-distribution identification and reliable measurement in the clinical image segmentation. The approaches presented in this research significantly enhance the reliability and accuracy rating of CNN-based clinical imaging methods.

Toward Automated Right Ventricle Segmentation via Edge Feature-Induced Self-Attention Multiscale Feature Aggregation Full Convolution Network

In the field of cardiac magnetic resonance (MR) image analysis, the accurate segmentation of right ventricle (RV) regions plays an important role in the quantitative examination and medical diagnosis of various cardiovascular diseases. However, the automated RV segmentation in cardiac MR images is still challenging, due to its obscure and ill-defined boundaries, variably crescent-shaped structures, and extremely-unbalanced area ratio between the RV region and the background. In this work, an edge feature-induced self-attention multi-scale feature aggregation full convolutional neural network, called EFiSaMsFAFUnet, is proposed, to address the RV segmentation tasks. Specifically, EFiSaMsFAFUnet introduces an edge feature extraction module (EFEM) to mine the crescent-shaped boundary features of the RV area, and a kind of self-attention multi-scale feature expansion block (SaMsFEB) is proposed to mine the internal structure characteristics of the crescent-shaped RV internal regions. In addition, a kind of composite loss function is used to address the problem of model degradation caused by the proportion imbalance between RV region and background. The proposed EFiSaMsFAFUnet was evaluated on the MICCAI2017 Automatic Cardiac Diagnosis Challenge (ACDC) dataset. Extensive confirmatory and comparative experiments show that the EFiSaMsFAFUnet can achieve better segmentation results than the representatively state-of-the-art methods, and it can achieve comparable or the closest segmentation results to the manual segmentation of clinical specialist.

Next in Surgical Data Science: Autonomous Non-Technical Skill Assessment in Minimally Invasive Surgery Training.

It is well understood that surgical skills largely define patient outcomes both in Minimally Invasive Surgery (MIS) and Robot-Assisted MIS (RAMIS). Non-technical surgical skills, including stress and distraction resilience, decision-making and situation awareness also contribute significantly. Autonomous, technologically supported objective skill assessment can be efficient tools to improve patient outcomes without the need to involve expert surgeon reviewers. However, autonomous non-technical skill assessments are unstandardized and open for more research. Recently, Surgical Data Science (SDS) has become able to improve the quality of interventional healthcare with big data and data processing techniques (capture, organization, analysis and modeling of data). SDS techniques can also help to achieve autonomous non-technical surgical skill assessments. An MIS training experiment is introduced to autonomously assess non-technical skills and to analyse the workload based on sensory data (video image and force) and a self-rating questionnaire (SURG-TLX). A sensorized surgical skill training phantom and adjacent training workflow were designed to simulate a complicated Laparoscopic Cholecystectomy task; the dissection of the cholecyst's peritonial layer and the safe clip application on the cystic artery in an uncomfortable environment. A total of 20 training sessions were recorded from 7 subjects (3 non-medicals, 2 residents, 1 expert surgeon and 1 expert MIS surgeon). Workload and learning curves were studied via SURG-TLX. For autonomous non-technical skill assessment, video image data with tracked instruments based on Channel and Spatial Reliability Tracker (CSRT) and force data were utilized. An autonomous time series classification was achieved by a Fully Convolutional Neural Network (FCN), where the class labels were provided by SURG-TLX. With unpaired t-tests, significant differences were found between the two groups (medical professionals and control) in certain workload components (mental demands, physical demands, and situational stress, p<0.0001, 95% confidence interval, p<0.05 for task complexity). With paired t-tests, the learning curves of the trials were also studied; the task complexity resulted in a significant difference between the first and the second trials. Autonomous non-technical skill classification was based on the FCN by applying the tool trajectories and force data as input. This resulted in a high accuracy (85%) on temporal demands classification based on the z component of the used forces and 75% accuracy for classifying mental demands/situational stress with the x component of the used forces validated with Leave One Out Cross-Validation. Non-technical skills and workload components can be classified autonomously based on measured training data. SDS can be effective via automated non-technical skill assessment.

End-to-End One-Shot Path-Planning Algorithm for an Autonomous Vehicle Based on a Convolutional Neural Network Considering Traversability Cost.

Path planning plays an important role in navigation and motion planning for robotics and automated driving applications. Most existing methods use iterative frameworks to calculate and plan the optimal path from the starting point to the endpoint. Iterative planning algorithms can be slow on large maps or long paths. This work introduces an end-to-end path-planning algorithm based on a fully convolutional neural network (FCNN) for grid maps with the concept of the traversability cost, and this trains a general path-planning model for 10 × 10 to 80 × 80 square and rectangular maps. The algorithm outputs the lowest-cost path while considering the cost and the shortest path without considering the cost. The FCNN model analyzes the grid map information and outputs two probability maps, which show the probability of each point in the lowest-cost path and the shortest path. Based on the probability maps, the actual optimal path is reconstructed by using the highest probability method. The proposed method has superior speed advantages over traditional algorithms. On test maps of different sizes and shapes, for the lowest-cost path and the shortest path, the average optimal rates were 72.7% and 78.2%, the average success rates were 95.1% and 92.5%, and the average length rates were 1.04 and 1.03, respectively.

A new approach for road extraction using data augmentation and semantic segmentation

Accurate road extraction from remote sensing images is a challenging task. Several methods of extraction have been developed but the precision of extraction is still limited for the unpaved and small-width roads. This paper proposes an accurate road extraction approach called DAA-SSEG since it uses data augmentation architecture (DAA) and semantic segmentation model (SSEG). The proposed approach DAA-SSEG is based on a modified full convolutional neural network that overcomes the vanishing gradient and the training saturation issues. It recognizes roads at the pixel level. Furthermore, The DAA-SSEG approach uses a new plan of data augmentation based on geometric transformation and images refinement techniques. It allows getting a richer dataset thus better training and an accurate extraction. The experiment denotes that the proposed approach DAA-SSEG, that combine data augmentation architecture and semantic segmentation method, outperforms some state-of-the-art methods in terms of F-measures. The results demonstrate that it ensures accurate extraction of unpaved and small-width roads, in urban and rural areas. Moreover, the proposed approach distinguishes between roads and trails and can extract some roads not labeled beforehand.

Fusion of Spectral and Textural Data of Hyperspectral Imaging for Glycine Content Prediction in Beef Using SFCN Algorithms

Glycine, the simplest free amino acid. It is one of the important factors affecting the flavor of beef. In this study, a fast and non-destructive method combining near-infrared hyperspectral (900–1700 nm) and textural data was first proposed to determine the content and distribution of glycine in beef. On the basis of spectral information pre-processing, spectral features were extracted by the interval variable iterative space shrinkage approach, competitive adaptive reweighting algorithm, and uninformative variable elimination (UVE). The glycine content prediction models were established by partial least squares regression, least squares support vector machine, and the optimized shallow full convolutional neural network (SFCN). Among them, the UVE-SFCN model was found to show better results with prediction set determination coefficient (RP2) of 0.8725. Furthermore, textural features were extracted by the gray-level co-occurrence matrix and fused with the spectral information of the best feature band to obtain an optimized UVE-FSCN-fusion model (RP2 = 0.9005, root mean square error = 0.3075, residual predictive deviation = 0.2688). Compared with the full spectrum and characteristic wavelength spectrum models, RP2 was improved by 6.41% and 3.10%. The best fusion model was visualized to represent the distribution of glycine in beef. The results showed that the prediction and visualization of glycine content in beef were feasible and effective, and provided a theoretical basis for the hyperspectral study of meat quality monitoring or the establishment of an online platform.

Marine aquaculture mapping using GF-1 WFV satellite images and full resolution cascade convolutional neural network

ABSTRACT Growing demand for seafood and reduced fishery harvests have raised intensive farming of marine aquaculture in coastal regions, which may cause severe coastal water problems without adequate environmental management. Effective mapping of mariculture areas is essential for the protection of coastal environments. However, due to the limited spatial coverage and complex structures, it is still challenging for traditional methods to accurately extract mariculture areas from medium spatial resolution (MSR) images. To solve this problem, we propose to use the full resolution cascade convolutional neural network (FRCNet), which maintains effective features over the whole training process, to identify mariculture areas from MSR images. Specifically, the FRCNet uses a sequential full resolution neural network as the first-level subnetwork, and gradually aggregates higher-level subnetworks in a cascade way. Meanwhile, we perform a repeated fusion strategy so that features can receive information from different subnetworks simultaneously, leading to rich and representative features. As a result, FRCNet can effectively recognize different kinds of mariculture areas from MSR images. Results show that FRCNet obtained better performance than other classical and recently proposed methods. Our developed methods can provide valuable datasets for large-scale and intelligent modeling of the marine aquaculture management and coastal zone planning.

Improving the detection of new lesions in multiple sclerosis with a cascaded 3D fully convolutional neural network approach.

Longitudinal magnetic resonance imaging (MRI) has an important role in multiple sclerosis (MS) diagnosis and follow-up. Specifically, the presence of new lesions on brain MRI scans is considered a robust predictive biomarker for the disease progression. New lesions are a high-impact prognostic factor to predict evolution to MS or risk of disability accumulation over time. However, the detection of this disease activity is performed visually by comparing the follow-up and baseline scans. Due to the presence of small lesions, misregistration, and high inter-/intra-observer variability, this detection of new lesions is prone to errors. In this direction, one of the last Medical Image Computing and Computer Assisted Intervention (MICCAI) challenges was dealing with this automatic new lesion quantification. The MSSEG-2: MS new lesions segmentation challenge offers an evaluation framework for this new lesion segmentation task with a large database (100 patients, each with two-time points) compiled from the OFSEP (Observatoire français de la sclérose en plaques) cohort, the French MS registry, including 3D T2-w fluid-attenuated inversion recovery (T2-FLAIR) images from different centers and scanners. Apart from a change in centers, MRI scanners, and acquisition protocols, there are more challenges that hinder the automated detection process of new lesions such as the need for large annotated datasets, which may be not easily available, or the fact that new lesions are small areas producing a class imbalance problem that could bias trained models toward the non-lesion class. In this article, we present a novel automated method for new lesion detection of MS patient images. Our approach is based on a cascade of two 3D patch-wise fully convolutional neural networks (FCNNs). The first FCNN is trained to be more sensitive revealing possible candidate new lesion voxels, while the second FCNN is trained to reduce the number of misclassified voxels coming from the first network. 3D T2-FLAIR images from the two-time points were pre-processed and linearly co-registered. Afterward, a fully CNN, where its inputs were only the baseline and follow-up images, was trained to detect new MS lesions. Our approach obtained a mean segmentation dice similarity coefficient of 0.42 with a detection F1-score of 0.5. Compared to the challenge participants, we obtained one of the highest precision scores (PPVL = 0.52), the best PPVL rate (0.53), and a lesion detection sensitivity (SensL of 0.53).

SAR Target Recognition Based on Inception and Fully Convolutional Neural Network Combining Amplitude Domain Multiplicative Filtering Method

The research of Synthetic aperture radar (SAR) target recognition plays a significant role in military and civilian fields. However, for small sample SAR target recognition, there are some problems that need to be solved urgently, including low recognition accuracy, slow training convergence rate, and serious overfitting. Aiming at the above problems, we propose a recognition method based on Inception and Fully Convolutional Neural Network (IFCNN) combined with Amplitude Domain Multiplicative Filtering (ADMF) image processing. To improve the recognition accuracy and convergence rate, the ADMF method is utilized to construct the pretraining set, and the initial parameters of the network are optimized by pretraining. In addition, this paper builds the IFCNN model by introducing the Inception structure and the mixed progressive convolution layer into the FCNN. The full convolution structure of FCNN is effective to alleviate the problem of network overfitting. The Inception structure can enhance the sparsity of features and improve the network classification ability. Meanwhile, the mixed progressive convolution layers can accelerate training. Based on the MSTAR dataset, the experimental results show that the method proposed achieves an average precision of 88.95% and the training convergence rate is significantly improved in small sample scenarios.

Autosegmentation of brain metastases using 3D FCNN models and methods to manage GPU memory limitations

Aims. To explore the efficacy of two different approaches to train a Fully Convolutional Neural Network (FCNN) with Graphical Processing Unit (GPU) memory limitations and investigate if pre-trained two-dimensional weights can be transferred into a three-dimensional model for the purpose of brain tumour segmentation. Materials and methods. Models were developed in Python using TensorFlow and Keras. T1 contrast-enhanced MRI scans and associated contouring data from 104 patients were used to train and validate the model. The data was resized to one-quarter of its original resolution, and the original data was also split into four quarters for comparison to fit within GPU limitations. Transferred weights from a two-dimensional VGG16 model trained on ImageNet were transformed into three dimensions for comparison with randomly generated initial weights. Results. Resizing the data produced superior dice similarity coefficients with fewer false positives than quartering the data. Quartering the data yielded a superior sensitivity. Transforming and transferring two-dimensional weights was not able to consistently produce improvement in training or final metrics. Conclusion. For segmentation of brain tumours, resizing the data results in better performance than quartering the data. For the model and approaches used in this report, transferring weights were not able to demonstrate any benefit.

A novel fully convolutional neural network approach for detection and classification of attacks on industrial IoT devices in smart manufacturing systems

Recently, Internet of things (IoT) devices have been widely implemented and technologically advanced in manufacturing settings to monitor, collect, exchange, analyze, and deliver data. However, this transition has increased the risk of cyber-attacks, exponentially. Subsequently, developing effective intrusion detection systems based on deep learning algorithms has proven to become a reliable intelligence tool to protect Industrial IoT devices against cyber threats. This paper presents the implementation of two different classifications and detection utilizing the long short-term memory (LSTM) architecture to address cybersecurity concerns on three benchmark industrial IoT datasets (BoT-IoT, UNSW-NB15, and TON-IoT) which take advantage of various deep learning algorithms. An overall analysis of the performance of the proposed models is provided. Augmenting the LSTM with convolutional neural network (CNN) and fully convolutional neural network (FCN) achieves state-of-the-art performance in detecting cybersecurity threats.

A deep learning approach to real-time volumetric measurements without image reconstruction for cardiovascular magnetic resonance

Objective. Cardiovascular magnetic resonance (CMR) can measure ventricular volumes for the quantitative assessment of cardiac function in clinical cardiology. Conventionally, CMR volumetric measurements require image reconstruction and segmentation. There are limited clinical applications of real-time CMR for volumetric measurements because real-time images cannot provide sufficient quality for accurate segmentation. The presented work aims to develop a new deep learning approach to measuring ventricular volumes without image reconstruction and demonstrate that this ‘imageless’ approach would improve volumetric measurements with real-time CMR. Approach. We have developed a deep learning model for measuring ventricular volumes directly from real-time CMR raw data without image reconstruction. This novel ‘imageless’ deep learning model, not being as sensitive to image quality, provided reliable volumetric measurements for real-time CMR. To demonstrate ‘imageless’ volumetric measurements, we conducted a real-time CMR study with healthy volunteers. Several performance metrics, including mean absolute error (MAE), the Pearson correlation coefficient, and Bland–Altman analysis, were used to evaluate the proposed ‘imageless’ deep learning model in reference to U-net and fully convolutional neural network (FCNN) models based on conventional image reconstruction and segmentation. Main results. With the same raw data, the ‘imageless’ deep learning model gave a lower MAE (‘imageless’ ≤9.6 ml; ‘image-based’ ≥12.1 ml), a higher correlation coefficient (‘imageless’ ≥0.75; ‘image-based’ ≤0.51) and smaller measurement difference ranges in Bland–Altman analysis (‘imageless’ ≤23.1 ml; ‘image-based’ ≥33.8 ml). To achieve comparable performance, the ‘imageless’ deep learning model needed 2/3 of the raw data used in image reconstruction for U-net and FCNN models, indicating there was a gain in imaging acceleration for real-time CMR. Significance. We have demonstrated a novel deep learning framework that can provide reliable volumetric measurements from real-time CMR raw data without image reconstruction. This ‘imageless’ approach to real-time volumetric measurements will improve the quantitative assessment of cardiac function in clinical cardiology.

Prediction of the proximate analysis parameters of refuse-derived fuel based on deep learning approach.

Determination of proximate characteristics can be achieved using conventional analyses methods that require a certain amount of time. In cement factories, refuse-derived fuel (RDF) is continuously fed to a kiln by a conveyor belt, so even if an inappropriate proximate characteristic is determined, it would be too late to prevent the feeding of RDF to the kiln. To overcome this problem, there is a need for instant measurement of the proximate characteristics (moisture, volatile matter, ash) that enables the feeding to be stopped. In such cases, the deep learning (DL) is a useful method based on the prediction of proximate characteristics. Therefore, in this study, the aim is to estimate the mentioned parameters developed by near-infrared spectroscopy (NIR) combined with deep learning models. For this purpose, the spectrographic measurements taken from RDF samples with an NIR spectrometer, and the results of proximate analysis in a laboratory, were used together as a dataset. A fully convolutional neural network (FCNN) and ResNet were used as a network, and they were trained using images of RDF samples and proximate analysis values. The FCNN model was more successful in prediction studies. According to the FCNN model, the results show that the models in the study can predict the moisture, ash, and volatile matter content of RDF with satisfactory R2 values between 0.979, 0.983, and 0.952.

Deep learning models for river classification at sub-meter resolutions from multispectral and panchromatic commercial satellite imagery

Remote sensing of the Earth’s surface water is critical in a wide range of environmental studies, from evaluating the societal impacts of seasonal droughts and floods to the large-scale implications of climate change. Consequently, a large literature exists on the classification of water from satellite imagery. Yet, previous methods have been limited by (1) the >10 m spatial resolution of public satellite imagery, (2) classification schemes that operate at the pixel level, and (3) the need for multiple spectral bands. We advance the state-of-the-art by (1) using commercial satellite imagery with panchromatic and multispectral resolutions of ∼ 30 cm and ∼ 1.2 m, respectively, (2) developing multiple fully convolutional neural networks (FCN) that can learn the morphological features of water bodies in addition to their spectral properties, and (3) FCN that can classify water even from panchromatic imagery. This study focuses on rivers in the Arctic, using images from the Quickbird-2, WorldView-1, WorldView-2, WorldView-3, and GeoEye satellites. Because no training data are available at such high resolutions, we construct those manually. First, we use the red, green, blue, and near-infrared bands of the 8-band multispectral sensors. Those trained models all achieve excellent precision and recall over 90% on validation data, aided by on-the-fly preprocessing of the training data specific to satellite imagery. In a novel approach, we then use results from the multispectral model to generate training data for FCN that only require panchromatic imagery, of which considerably more is available. Despite the smaller feature space, these models still achieve a precision and recall of over 85%. We provide our open-source codes and trained model parameters to the remote sensing community, which paves the way to a wide range of environmental hydrology applications at vastly superior accuracies and 1–2 orders of magnitude higher spatial resolution than previously possible.

Deep Learning-Based CT Imaging in the Diagnosis of Treatment Effect of Pulmonary Nodules and Radiofrequency Ablation.

To study the effect of computerized tomography (CT) images based on deep learning algorithms on the diagnosis of pulmonary nodules and the effect of radiofrequency ablation (RFA), the U-shaped fully convolutional neural network (FCNN) (U-Net) was enhanced. The convolutional neural network (CNN) algorithm was compared with the U-Net algorithm, and segmentation performances were analyzed. Then, it was applied to the CT image diagnosis of 110 lung cancer patients admitted to hospital. The patients in the observation group (55 cases) were diagnosed based on the improved U-Net algorithm, while those in the control group (55 cases) were diagnosed by traditional methods and then treated with RFA. The Dice coefficient (0.8753) and intersection over union (IOU) (0.8788) obtained by the proposed algorithm were remarkably higher than the Dice coefficient (0.7212) and IOU (0.7231) obtained by the CNN algorithm, and the differences were considerable (P < 0.05). The boundary of the pulmonary nodule can be segmented more accurately by the proposed algorithm, which had the segmentation result closest to the gold standard among the three algorithms. The diagnostic accuracy of the pulmonary nodule in the observation group (95.3%) was superior to that of the control group (90.7%). The long diameter, volume, and maximum area of the pulmonary nodule of the observation group were significantly higher than those of the control group, with substantial differences (P < 0.05). Patients were reexamined after one, three, and six months of treatment, and 71 patients (64.55%) had complete remission, 32 patients (29.10%) had partial remission, 6 patients (5.45%) had stable disease, and 1 patient (0.90%) had disease progression. The remission rate (complete remission + partial remission) was 93.65%. The improved U-NET algorithm had good image segmentation performance and ideal segmentation effect. It can clearly display the shape of pulmonary nodules, locate the lesions, and accurately evaluate the therapeutic effect of RFA, which had clinical application value.

Research on Wear Detection of End Milling Cutter Edge Based on Image Stitching

Obtaining the complete wear state of the milling cutter during processing can help predict tool life and avoid the impact of tool breakage. A cylindrical model of tool collection is proposed, which uses the collected partial pictures of the side edge to construct a panoramic picture of tool wear. After evaluating the splicing accuracy, the fully convolutional neural network (FCN) segmentation algorithm of the VGG16 structure is used to segment the panorama of the side edge of the end mill after splicing. The FCN model is built using Tensorflow to complete the image segmentation training and testing of the side edge wear area. Experimental results show that the FCN model can segment the side wear image and effectively solve the illumination change problem and different tool wear differences. Compared with the Otsu threshold adaptive segmentation algorithm and K-means clustering algorithm, the error of the extracted wear value is 1.34% to 8.93%, and the average error rate is 5.23%. This method can obtain a more intuitive panorama of the cutter side edge wear of the end milling and provide technical support for improving tool utilization rate, machining quality, and tool selection and optimization.

Mapping Paddy Rice in Complex Landscapes with Landsat Time Series Data and Superpixel-Based Deep Learning Method

The spatial pattern and temporal variation in paddy rice areas captured by remote sensing imagery provide an effective way of performing crop management and developing suitable agricultural policies. However, fragmented and scattered rice paddies due to undulating and varied topography, and the availability and quality of remote sensing images (e.g., frequent cloud coverage) pose significant challenges to accurate long-term rice mapping, especially for traditional pixel and phenological methods in subtropical monsoon regions. This study proposed a superpixel and deep-learning-based time series method to analyze Landsat time series data for paddy rice classification in complex landscape regions. First, a superpixel segmentation map was generated using a dynamic-time-warping-based simple non-iterative clustering algorithm with preprocessed spectral indices (SIs) time series data. Second, the SI images were overlaid onto the superpixel map to construct mean SIs time series for each superpixel. Third, a multivariate long short-term memory full convolution neural network (MLSTM-FCN) classifier was employed to learn time series features of rice paddies to produce accurate paddy rice maps. The method was evaluated using Landsat imagery from 2000 to 2020 in Cengong County, Guizhou Province, China. Results indicate that the superpixel MLSTM-FCN achieved a high performance with an overall accuracy varying from 0.9547 to 0.9721, which presents an 0.17–1.23% improvement compared to the random forest method. This study showed that combining spectral, spatial, and temporal features with deep learning methods can generate accurate paddy rice maps in complex landscape regions.

Capitalizing on RGB-FIR Hybrid Imaging for Road Detection

Traditionally, road detection approaches mostly capitalize on RGB images, 3D LiDAR point cloud or their fusion. However, RGB camera is sensitive to light conditions, while LiDAR point cloud is sparse compared with dense image pixels. In this work, a new hybrid image dataset is provided for the task of road detection based on cameras. In this dataset, the hybrid images are acquired by an optically aligned hybrid imaging device, consisting of a far-infrared (FIR) imager and an RGB camera to output pixel-wise registration of thermal and RGB frames. Then we investigate on three methods based on fully convolutional neural network (F-CNN) to demonstrate the advantages by fusing RGB-FIR images in road detection. First, a middle-fusion based model is built, where the output feature maps of encoder branches from RGB and FIR images are directly concatenated into a single-fusion branch as the decoder. Next, the originally discarded layers after fusion operation for both RGB and FIR branches are recovered as the mimic branches to imitate the distributions of the fusion outputs, which constitutes an extended cross model (ECM). Moreover, the outputs of mimic branches at different scales are also used to imitate the corresponding outputs in the fusion branch, called a hierarchical cross model (HCM). The experimental results demonstrate the effectiveness and efficiency of our fusion strategies.

Adaptive cone of influence-based blade-vortex interference signal detection

An adaptive cone of influence (A-COI)-based method in the form of a full convolutional neural network (FCNN) is proposed, which leverages the principle of edge detection to detect the blade-vortex interference (BVI) signal of a helicopter under low signal-to-noise ratio. First, the COI feature is defined, which captures the dominant information of a BVI signal based on the time-scale diagram obtained through the continuous wavelet transform. Then, the FCNN-based A-COI model is designed and trained, which can conduct edge detection for the time-scale diagram and then adaptively acquire the COI features. Finally, with the obtained COI feature, a model of BVI signal representation is constructed and then a generalized likelihood ratio-based detection algorithm in the wavelet domain is derived to detect the BVI signal under low signal-to-noise ratio. The simulated and measured data-based experimental results demonstrate that the proposed method can effectively improve the performance of detecting BVI signals under the low signal-to-noise ratio.