Fully Convolutional Network Research Articles

S-Net: a multiple cross aggregation convolutional architecture for automatic segmentation of small/thin structures for cardiovascular applications.

With the success of U-Net or its variants in automatic medical image segmentation, building a fully convolutional network (FCN) based on an encoder-decoder structure has become an effective end-to-end learning approach. However, the intrinsic property of FCNs is that as the encoder deepens, higher-level features are learned, and the receptive field size of the network increases, which results in unsatisfactory performance for detecting low-level small/thin structures such as atrial walls and small arteries. To address this issue, we propose to keep the different encoding layer features at their original sizes to constrain the receptive field from increasing as the network goes deeper. Accordingly, we develop a novel S-shaped multiple cross-aggregation segmentation architecture named S-Net, which has two branches in the encoding stage, i.e., a resampling branch to capture low-level fine-grained details and thin/small structures and a downsampling branch to learn high-level discriminative knowledge. In particular, these two branches learn complementary features by residual cross-aggregation; the fusion of the complementary features from different decoding layers can be effectively accomplished through lateral connections. Meanwhile, we perform supervised prediction at all decoding layers to incorporate coarse-level features with high semantic meaning and fine-level features with high localization capability to detect multi-scale structures, especially for small/thin volumes fully. To validate the effectiveness of our S-Net, we conducted extensive experiments on the segmentation of cardiac wall and intracranial aneurysm (IA) vasculature, and quantitative and qualitative evaluations demonstrated the superior performance of our method for predicting small/thin structures in medical images.

DualConv: Dual Convolutional Kernels for Lightweight Deep Neural Networks.

Convolutional neural network (CNN) architectures are generally heavy on memory and computational requirements which make them infeasible for embedded systems with limited hardware resources. We propose dual convolutional kernels (DualConv) for constructing lightweight deep neural networks. DualConv combines 3×3 and 1×1 convolutional kernels to process the same input feature map channels simultaneously and exploits the group convolution technique to efficiently arrange convolutional filters. DualConv can be employed in any CNN model such as VGG-16 and ResNet-50 for image classification, you only look once (YOLO) and R-CNN for object detection, or fully convolutional network (FCN) for semantic segmentation. In this work, we extensively test DualConv for classification since these network architectures form the backbone for many other tasks. We also test DualConv for image detection on YOLO-V3. Experimental results show that, combined with our structural innovations, DualConv significantly reduces the computational cost and number of parameters of deep neural networks while surprisingly achieving slightly higher accuracy than the original models in some cases. We use DualConv to further reduce the number of parameters of the lightweight MobileNetV2 by 54% with only 0.68% drop in accuracy on CIFAR-100 dataset. When the number of parameters is not an issue, DualConv increases the accuracy of MobileNetV1 by 4.11% on the same dataset. Furthermore, DualConv significantly improves the YOLO-V3 object detection speed and improves its accuracy by 4.4% on PASCAL visual object classes (VOC) dataset.

Ceph-Net: automatic detection of cephalometric landmarks on scanned lateral cephalograms from children and adolescents using an attention-based stacked regression network

BackgroundThe success of cephalometric analysis depends on the accurate detection of cephalometric landmarks on scanned lateral cephalograms. However, manual cephalometric analysis is time-consuming and can cause inter- and intra-observer variability. The purpose of this study was to automatically detect cephalometric landmarks on scanned lateral cephalograms with low contrast and resolution using an attention-based stacked regression network (Ceph-Net).MethodsThe main body of Ceph-Net compromised stacked fully convolutional networks (FCN) which progressively refined the detection of cephalometric landmarks on each FCN. By embedding dual attention and multi-path convolution modules in Ceph-Net, the network learned local and global context and semantic relationships between cephalometric landmarks. Additionally, the intermediate deep supervision in each FCN further boosted the training stability and the detection performance of cephalometric landmarks.ResultsCeph-Net showed a superior detection performance in mean radial error and successful detection rate, including accuracy improvements in cephalometric landmark detection located in low-contrast soft tissues compared with other detection networks. Moreover, Ceph-Net presented superior detection performance on the test dataset split by age from 8 to 16 years old.ConclusionsCeph-Net demonstrated an automatic and superior detection of cephalometric landmarks by successfully learning local and global context and semantic relationships between cephalometric landmarks in scanned lateral cephalograms with low contrast and resolutions.

Hybrid CNN-transformer network for interactive learning of challenging musculoskeletal images

Background and objectivesSegmentation of regions of interest (ROIs) such as tumors and bones plays an essential role in the analysis of musculoskeletal (MSK) images. Segmentation results can help with orthopaedic surgeons in surgical outcomes assessment and patient's gait cycle simulation. Deep learning-based automatic segmentation methods, particularly those using fully convolutional networks (FCNs), are considered as the state-of-the-art. However, in scenarios where the training data is insufficient to account for all the variations in ROIs, these methods struggle to segment the challenging ROIs that with less common image characteristics. Such characteristics might include low contrast to the background, inhomogeneous textures, and fuzzy boundaries. Methodswe propose a hybrid convolutional neural network – transformer network (HCTN) for semi-automatic segmentation to overcome the limitations of segmenting challenging MSK images. Specifically, we propose to fuse user-inputs (manual, e.g., mouse clicks) with high-level semantic image features derived from the neural network (automatic) where the user-inputs are used in an interactive training for uncommon image characteristics. In addition, we propose to leverage the transformer network (TN) – a deep learning model designed for handling sequence data, in together with features derived from FCNs for segmentation; this addresses the limitation of FCNs that can only operate on small kernels, which tends to dismiss global context and only focus on local patterns. ResultsWe purposely selected three MSK imaging datasets covering a variety of structures to evaluate the generalizability of the proposed method. Our semi-automatic HCTN method achieved a dice coefficient score (DSC) of 88.46 ± 9.41 for segmenting the soft-tissue sarcoma tumors from magnetic resonance (MR) images, 73.32 ± 11.97 for segmenting the osteosarcoma tumors from MR images and 93.93 ± 1.84 for segmenting the clavicle bones from chest radiographs. When compared to the current state-of-the-art automatic segmentation method, our HCTN method is 11.7%, 19.11% and 7.36% higher in DSC on the three datasets, respectively. ConclusionOur experimental results demonstrate that HCTN achieved more generalizable results than the current methods, especially with challenging MSK studies.

Deep unsupervised part-whole relational visual saliency

Deep Supervised Salient Object Detection (SSOD) excessively relies on large-scale annotated pixel-level labels which consume intensive labour acquiring high quality labels. In such precondition, deep Unsupervised Salient Object Detection (USOD) draws public attention. Under the framework of the existing deep USOD methods, they mostly generate pseudo labels by fusing several hand-crafted detectors’ results. On top of that, a Fully Convolutional Network (FCN) will be trained to detect salient regions separately. While the existing USOD methods have achieved some progress, there are still challenges for them towards satisfactory performance on the complex scene, including (1) poor object wholeness owing to neglecting the hierarchy of those salient regions; (2) unsatisfactory pseudo labels causing by unprimitive fusion of hand-crafted results. To address these issues, in this paper, we introduce the property of part-whole relations endowed by a Belief Capsule Network (BCNet) for deep USOD, which is achieved by a multi-stream capsule routing strategy with a belief score for each stream within the CapsNets architecture. To train BCNet well, we generate high-quality pseudo labels from multiple hand-crafted detectors by developing a consistency-aware fusion strategy. Concretely, a weeding out criterion is first defined to filter out unreliable training samples based on the inter-method consistency among four hand-crafted saliency maps. In the following, a dynamic fusion mechanism is designed to generate high-quality pseudo labels from the remaining samples for BCNet training. Experiments on five public datasets illustrate the superiority of the proposed method. Codes have been released on: https://github.com/Mirlongue/Deep-Unsupervised-Part-Whole-Relational-Visual-Saliency.

Research on Automatic Classification of Coal Mine Microseismic Events Based on Data Enhancement and FCN-LSTM Network

Efficient and accurate classification of the microseismic data obtained in coal mine production is of great significance for the guidance of coal mine production safety, disaster prevention and early warning. In the early stage, the classification of microseismic events relies on human experiences, which is not only inefficient but also often causes some misclassifications. In recent years, the neural network-based classification method has become more favored by people because of its advantages in modeling procedures. A microseismic signal is a kind of time-series signal and the application of the classification method is widely optimistic. The number and the balance of the training data samples have an important impact on the accuracy of the classification result. However, the quality of the training data set obtained from the production cannot be guaranteed. A long short-term memory (LSTM) network can analyze the time-series input data, where the image classification at the pixel level can be achieved by the fully convolutional network (FCN). The two structures in the network can not only use the advantages of the FCN for extracting signal details but also use the characteristics of LSTM for conveying and expressing the long time-series information effectively. In this paper, a time-series data enhancement combination process is proposed for the actual poor microseismic data. A hybrid FCN-LSTM network structure was built, the optimal network parameters were obtained by experiments, and finally a reasonable microseismic data classifier was obtained.

Deep learning‐based robust hybrid approaches for brain tumor classification in magnetic resonance images

AbstractThe classification of brain images is an immensely challenging task and one of the most valuable and frequently employed procedures in the field of medical diagnostics. Deep Learning (DL), which falls under the umbrella of artificial intelligence, has introduced novel techniques for automated analysis of medical images. The objective of this study was to develop two hybrid DL models for the segmentation and classification of brain magnetic resonance imaging (MRI) data. The first hybrid model combines fully convolutional networks (FCNs) and residual networks (ResNets) for both segmentation and classification purposes. The second model is a hybrid of SegNet for segmentation and MobileNet for classification. Rest assured, it will provide you with a dependable and promising performance. The analysis was conducted on a labeled dataset comprising images of glioma, meningioma, pituitary, and no‐tumor cases. Python was utilized to implement the aforementioned hybrid models. The proposed models achieved remarkable accuracies of 93.9% and 91.3%, respectively, when evaluated on a brain tumor MRI dataset. Additionally, a comprehensive evaluation of the proposed models was conducted, comparing them with other existing hybrid models in terms of metrics such as the dice score, Jaccard index, positive predictive value (PPV), false predictive value (FPV), testing time, precision, recall, specificity, and F1 score. The results demonstrated that the proposed DL technique can significantly assist doctors and radiologists in the early detection of brain tumor cells.

3D Ultrasonic Brain Imaging with Deep Learning Based on Fully Convolutional Networks.

Compared to magnetic resonance imaging (MRI) and X-ray computed tomography (CT), ultrasound imaging is safer, faster, and more widely applicable. However, the use of conventional ultrasound in transcranial brain imaging for adults is predominantly hindered by the high acoustic impedance contrast between the skull and soft tissue. This study introduces a 3D AI algorithm, Brain Imaging Full Convolution Network (BIFCN), combining waveform modeling and deep learning for precise brain ultrasound reconstruction. We constructed a network comprising one input layer, four convolution layers, and one pooling layer to train our algorithm. In the simulation experiment, the Pearson correlation coefficient between the reconstructed and true images was exceptionally high. In the laboratory, the results showed a slightly lower but still impressive coincidence degree for 3D reconstruction, with pure water serving as the initial model and no prior information required. The 3D network can be trained in 8 h, and 10 samples can be reconstructed in just 12.67 s. The proposed 3D BIFCN algorithm provides a highly accurate and efficient solution for mapping wavefield frequency domain data to 3D brain models, enabling fast and precise brain tissue imaging. Moreover, the frequency shift phenomenon of blood may become a hallmark of BIFCN learning, offering valuable quantitative information for whole-brain blood imaging.

Application of optimized CNN algorithm in landslide boundary detection

Landslide, as a natural geological phenomenon with great harm, seriously threatens human social activities and life safety. It has a variety of latent and immeasurable destructiveness, which has a significant impact on the economic losses in rural areas. Therefore, it is urgent to take measures to accurately identify landslides to reduce their negative impacts. However, traditional manual visual interpretation has been unable to meet the current needs for emergency rescue of landslides, so computer intelligent methods have been paid attention to. This study proposes a new recognition network to address the problem of low accuracy of intelligent landslide boundary recognition methods. Firstly, the experiment incorporated boundary structure information into the Full Convolutional Network (FCN) for optimization, and constructed an Improved Full Convolutional Network (IFCN) model to better achieve image reconstruction. After that, Attention Mechanism (AM) is further introduced to achieve accurate detection of landslide boundaries in images, namely the IFCN-AM model. The attention mechanism introduced include spatial attention mechanism and multi-channel attention mechanism. Both are responsible for enhancing the language representation ability of the model and aggregating the interrelated features between different channels. The experimental results show that IFCN-AM has a 3 % to 7 % improvement in accuracy, recall, F1 value, and MIoU value.

Multi-state feature optimization of sign glosses for continuous sign language recognition

Vision-based Continuous Sign Language Recognition (CSLR) is a challenging and weakly supervised task aimed at segmenting sign language from weakly annotated image stream sequences for recognition. Compared with Isolated Sign Language Recognition (ISLR), the biggest challenge of this work is that the image stream sequences have ambiguous time boundaries. Recent CSLR works have shown that the visual-level sign language recognition task focuses on image stream feature extraction and feature alignment, and overfitting is the most critical problem in the CSLR training process. After investigating the advanced CSLR models in recent years, we have identified that the key to this study is the adequate training of the feature extractor. Therefore, this paper proposes a CSLR model with Multi-state Feature Optimization (MFO), which is based on Fully Convolutional Network (FCN) and Connectionist Temporal Classification (CTC). The MFO mechanism supervises the multiple states of each Sign Gloss in the modeling process and provides more refined labels for training the CTC decoder, which can effectively solve the overfitting problem caused by training, while also significantly reducing the training cost in time. We validate the MFO method on the popular CSLR dataset and demonstrate that the model has better performance.

Face recognition for occluded face with mask region convolutional neural network and fully convolutional network: a literature review

<p><span lang="EN-US">Face recognition technology has been used in many ways, such as in the authentication and identification process. The object raised is a piece of face image that does not have complete facial information (occluded face), it can be due to acquisition from a different point of view or shooting a face from a different angle. This object was raised because the object can affect the detection and identification performance of the face image as a whole. Deep leaning method can be used to solve face recognition problems. In previous research, more focused on face detection and recognition based on resolution, and detection of face. Mask region convolutional neural network (mask<br /> R-CNN) method still has deficiency in the segmentation section which results in a decrease in the accuracy of face identification with incomplete face information objects. The segmentation used in mask R-CNN is fully convolutional network (FCN). In this research, exploration and modification of many FCN parameters will be carried out using the CNN backbone pooling layer, and modification of mask R-CNN for face identification, besides that, modifications will be made to the bounding box regressor. it is expected that the modification results can provide the best recommendations based on accuracy.</span></p>

Portraying the Influence Factor of Urban Vibrancy at Street Level Using Multisource Urban Data

Exploring the factors influencing urban vibrancy can help policy development and advance urban planning and sustainable development. Previous studies have typically focused on the effects of physical environmental factors (e.g., built environment, urban landscape) on urban vibrancy, ignoring the role of non-physical environmental factors (e.g., urban psychological perceptions). In addition, these studies remain focused on relatively coarse spatial units and lack the exploration of finer-grained spatial structures. In this study, a novel framework is proposed to analyze urban vibrancy and its influencing factors at a more fine-grained street level. Firstly, two types of urban sensing data, POIs and Weibo check-ins, are integrated to portray the spatial distribution patterns of urban vibrancy on the streets. Secondly, a full convolutional network (FCN-8s) is used to segment the streetscape images of Beijing and use them as a basis to extract potential visual–spatial features and urban psychological perceptual features that influence urban vibrancy. Thirdly, we reveal the deeper causes of the impact of psychological perception on urban vibrancy. Finally, an improved ridge regression model is proposed to model the relationship between features and vibrancy, reducing the covariance between features while avoiding the reduction of important features. Satisfactory regression model performances were attained with adjusted R2 values of 0.706, 0.743, and 0.807 at each characteristic level. The results of the study show that: Urban vibrancy is highly dependent on the proposed visual–spatial and urban psychological perception characteristics at the street level. In particular, positive urban psychological perceptions (safety, lively, wealthy) are positively correlated with urban vibrancy, while negative street perceptions (boring) are negatively correlated with urban vibrancy. Unlike previous research scales, our study shows that urban vibrancy portrayal based on the street scale has a greater potential to demonstrate fine-grained vibrancy distribution compared to the neighborhood scale. These findings may provide important insights for people-oriented urban development and planning.

Prediction of stochastic responses of vehicle running through the multi-spans bridge based on an optimized method

The dynamic response of a vehicle-bridge system (VBS) has been a key problem in assessments of train-running safety on bridges, ultimately requiring reliability analyses owing to uncertain parameters and random rail irregularities. To improve the computational efficiency and promote the development of a reliability analysis for VBS, this paper proposes a novel approach incorporating a stochastic pseudo-excitation method (SPEM) to optimize a deep learning approach. This approach can more efficiently calculate the dynamic responses of a VBS than the traditional methods. In this study, the dynamic responses of the uncertain system produced by the SPEM were utilized as the training datasets for the deep learning approach. Based on the training datasets, a neural network was established to construct a numerical model consisting of two functional modules: a full convolutional network (FCN) training for the input rail irregularities and gated recurrent unit (GRU) layer for the VBS response prediction, i.e., a SPEM–FCN–GRU (SFG). The GRU cell was simulated by introducing the randomness of the excitation and uncertain parameters of the system into a portion of the cell, thereby enabling the numerical model to convey the randomness of the VBS and to obtain its stochastic response. Addition, long-time-domain samples of the VBS responses were predicted based on the short-time-domain samples. To verify the accuracy and applicability of the proposed method, a train running through a multi-span simply supported railway bridge was investigated. The dynamic responses of the train were analyzed using the training set data and Monte Carlo method (MCM) and the results showed good agreement with each other. Next, by introducing an algorithm combining a convolutional neural network (CNN) with the MCM and GRU (an MCM–CNN–GRU) and comparing it with the method proposed herein, it was verified that the proposed method has better predictions and higher efficiency than those of a traditional neural network. The prediction efficiency of the SFG was analyzed. Based on the shortcomings of the SFG, we proposed an optimized SFG algorithm. The short-time-domain results predicted by the proposed method were input into the training datasets for the deep learning architecture to enhance the number of training data for the prediction of long-time-domain responses. The efficiency and accuracy of the proposed method were verified by comparing them with those of traditional neural networks. As an example, the dynamic responses of a train running through a three-span bridge were computed by the SPEM; these could be adopted to predict those of a train running through a 10-span bridge. Finally, based on the simulated and predicted results, the reliability of the VBS was studied using a first-order reliability method.

An Optimization Method of Deep Transfer Learning for Vegetation Segmentation under Rainy and Dry Season Differences in a Dry Thermal Valley.

Deep learning networks might require re-training for different datasets, consuming significant manual labeling and training time. Transfer learning uses little new data and training time to enable pre-trained network segmentation in relevant scenarios (e.g., different vegetation images in rainy and dry seasons); however, existing transfer learning methods lack systematicity and controllability. So, an MTPI method (Maximum Transfer Potential Index method) was proposed to find the optimal conditions in data and feature quantity for transfer learning (MTPI conditions) in this study. The four pre-trained deep networks (Seg-Net (Semantic Segmentation Networks), FCN (Fully Convolutional Networks), Mobile net v2, and Res-Net 50 (Residual Network)) using the rainy season dataset showed that Res-Net 50 had the best accuracy with 93.58% and an WIoU (weight Intersection over Union) of 88.14%, most worthy to transfer training in vegetation segmentation. By obtaining each layer's TPI performance (Transfer Potential Index) of the pre-trained Res-Net 50, the MTPI method results show that the 1000-TDS and 37-TP were estimated as the best training speed with the smallest dataset and a small error risk. The MTPI transfer learning results show 91.56% accuracy and 84.86% WIoU with 90% new dataset reduction and 90% iteration reduction, which is informative for deep networks in segmentation tasks between complex vegetation scenes.

A multilevel segmentation method of asymmetric semantics based on deep learning

AbstractAn asymmetric semantic multi‐level segmentation method based on depth learning is proposed in order to improve the precision and effect of semantic segmentation. A ‘content tree’ structure and an adjacency matrix are constructed to represent the parent‐child relationship between each image sub region in a complete image. Through multiple combinations of spatial attention mechanism and channel attention mechanism, the similarity semantic features of the target object can be selectively aggregated, so as to enhance its feature expression and avoid the impact of significant objects. The asymmetric semantic segmentation model asymmetric pyramid feature convolutional network (APFCN) is constructed, and the path feature extraction and parameter adjustment are realised through APFCN. On the basis of APFCN network, a full convolution network is introduced for end‐to‐end image semantic segmentation. Combining the advantages of convolution network in extracting image features and the advantages of short‐term and short‐term memory network in solving long‐term dependence, an end‐to‐end hybrid depth network is constructed for image semantic multi‐level segmentation. The experimental results show that the mean intersection over Union value and mean pixel accuracy value are higher than that of the literature method, both of which are increased by more than 3%, and the segmentation effect is good.

Using Fully Convolutional Network to Locate Transcription Factor Binding Sites Based on DNA Sequence and Conservation Information.

Transcription factors (TFs) play a part in gene expression. TFs can form complex gene expression regulation system by combining with DNA. Thereby, identifying the binding regions has become an indispensable step for understanding the regulatory mechanism of gene expression. Due to the great achievements of applying deep learning (DL) to computer vision and language processing in recent years, many scholars are inspired to use these methods to predict TF binding sites (TFBSs), achieving extraordinary results. However, these methods mainly focus on whether DNA sequences include TFBSs. In this paper, we propose a fully convolutional network (FCN) coupled with refinement residual block (RRB) and global average pooling layer (GAPL), namely FCNARRB. Our model could classify binding sequences at nucleotide level by outputting dense label for input data. Experimental results on human ChIP-seq datasets show that the RRB and GAPL structures are very useful for improving model performance. Adding GAPL improves the performance by 9.32% and 7.61% in terms of IoU (Intersection of Union) and PRAUC (Area Under Curve of Precision and Recall), and adding RRB improves the performance by 7.40% and 4.64%, respectively. In addition, we find that conservation information can help locate TFBSs.

Faulty-Feeder Detection for Single Phase-to-Ground Faults in Distribution Networks Based on Waveform Encoding and Waveform Segmentation

Faulty feeder detection helps ensure the stability and safety of power grids after single-phase-to-ground (SPG) faults occur in distribution networks. The existing detection techniques identify the faulty feeder by extracting representative fault features, while they fail to show reliable detection performance due to variable fault conditions and complex fault transients. To address these drawbacks, this paper proposes a method based on waveform encoding and waveform segmentation. Since the waveforms have complete fault features in fault signals, it is suitable to recognize the signals on the waveform scale, rather than extracting and fusing several fault features. Firstly, raw sampled zero-sequence voltage (ZSV) and zero-sequence current (ZSC) are processed by using the proposed encoding method, and the ZSV-ZSC image can be generated quickly. Secondly, to learn and understand the waveforms of ZSV and ZSC, a two-path fully convolutional network (FCN) is established to make pixel-wise prediction on the ZSV-ZSC image. Finally, the fault degree of each feeder can be estimated based on the segmented waveform in the ZSV-ZSC image. The performance evaluation is implemented in the NVIDIA Jetson Xavier embedded platform, and the experimental results demonstrate that the proposed method can identify the faulty feeder with high accuracy within 28 ms.

Crack segmentation in the wild using convolutional neural networks and bootstrapping

AbstractComputer vision and deep learning methods have found numerous practical applications over the last decade. The field of structural health monitoring has greatly benefited from such advancements. Accurate crack detection and segmentation are a critical part of structural health monitoring and assessment. In the past decade, researchers have developed different computer vision and deep learning methods to address this challenging task. In this article, we propose to use convolutional neural networks and bootstrapping to improve crack segmentation in the wild. Specifically, we investigate fully convolutional network (FCN) and DeepLabV3 with different ResNet architectures as the back bone and assess their performance. A unique feature of this work is the use of bootstrapping with a segmentation network and image augmentation. Bootstrapping is an important component of our proposed methodology for better extracting features from the datasets. We also incorporate superpixel pooling for FCN which improves performance. We assess the performance of our method using five publicly available datasets, which include a wide variety of crack images such as thin pavement cracks, tree shadows on cracks, cracks in asphalt roads, concrete wall cracks, wide cracks on concrete surfaces, and so on. A comparative study is also performed with several established methods in the literature. Results indicate that overall, our proposed method outperforms other state‐of‐the‐art methods for crack segmentation.

Water index and Swin Transformer Ensemble (WISTE) for water body extraction from multispectral remote sensing images

ABSTRACT Automatic surface water body mapping using remote sensing technology is greatly meaningful for studying inland water dynamics at regional to global scales. Convolutional neural networks (CNN) have become an efficient semantic segmentation technique for the interpretation of remote sensing images. However, the receptive field value of a CNN is restricted by the convolutional kernel size because the network only focuses on local features. The Swin Transformer has recently demonstrated its outstanding performance in computer vision tasks, and it could be useful for processing multispectral remote sensing images. In this article, a Water Index and Swin Transformer Ensemble (WISTE) method for automatic water body extraction is proposed. First, a dual-branch encoder architecture is designed for the Swin Transformer, aggregating the global semantic information and pixel neighbor relationships captured by fully convolutional networks (FCN) and multihead self-attention. Second, to prevent the Swin Transformer from ignoring multispectral information, we construct a prediction map ensemble module. The predictions of the Swin Transformer and the Normalized Difference Water Index (NDWI) are combined by a Bayesian averaging strategy. Finally, the experimental results obtained on two distinct datasets demonstrate that the WISTE has advantages over other segmentation methods and achieves the best results. The method proposed in this study can be used for improving regional to continental surface water mapping and related hydrological studies.

Data anomaly detection with automatic feature selection and deep learning

As increasing sensor-based structural health monitoring (SHM) systems are implemented on civil infrastructures, sensor data reliability plays a crucial role in the assessment of operational performance of bridges. Sensors are heavily exposed to harsh environmental conditions during their operations and inevitably lead to possible unstable performance or failure. Thus, to accurately identify faulty sensors is a prerequisite to processing and analyzing the collected data for assessment purpose. Recently, researchers adopted the convolutional neural network (CNN) approach to identify faulty sensors, focusing on image features. Such approach may overlook some important detailed signal features and the time series approach may still be needed. However, algorithms based on time series tend to be time consuming because of the lengthy and high dimensional dataset. This may be effectively resolved using an automatic feature selection technique, namely Tsfresh, as proposed in this paper to select highly relevant signal features based on statistical tests of significance. A deep learning technique based on fully convolutional network (FCN) can then be efficiently employed for anomaly classification. The algorithm is validated using a dataset collected from a real cable-stayed bridge and results show that the proposed method significantly reduces the training time for the neural network, albeit with high classification accuracy.