Convolutional Encoder Research Articles

Style-Controlled Image Synthesis of Concrete Damages Based on Fusion of Convolutional Encoder and Attention-Enhanced Conditional Generative Adversarial Network

Style-Controlled Image Synthesis of Concrete Damages Based on Fusion of Convolutional Encoder and Attention-Enhanced Conditional Generative Adversarial Network

Efficient Metal Corrosion Area Detection Model Combining Convolution and Transformer

In the context of rapid industrialization, efficiently detecting metal corrosion areas has become a critical task in preventing material damage. Unlike conventional semantic segmentation targets, metal corrosion characteristics vary significantly in color, texture, and size. Traditional image segmentation methods need improvement in scenarios involving occlusions, shadows, and defects. This paper proposes a convolution and sequence encoding combined network, MCD-Net, for metal corrosion area segmentation. First, a visual Transformer sequence encoder is introduced into the convolutional encoder–decoder network to enhance global information processing capabilities and establish long-range feature dependencies. A feature fusion method based on an attention module is proposed to enhance the model’s ability to recognize corrosion boundaries, thereby enhancing segmentation accuracy and model robustness. Finally, in the model’s decoding stage, a score-based multi-scale feature enhancement method is employed to emphasize significant features in the corrosion areas. Experimental results indicate that this method attained an F1 score of 84.53% on a public corrosion dataset, demonstrating the model’s deeper understanding and reasoning capabilities for shadow and defect features, as well as excellent noise resistance performance.

A novel daily runoff forecasting model based on global features and enhanced local feature interpretation

The development of artificial intelligence has introduced new perspectives to the field of hydrological forecasting. However, there is still a lack of research on efficiently identifying the physical characteristics of runoff sequences and developing prediction models that consider global and local sequence features. This study proposes a parallel computing prediction model called IMCAEN (Integrated Multi-Feature Causal Dilated Convolutional Attention Encoder Network) to address these issues. Unlike existing models, this model can monitor fluctuations and anomalies in time series. Incorporating the CDC-AA (Causal Dilated Convolutional Network with Aggregation Attention) and encoder structure captures both local sequence variations and global abrupt anomalies, allowing for comprehensive attention to sequence features. When predicting runoff data from three different hydrological conditions, the IMCAEN model achieved NSEC (Nash-Sutcliffe Efficiency Coefficient) values of 0.98, 0.97, and 0.88, respectively, and outperformed benchmark models in other evaluation indicators as well. Given the opacity of the feature distribution process in AI models, SHAP (SHapleyAdditive exPlanations) analysis and spatial expression of feature distribution are used to assess the contribution of each feature variable to the long-term trend of runoff and to verify the distribution of features trained in each module. The proposed IMCAEN model efficiently captures local and global information in the runoff evolution process through parallel computing and shared features, enabling accurate runoff forecasting and providing critical references for timely warnings and predictions.

Computer Modelling of Textures on Images with Human Skin Wound Areas

In this paper, we concentrate on the images of the wounds on the human skin and propose to consider each image as a set of smaller pieces – crops or patches containing different textures. We overview, develop and compare deep learning feature extraction methods to model image crops as 200-dimensional feature vectors using various artificial neural network architectures: convolutional autoencoders, variational convolutional autoencoders, and Siamese convolutional networks trained in the contrastive learning manner. Also, we develop a custom convolutional encoder and decoder, use them in the aforementioned architectures and compare them with the ResNet encoder and decoder alternatives. Finally, we train and evaluate k-nearest neighbors and Multi-Layer Perceptron classifiers on the features extracted with the model above options to discriminate skin, wound, and background image patches. Classification evaluation results on the features, extracted with the Siamese network, show the best test accuracy for all implementations without a significant shift between model versions (accuracy > 93%); variational autoencoders show random results for all options (accuracy around 33%), and convolutional autoencoders reached good results (accuracy > 77%) but with a noticeable difference between the custom and ResNet versions; the latter is better. Custom encoder and decoder implementations are faster and smaller than the ResNet alternatives but may be less stable on larger datasets, which still needs investigation. Possible applications of the feature vectors include an area of interest extraction during wound segmentation or classification and usage as patch embeddings while training vision transformer architectures.

HyFormer: a hybrid transformer-CNN architecture for retinal OCT image segmentation

Optical coherence tomography (OCT) has become the leading imaging technique in diagnosing and treatment planning for retinal diseases. Retinal OCT image segmentation involves extracting lesions and/or tissue structures to aid in the decisions of ophthalmologists, and multi-class segmentation is commonly needed. As the target regions often spread widely inside the retina, and the intensities and locations of different categories can be close, good segmentation networks must possess both global modeling capabilities and the ability to capture fine details. To address the challenge in capturing both global and local features simultaneously, we propose HyFormer, an efficient, lightweight, and robust hybrid network architecture. The proposed architecture features parallel Transformer and convolutional encoders for independent feature capture. A multi-scale gated attention block and a group positional embedding block are introduced within the Transformer encoder to enhance feature extraction. Feature integration is achieved in the decoder composed of the proposed three-path fusion modules. A class activation map-based cross-entropy loss function is also proposed to improve segmentation results. Evaluations are performed on a private dataset with myopic traction maculopathy lesions and the public AROI dataset for retinal layer and lesion segmentation with age-related degeneration. The results demonstrate HyFormer's superior segmentation performance and robustness compared to existing methods, showing promise for accurate and efficient OCT image segmentation. .

Improved Urdu-English Neural Machine Translation with a fully Convolutional Neural Network Encoder

Neural machine translation (NMT) approaches driven by artificial intelligence (AI) has gained more and more attention in recent years, mainly due to their simplicity yet state-of-the-art performance. Despite NMT models with attention mechanism relying heavily on the accessibility of substantial parallel corpora, they have demonstrated efficacy even for languages with limited linguistic resources. The convolutional neural network (CNN) is frequently employed in tasks involving visual and speech recognition. Implementing CNN for MT is still challenging compared to the predominant approaches. Recent research has shown that the CNN-based NMT model cannot capture long-term dependencies present in the source sentence. The CNN-based model can only capture the word dependencies within the width of its filters. This unnatural character often causes a worse performance for CNN-based NMT than the RNN-based NMT models. This study introduces a simple method to improve neural translation of a low-resource language, specifically Urdu-English (UR-EN). In this paper, we use a Fully Convolutional Neural Network (FConv-NN) based NMT architecture to create a powerful MT encoder for UR-EN translation that can capture the long dependency of words in a sentence. Although the model is quite simple, it yields strong empirical results. Experimental results show that the FConv-NN model consistently outperforms the traditional CNN-based model with filters. On the Urdu-English Dataset, the FConv-NN model produces translation with a gain of 18.42 BLEU points. Moreover, the quantitative and comparative analysis shows that in a low-resource setting, FConv-NN-based NMT outperforms conventional CNN-based NMT models.

Attention-guided hierarchical fusion U-Net for uncertainty-driven medical image segmentation

Small inaccuracies in the system components or artificial intelligence (AI) models for medical imaging could have significant consequences leading to life hazards. To mitigate those risks, one must consider the precision of the image analysis outcomes (e.g., image segmentation), along with the confidence in the underlying model predictions. U-shaped architectures, based on the convolutional encoder–decoder, have established themselves as a critical component of many AI-enabled diagnostic imaging systems. However, most of the existing methods focus on producing accurate diagnostic predictions without assessing the uncertainty associated with such predictions or the introduced techniques. Uncertainty maps highlight areas in the predicted segmented results, where the model is uncertain or less confident. This could lead radiologists to pay more attention to ensuring patient safety and pave the way for trustworthy AI applications. In this paper, we therefore propose the Attention-guided Hierarchical Fusion U-Net (named AHF-U-Net) for medical image segmentation. We then introduce the uncertainty-aware version of it called UA-AHF-U-Net which provides the uncertainty map alongside the predicted segmentation map. The network is designed by integrating the Encoder Attention Fusion module (EAF) and the Decoder Attention Fusion module (DAF) on the encoder and decoder sides of the U-Net architecture, respectively. The EAF and DAF modules utilize spatial and channel attention to capture relevant spatial information and indicate which channels are appropriate for a given image. Furthermore, an enhanced skip connection is introduced and named the Hierarchical Attention-Enhanced (HAE) skip connection. We evaluated the efficiency of our model by comparing it with eleven well-established methods for three popular medical image segmentation datasets consisting of coarse-grained images with unclear boundaries. Based on the quantitative and qualitative results, the proposed method ranks first in two datasets and second in a third. The code can be accessed at: https://github.com/AfsanaAhmedMunia/AHF-Fusion-U-Net

Enhancing Gene Expression Predictions Using Deep Learning and Functional Annotations.

Transcriptome-wide association studies (TWAS) aim to uncover genotype-phenotype relationships through a two-stage procedure: predicting gene expression from genotypes using an expression quantitative trait locus (eQTL) data set, then testing the predicted expression for trait associations. Accurate gene expression prediction in stage 1 is crucial, as it directly impacts the power to identify associations in stage 2. Currently, the first stage of such studies is primarily conducted using linear models like elastic net regression, which fail to capture the nonlinear relationships inherent in biological systems. Deep learning methods have the potential to model such nonlinear effects, but have yet to demonstrably outperform linear methods at this task. To address this gap, we propose a new deep learning architecture to predict gene expression from genotypic variation across individuals. Our method utilizes a learnable input scaling layer in conjunction with a convolutional encoder to capture nonlinear effects and higher-order interactions without compromising on interpretability. We further augment this approach to allow for parameter sharing across multiple networks, enabling us to utilize prior information for individual variants in the form of functional annotations. Evaluations on real-world genomic data show that our method consistently outperforms elastic net regression across a large set of heritable genes. Furthermore, our model statistically significantly improved predictive performance by leveraging functional annotations, whereas elastic net regression failed to show equivalent gains when using the same information, suggesting that our method can capture nonlinear functional information beyond the capability of linear models.

Investigation of block interleavers in OFDM radio communication systems with convolutional encoding in multipath channels with fast fading

Noise immunity codecs are good at correcting single errors. Group errors often occur in multipath radio channels with fading. Interleaving is used to convert group errors into single errors. Matching the interleaver structure with the signal and code parameters allows the interleaver to work efficiently. Optimization of the interleaver parameters depending on the characteristics of the communication channel increases the efficiency of the interleaver. The purpose of the work is to determine the parameters of block interleavers for an OFDM radio communication system with convolutional coding in conditions of signal scattering in time and frequency. A model of a channel with two beams and Rayleigh fades is considered. The influence of interleaver parameters on the noise immunity of the OFDM radio communication system depending on the channel parameters (fading rate, multipath interval) has been analyzed. Block deterministic and pseudorandom interleavers are considered. With magnification lengths the interleaver increases noise immunity of the communication system. But at the same time, the delay in transmitting the message increases. The optimal length has been determined, providing a compromise between delay and noise immunity. This length is determined depending on the coherence time of the channel. The number of rows of the matrix interleaver is depending on the decoding depth. This is the number of rows provides better noise immunity at optimal length. But sometimes the optimal length is too long. Therefore a function has been introduced that characterizes the minimum distance between bits in the “time-frequency-code” space at the input of the interleaver. The distance between the bits in the time domain is normalized for the coherence time of the channel. The distance between the bits in the frequency domain is calculated as a normalized deviation from the oscillation period in the signal power spectrum at the output of a channel with two beams. The distance between the bits in the “code space” is calculated as the distance normalized the effective decoding depth of the code used. Given the function describes optimal and critical values well parameters of deterministic interleavers of any length. Deterministic interleavers have a higher probability of error per bit than random interleavers. The definition of parameters using the proposed function allows the deterministic interleaver to work with the same low the probability of error per bit as well as random interleavers.

Dynamical system prediction from sparse observations using deep neural networks with Voronoi tessellation and physics constraint

Despite the success of various methods in addressing the issue of spatial reconstruction of dynamical systems with sparse observations, spatio-temporal prediction for sparse fields remains a challenge. Existing Kriging-based frameworks for spatio-temporal sparse field prediction fail to meet the accuracy and inference time required for nonlinear dynamic prediction problems. In this paper, we introduce the Dynamical System Prediction from Sparse Observations using Voronoi Tessellation (DSOVT) framework, an innovative methodology based on Voronoi tessellation which combines convolutional encoder–decoder (CED) and long short-term memory (LSTM) and utilizing Convolutional Long Short-Term Memory (ConvLSTM). By integrating Voronoi tessellations with spatio-temporal deep learning models, DSOVT is adept at predicting dynamical systems with unstructured, sparse, and time-varying observations. CED-LSTM maps Voronoi tessellations into a low-dimensional representation for time series prediction, while ConvLSTM directly uses these tessellations in an end-to-end predictive model. Furthermore, we incorporate physics constraints during the training process for dynamical systems with explicit formulas. Compared to purely data-driven models, our physics-based approach enables the model to learn physical laws within explicitly formulated dynamics, thereby enhancing the robustness and accuracy of rolling forecasts. Numerical experiments on real sea surface data and shallow water systems clearly demonstrate our framework’s accuracy and computational efficiency with sparse and time-varying observations.

A 3D hierarchical cross-modality interaction network using transformers and convolutions for brain glioma segmentation in MR images.

Precise glioma segmentation from multi-parametric magnetic resonance (MR) images is essential for brain glioma diagnosis. However, due to the indistinct boundaries between tumor sub-regions and the heterogeneous appearances of gliomas in volumetric MR scans, designing a reliable and automated glioma segmentation method is still challenging. Although existing 3D Transformer-based or convolution-based segmentation networks have obtained promising results via multi-modal feature fusion strategies or contextual learning methods, they widely lack the capability of hierarchical interactions between different modalities and cannot effectively learn comprehensive feature representations related to all glioma sub-regions. To overcome these problems, in this paper, we propose a 3D hierarchical cross-modality interaction network (HCMINet) using Transformers and convolutions for accurate multi-modal glioma segmentation, which leverages an effective hierarchical cross-modality interaction strategy to sufficiently learn modality-specific and modality-shared knowledge correlated to glioma sub-region segmentation from multi-parametric MR images. In the HCMINet, we first design a hierarchical cross-modality interaction Transformer (HCMITrans) encoder to hierarchically encode and fuse heterogeneous multi-modal features by Transformer-based intra-modal embeddings and inter-modal interactions in multiple encoding stages, which effectively captures complex cross-modality correlations while modeling global contexts. Then, we collaborate an HCMITrans encoder with a modality-shared convolutional encoder to construct the dual-encoder architecture in the encoding stage, which can learn the abundant contextual information from global and local perspectives. Finally, in the decoding stage, we present a progressive hybrid context fusion (PHCF) decoder to progressively fuse local and global features extracted by the dual-encoder architecture, which utilizes the local-global context fusion (LGCF) module to efficiently alleviate the contextual discrepancy among the decoding features. Extensive experiments are conducted on two public and competitive glioma benchmark datasets, including the BraTS2020 dataset with 494 patients and the BraTS2021 dataset with 1251 patients. Results show that our proposed method outperforms existing Transformer-based and CNN-based methods using other multi-modal fusion strategies in our experiments. Specifically, the proposed HCMINet achieves state-of-the-art mean DSC values of 85.33% and 91.09% on the BraTS2020 online validation dataset and the BraTS2021 local testing dataset, respectively. Our proposed method can accurately and automatically segment glioma regions from multi-parametric MR images, which is beneficial for the quantitative analysis of brain gliomas and helpful for reducing the annotation burden of neuroradiologists.

Multi-scale feature extraction and TrasMLP encoder module for ocean HABs segmentation

Due to tiny edge and texture details of harmful algae blooms(HABs), existing segmentation networks are not effective for HABs segmentation. In order to solve the above problems, this paper proposes a Multi-scale Feature extraction and TrasMLP Encoder Fusion-based network (MFTS). To tackle the complex morphological characteristics and the complex backgrounds of HABs, a TrasMlp module which can effectively identify long-range patterns and adapt network parameters is introduced, enabling accurately parsing of complex algae images. Secondly, the deep convolution module is constructed by combining deep separable convolution with a two-channel attention mechanism to separate the target region from the background. In addition, this paper proposes a Weighted Feature Fusion of Deep Convolution and INRS Encoder Module classification network(FDIR) is proposed to quantify the performance of the image segmentation network. The segmentation results on HABs dataset from AICO Lab show that our proposed MFTS model achieves a miou of 90.02%, outperforming the performance of classical segmentation networks such as U-Net and Mask R-CNN. Compared to original HABs dataset, the segmented result shows a 5.1% improvement in the classification accuracy of the FDIR model.

WaveFrSnow: Comprehensive perception wavelet transform frequency separation transformer for image snow removal

Types of snow degradation are complex and diverse. Snow removal often requires the construction of sufficient visual representations. Although convolution-based methods perform well in local perception, they struggle to model globally. On the other hand, methods based on self-attention can capture long-range dependencies but often overlook local information and texture details. In this paper, we proposed a hybrid network called WaveFrSnow, aimed at enhancing the performance of single-image snow removal by combining the advantages of convolution and cross-attention. Firstly, we introduced a frequency-separation cross-attention mechanism based on wavelet transform (WaveFrSA) to enhance the global and texture representations of snow removal. Specifically, frequency-separated attention perceives the texture in the high-frequency branch, captures global information in the low-frequency branch, and introduces convolution to obtain local features. In addition, we constructed local representations through efficient convolutional encoder branches. Furthermore, we develop a Multi-Scale Degradation Aggregation (MSDA) module to integrate rich implicit degradation features. Based on the MSDA module, a Degradation Area Restoration (DAR) network is constructed, aiming to achieve high-quality image restoration following the snow removal process. Taken together, comprehensive experimental results on multiple publicly available datasets demonstrate the superiority of the proposed method over the state-of-the-art method. Additionally, the desnowing results effectively improve the accuracy of downstream vision tasks. The code and datasets in this study are available at https://github.com/dxw2000/WaveFrSnow.

Multi-scale convolutional auto encoder for anomaly detection in 6G environment

With the increasing deployment of 6G networks in all industries, there is a growing risk of vulnerability and complexity due to continuous data flow from edge devices to specialized computers. One potential threat to 6G networks is fuzzing attacks, which involve sending random and invalid inputs to identify vulnerabilities and flaws. A successful fuzzing attack could compromise the network’s protocols, interfaces, and security mechanisms, leading to potential disruption or compromise of critical infrastructure. The proposed framework involves gathering relevant data from different sources within the 6G network and pre-processing it to prepare for analysis. Relevant features that indicate the presence of anomalies are identified using Tuna Swarm Optimization (TSO) Algorithm. The Multi-Scale Convolutional Auto Encoder (MSCAE) is then utilized by the proposed model to extract the feature and classify data. The intrusion Detection System is built to monitor and classify nodes producing anomalies. The proposed model is assessed utilizing various metrics, including recall, precision, accuracy, detection latency, and F1 score. The proposed framework achieved 97.50% accuracy, 94.81% precision, 93.50% F1-score, and 94.50% recall with notable improvements that surmount the deficiencies of previous studies. The results demonstrate that the proposed algorithm is more efficient and safe than current edge security methods, potentially mitigating the risk of fuzzing attacks in 6G networks.

CAEM-GBDT: a cancer subtype identifying method using multi-omics data and convolutional autoencoder network.

The identification of cancer subtypes plays a very important role in the field of medicine. Accurate identification of cancer subtypes is helpful for both cancer treatment and prognosis Currently, most methods for cancer subtype identification are based on single-omics data, such as gene expression data. However, multi-omics data can show various characteristics about cancer, which also can improve the accuracy of cancer subtype identification. Therefore, how to extract features from multi-omics data for cancer subtype identification is the main challenge currently faced by researchers. In this paper, we propose a cancer subtype identification method named CAEM-GBDT, which takes gene expression data, miRNA expression data, and DNA methylation data as input, and adopts convolutional autoencoder network to identify cancer subtypes. Through a convolutional encoder layer, the method performs feature extraction on the input data. Within the convolutional encoder layer, a convolutional self-attention module is embedded to recognize higher-level representations of the multi-omics data. The extracted high-level representations from the convolutional encoder are then concatenated with the input to the decoder. The GBDT (Gradient Boosting Decision Tree) is utilized for cancer subtype identification. In the experiments, we compare CAEM-GBDT with existing cancer subtype identifying methods. Experimental results demonstrate that the proposed CAEM-GBDT outperforms other methods. The source code is available from GitHub at https://github.com/gxh-1/CAEM-GBDT.git.

UET4Rec: U-net encapsulated transformer for sequential recommender

Recommending a tempting sequence of items according to a user’s previous history of purchases and clicks, for instance, in the online shopping portals is challenging. And yet it is a crucial task for all service providers. One of the core components in the recommender systems is a sequential model with which an input sequence is transformed into the predicted items. Among many, deep neural networks, such as RNN, LSTM, and Transformer, have been favored for this purpose. However, improving the performance of these models remains an important task. To address this, we propose a novel sequential model by combining a U-net and Transformer, called U-net Encapsulated Transformer. This hybrid architecture places a transformer model between a convolutional encoder and its decoder, wherein each convolutional layer processes 1D signal, i.e. text. The primary benefit of this structure is that the computational burden is reduced since the embedding size of the input to the Transformer is drastically decreased as the signal has to go through the multi-layer convolutional encoder. This solution leverages recommendation lists for action predictions and uses user feedback as direct rewards for updating the model. In addition, the loss function mechanism is improved by including contrastive and reinforcement learning losses. Evaluation of the proposed model, including extensive ablation study, is carried out on four standard benchmark datasets, such as RC15, RetailRocket, MovieLens-1M, and Amazon-Beauty, demonstrating superior performance compared to state-of-the-art methods.

Joint relational triple extraction with enhanced representation and binary tagging framework in cybersecurity

The cyber threat intelligence (CTI) knowledge graph is a valuable tool for aiding security practitioners in the identification and analysis of cyberattacks. These graphs are constructed from CTI data, organized into relational triples, where each triple comprises two entities linked by a particular relation. However, as the volume of CTI data is expanding at a faster rate than predicted, existing technologies are unable to extract relational triples quickly and accurately. This work mainly focuses on the extraction of relational triples in CTI data, which is achieved by an enhanced representation and binary tagging framework (ERBTF). Firstly, we introduce embedding representations for relations and concatenate these with word embeddings to obtain the initial hidden representation. Subsequently, we employ a novel dilated convolutional encoder that consists of a dilated convolution neural network, gate mechanism and residual connection to enhance the learned contextual representation. Afterwards, we adopt an attention module that includes multi-head self-attention and position-wise feed-forward neural network to allocate greater attention to words that significantly influence the specific relation. Additionally, we utilize the straightforward yet efficient binary entity tagger to identify subject and object entities under different relations for constructing relational triples. We conduct massive experiments on relational triple extraction from CTI data, the results show that ERBTF is superior to the existing relation extraction models, and achieves state-of-the-art performance.

WNet: A dual‐encoded multi‐human parsing network

AbstractIn recent years, multi‐human parsing has become a focal point in research, yet prevailing methods often rely on intermediate stages and lacking pixel‐level analysis. Moreover, their high computational demands limit real‐world efficiency. To address these challenges and enable real‐time performance, low‐latency end‐to‐end network is proposed. This approach leverages vision transformer and convolutional neural network in a dual‐encoded network, featuring a lightweight Transformer‐based vision encoder) and a convolution encoder based on Darknet. This combination adeptly captures long‐range dependencies and spatial relationships. Incorporating a fuse block enables the seamless merging of features from the encoders. Residual connections in the decoder design amplify information flow. Experimental validation on crowd instance‐level human parsing and look into person datasets showcases the WNet's effectiveness, achieving high‐speed multi‐human parsing at 26.7 frames per second. Ablation studies further underscore WNet's capabilities, emphasizing its efficiency and accuracy in complex multi‐human parsing tasks.

CMAF-Net: a cross-modal attention fusion-based deep neural network for incomplete multi-modal brain tumor segmentation.

The information between multimodal magnetic resonance imaging (MRI) is complementary. Combining multiple modalities for brain tumor image segmentation can improve segmentation accuracy, which has great significance for disease diagnosis and treatment. However, different degrees of missing modality data often occur in clinical practice, which may lead to serious performance degradation or even failure of brain tumor segmentation methods relying on full-modality sequences to complete the segmentation task. To solve the above problems, this study aimed to design a new deep learning network for incomplete multimodal brain tumor segmentation. We propose a novel cross-modal attention fusion-based deep neural network (CMAF-Net) for incomplete multimodal brain tumor segmentation, which is based on a three-dimensional (3D) U-Net architecture with encoding and decoding structure, a 3D Swin block, and a cross-modal attention fusion (CMAF) block. A convolutional encoder is initially used to extract the specific features from different modalities, and an effective 3D Swin block is constructed to model the long-range dependencies to obtain richer information for brain tumor segmentation. Then, a cross-attention based CMAF module is proposed that can deal with different missing modality situations by fusing features between different modalities to learn the shared representations of the tumor regions. Finally, the fused latent representation is decoded to obtain the final segmentation result. Additionally, channel attention module (CAM) and spatial attention module (SAM) are incorporated into the network to further improve the robustness of the model; the CAM to help focus on important feature channels, and the SAM to learn the importance of different spatial regions. Evaluation experiments on the widely-used BraTS 2018 and BraTS 2020 datasets demonstrated the effectiveness of the proposed CMAF-Net which achieved average Dice scores of 87.9%, 81.8%, and 64.3%, as well as Hausdorff distances of 4.21, 5.35, and 4.02 for whole tumor, tumor core, and enhancing tumor on the BraTS 2020 dataset, respectively, outperforming several state-of-the-art segmentation methods in missing modalities situations. The experimental results show that the proposed CMAF-Net can achieve accurate brain tumor segmentation in the case of missing modalities with promising application potential.

Convolutional Encoding and Normalizing Flows: A Deep Learning Approach for Offshore Wind Speed Probabilistic Forecasting in the Mediterranean Sea

Abstract The safe and efficient execution of offshore operations requires short-term (1–6 h ahead) high-quality probabilistic forecasts of metocean variables. The development areas for offshore wind projects, potentially in high depths, make it difficult to gather measurement data. This paper explores the use of deep learning for wind speed forecasting at an unobserved offshore location. The proposed convolutional architecture jointly exploits coastal measurements and numerical weather predictions to emulate multivariate probabilistic short-term forecasts. We explore both Gaussian and non-Gaussian neural representations using normalizing flows. We benchmark these approaches with respect to state-of-the-art data-driven schemes, including analog methods and quantile forecasting. The performance of the models and resulting forecast quality are analyzed in terms of probabilistic calibration, probabilistic and deterministic metrics, and as a function of weather situations. We report numerical experiments for a real case study off the French Mediterranean coast. Our results highlight the role of regional numerical weather prediction and coastal in situ measurement in the performance of postprocessing. For single-valued forecasts, a 40% decrease in RMSE is observed compared to the direct use of numerical weather predictions. Significant skill improvements are also obtained for the probabilistic forecasts, in terms of various scores, as well as an acceptable probabilistic calibration. The proposed architecture can process a large amount of heterogeneous input data and offers a versatile probabilistic framework for multivariate forecasting.