Input Patch Research Articles

Recognizing beam profiles from silicon photonics gratings using a transformer model

Over the past decade, there has been extensive work in developing integrated silicon photonics (SiPh) gratings for the optical addressing of trapped ion qubits among the ion trap quantum computing community. However, when viewing beam profiles from gratings using infrared (IR) cameras, it is often difficult to determine the corresponding heights where the beam profiles are located. In this work, we developed transformer models to recognize the corresponding height categories of beam profiles in light from SiPh gratings. The models are trained using two techniques: (1) input patches and (2) input sequence. For the model trained with input patches, the model achieved a recognition accuracy of 0.924. Meanwhile, the model trained with input sequence shows a lower accuracy of 0.892. However, when repeating the model training for 150 cycles, a model trained with input patches shows inconsistent accuracy ranges between 0.289 to 0.959, while the model trained with input sequence shows accuracy values between 0.75 to 0.947. The obtained outcomes can be expanded to various applications, including auto-focusing of light beams and auto-adjustment of the z-axis stage to acquire desired beam profiles.

ETFT: Equiangular Tight Frame Transformer for Imbalanced Semantic Segmentation

Semantic segmentation often suffers from class imbalance, where the label ratio for each class in the dataset is not uniform. Recent studies have addressed the issue of class imbalance in semantic segmentation by leveraging the neural collapse phenomenon in conjunction with an Equiangular Tight Frame (ETF). While the use of ETF aids in enhancing the discriminability of minor classes, class correlation is another crucial factor that must be taken into account. However, managing the balance between class correlation and discrimination through neural collapse remains challenging, as these properties inherently conflict with one another. Moreover, this control is established during the training stage, resulting in a fixed classifier. There is no guarantee that this classifier will consistently perform well with different input images. To address this problem, we propose an Equiangular Tight Frame Transformer (ETFT), a transformer-based model that jointly processes the features and classifier using ETF structure, and dynamically generates the classifier as a function of the input for imbalanced semantic segmentation. Specifically, the classifier initialized with the ETF structure is jointly processed with the input patch tokens during the attention process. As a result, the transformed patch tokens, aided by the ETF structure, achieve discriminability between classes while preserving contextual correlation. The classifier, initially structured as an ETF, is adjusted to incorporate the correlation information, benefiting from the attention mechanism. Furthermore, the learned classifier is combined with the fixed ETF classifier, leveraging the advantages of both. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art methods for imbalanced semantic segmentation on both the ADE20K and Cityscapes datasets.

PatchSkip: A lightweight technique for effectively alleviating over-smoothing in vision transformers

Recently vision transformers (ViTs) have encountered the over-smoothing problem, which reduces their capacity by mapping input patches into a similar latent representation. Existing methods introduce regularization terms to alleviate over-smoothing but often increase computational costs. To address this, this paper proposes PatchSkip, a novel and flexible dropout variant, alleviating the over-smoothing problem of ViTs in a lightweight manner. Specifically, PatchSkip draws inspiration from the fact that a similar over-smoothing problem in GNNs is primarily caused by static adjacent matrices leading to solitary message passing mode between nodes. PatchSkip constructs graphs with patch embeddings and analyzes the adjacent matrices in ViTs. By randomly selecting specific patch embeddings to bypass transformer blocks, PatchSkip is proved to generate various adjacent matrices and acts as a multi-mode message passing engine, providing diverse modes of message passing between patches. The effectiveness of PatchSkip in preventing over-smoothing is demonstrated through theoretical proofs and empirical visualizations. Furthermore, PatchSkip is evaluated on various datasets and backbones, showing significant performance improvements while reducing computational costs. For example, when trained on Tiny-ImageNet from scratch, PatchSkip improves the performance of the vanilla CrossViT by 3.85% while reducing computational costs by over 20%.

Shape-Scale Co-Awareness Network for 3D Brain Tumor Segmentation.

The accurate segmentation of brain tumor is significant in clinical practice. Convolutional Neural Network (CNN)-based methods have made great progress in brain tumor segmentation due to powerful local modeling ability. However, brain tumors are frequently pattern-agnostic, i.e. variable in shape, size and location, which can not be effectively matched by traditional CNN-based methods with local and regular receptive fields. To address the above issues, we propose a shape-scale co-awareness network (S2CA-Net) for brain tumor segmentation, which can efficiently learn shape-aware and scale-aware features simultaneously to enhance pattern-agnostic representations. Primarily, three key components are proposed to accomplish the co-awareness of shape and scale. The Local-Global Scale Mixer (LGSM) decouples the extraction of local and global context by adopting the CNN-Former parallel structure, which contributes to obtaining finer hierarchical features. The Multi-level Context Aggregator (MCA) enriches the scale diversity of input patches by modeling global features across multiple receptive fields. The Multi-Scale Attentive Deformable Convolution (MS-ADC) learns the target deformation based on the multiscale inputs, which motivates the network to enforce feature constraints both in terms of scale and shape for optimal feature matching. Overall, LGSM and MCA focus on enhancing the scale-awareness of the network to cope with the size and location variations, while MS-ADC focuses on capturing deformation information for optimal shape matching. Finally, their effective integration prompts the network to perceive variations in shape and scale simultaneously, which can robustly tackle the variations in patterns of brain tumors. The experimental results on BraTS 2019, BraTS 2020, MSD BTS Task and BraTS2023-MEN show that S2CA-Net has superior overall performance in accuracy and efficiency compared to other state-of-the-art methods. Code: https://github.com/jiangyu945/S2CA-Net.

PATH-13. LEARNED RESIZING WITH EFFICIENT TRAINING (LRET) FACILITATES IMPROVED PERFORMANCE OF LARGE-SCALE BRAIN TUMOR HISTOLOGY IMAGE CLASSIFICATION MODELS

Abstract BACKGROUND Histologic examination is vital in oncology research and diagnostics. The adoption of digital scanning of whole slide images (WSI) has created an opportunity to leverage deep learning-based image classification methods to enhance diagnosis and risk stratification. However, technical limitations prevent training and deployment of accurate comprehensive multiclass deep convolutional neural networks (DCNN) models for histopathology image classification. The input dimensions of DCNN architectures are small compared to the typical pathologist field of view, degrading performance by excluding important architectural features. Furthermore, data requirements for comprehensive models are sufficiently large to overwhelm the system memory during training. METHODS A method termed Learned Resizing with Efficient Training (LRET) was developed to address the main limitations of traditional histopathology classification model training. The LRET method couples efficient training techniques with image resizing to facilitate seamless integration of larger histology image patches into state-of-the-art classification models while preserving important structural information. The LRET method was coupled with two distinct resizing techniques to train three diverse histology image datasets using five different DCNN architectures. Performance metrics were compared on cross validation and hold out test sets. RESULTS LRET-trained models were flexible to multiple input patch dimensions and DCNN models. We demonstrated performance improvement across all datasets while significantly reducing the training time and resources over traditional methods. Using a large-scale, multiclass brain tumor classification dataset consisting of 74 distinct histopathologic classes, LRET-trained models outperformed existing methods by 15-28% in accuracy, yielding 94% accuracy for the best model. CONCLUSION The LRET method for DCNN training significantly enhances the performance of large-scale multiclass histopathology image classification. The implications of this work extend to broader applications within medical imaging and beyond, where efficient integration of high-resolution images into deep learning pipelines is paramount for driving advancements research and clinical practice.

An Innovative Approach: Sea Ice Types Classification Using Convolutional Neural Networks with DDDTDWT Filter

Studying sea ice and its interaction with climate change is crucial due to its significant impact on the environment, society, and global stability. The pressing need to address the underlying reasons for the rapid melting of Arctic and Antarctic sea ice is underscored by its adverse effects on the environment and society. In this proposed study, a Convolutional Neural Network (CNN) is utilized to predict ice types using data from the NSIDC DAAC Advanced Microwave Scanning Radiometer - Earth Observing System Sensor (AMSR-E) collection. This dataset contains parameters such as sea ice types and spans data products from June 2002, obtained from the NASA Data Centre. By employing hand-crafted features as input and a single layer of hidden nodes, the CNN used in this approach generates improved estimates of ice types, outperforming traditional image analysis methods. At each stage, ConvNets use diverse filter banks, feature extraction pooling layers, and fully connected layers with basic activation functions like Relu. This allows the network to build multifaceted hierarchies of features. The sea ice type estimates produced by the CNN are then compared with those obtained from passive microwave brightness temperature data using existing algorithms as well as a proposed CNN algorithm, resulting in an increased classification accuracy of 98.58%. This improvement is particularly notable in the reduction of the error rate, which has been effectively minimized from 3.01% without feature selection to 1.42% with infinite feature selection. When compared to existing algorithms, the CNN demonstrates superior performance. These findings underscore the impact of input patch size, CNN layer count, and input size on the model's performance.

Pattern classification of interstitial lung diseases from computed tomography images using a ResNet-based network with a split-transform-merge strategy and split attention.

In patients with interstitial lung disease (ILD), accurate pattern assessment from their computed tomography (CT) images could help track lung abnormalities and evaluate treatment efficacy. Based on excellent image classification performance, convolutional neural networks (CNNs) have been massively investigated for classifying and labeling pathological patterns in the CT images of ILD patients. However, previous studies rarely considered the three-dimensional (3D) structure of the pathological patterns of ILD and used two-dimensional network input. In addition, ResNet-based networks such as SE-ResNet and ResNeXt with high classification performance have not been used for pattern classification of ILD. This study proposed a SE-ResNeXt-SA-18 for classifying pathological patterns of ILD. The SE-ResNeXt-SA-18 integrated the multipath design of the ResNeXt and the feature weighting of the squeeze-and-excitation network with split attention. The classification performance of the SE-ResNeXt-SA-18 was compared with the ResNet-18 and SE-ResNeXt-18. The influence of the input patch size on classification performance was also evaluated. Results show that the classification accuracy was increased with the increase of the patch size. With a 32 × 32 × 16 input, the SE-ResNeXt-SA-18 presented the highest performance with average accuracy, sensitivity, and specificity of 0.991, 0.979, and 0.994. High-weight regions in the class activation maps of the SE-ResNeXt-SA-18 also matched the specific pattern features. In comparison, the performance of the SE-ResNeXt-SA-18 is superior to the previously reported CNNs in classifying the ILD patterns. We concluded that the SE-ResNeXt-SA-18 could help track or monitor the progress of ILD through accuracy pattern classification.

Rethinking Attention Mechanisms in Vision Transformers with Graph Structures.

In this paper, we propose a new type of vision transformer (ViT) based on graph head attention (GHA). Because the multi-head attention (MHA) of a pure ViT requires multiple parameters and tends to lose the locality of an image, we replaced MHA with GHA by applying a graph to the attention head of the transformer. Consequently, the proposed GHA maintains both the locality and globality of the input patches and guarantees the diversity of the attention. The proposed GHA-ViT commonly outperforms pure ViT-based models using small-sized CIFAR-10/100, MNIST, and MNIST-F datasets and a medium-sized ImageNet-1K dataset in scratch training. A Top-1 accuracy of 81.7% was achieved for ImageNet-1K using GHA-B, which is a base model with approximately 29 M parameters. In addition, with CIFAR-10/100, the existing ViT and parameters are reduced 17-fold and the performance increased by 0.4/4.3%, respectively. The proposed GHA-ViT shows promising results in terms of the number of parameters and operations and the level of accuracy in comparison with other state-of-the-art ViT-lightweight models.

Medical image synthesis via conditional GANs: Application to segmenting brain tumours

Accurate brain tumour segmentation is critical for tasks such as surgical planning, diagnosis, and analysis, with magnetic resonance imaging (MRI) being the preferred modality due to its excellent visualisation of brain tissues. However, the wide intensity range of voxel values in MR scans often results in significant overlap between the density distributions of different tumour tissues, leading to reduced contrast and segmentation accuracy. This paper introduces a novel framework based on conditional generative adversarial networks (cGANs) aimed at enhancing the contrast of tumour subregions for both voxel-wise and region-wise segmentation approaches. We present two models: Enhancement and Segmentation GAN (ESGAN), which combines classifier loss with adversarial loss to predict central labels of input patches, and Enhancement GAN (EnhGAN), which generates high-contrast synthetic images with reduced inter-class overlap. These synthetic images are then fused with corresponding modalities to emphasise meaningful tissues while suppressing weaker ones. We also introduce a novel generator that adaptively calibrates voxel values within input patches, leveraging fully convolutional networks. Both models employ a multi-scale Markovian network as a GAN discriminator to capture local patch statistics and estimate the distribution of MR images in complex contexts. Experimental results on publicly available MR brain tumour datasets demonstrate the competitive accuracy of our models compared to current brain tumour segmentation techniques

Automated lung tumor segmentation robust to various tumor sizes using a consistency learning-based multi-scale dual-attention network.

It is often difficult to automatically segment lung tumors due to the large tumor size variation ranging from less than 1 cm to greater than 7 cm depending on the T-stage. This study aims to accurately segment lung tumors of various sizes using a consistency learning-based multi-scale dual-attention network (CL-MSDA-Net). To avoid under- and over-segmentation caused by different ratios of lung tumors and surrounding structures in the input patch according to the size of the lung tumor, a size-invariant patch is generated by normalizing the ratio to the average size of the lung tumors used for the training. Two input patches, a size-invariant patch and size-variant patch are trained on a consistency learning-based network consisting of dual branches that share weights to generate a similar output for each branch with consistency loss. The network of each branch has a multi-scale dual-attention module that learns image features of different scales and uses channel and spatial attention to enhance the scale-attention ability to segment lung tumors of different sizes. In experiments with hospital datasets, CL-MSDA-Net showed an F1-score of 80.49%, recall of 79.06%, and precision of 86.78%. This resulted in 3.91%, 3.38%, and 2.95% higher F1-scores than the results of U-Net, U-Net with a multi-scale module, and U-Net with a multi-scale dual-attention module, respectively. In experiments with the NSCLC-Radiomics datasets, CL-MSDA-Net showed an F1-score of 71.7%, recall of 68.24%, and precision of 79.33%. This resulted in 3.66%, 3.38%, and 3.13% higher F1-scores than the results of U-Net, U-Net with a multi-scale module, and U-Net with a multi-scale dual-attention module, respectively. CL-MSDA-Net improves the segmentation performance on average for tumors of all sizes with significant improvements especially for small sized tumors.

Oil species identification based on fluorescence excitation-emission matrix and transformer-based deep learning

After oil spills are found at sea, the identification on oil species can help determine the source of leakage and form the plan of post-accident treatment. Since the fluorometric characteristics of petroleum hydrocarbon reflect its molecular structure, the composition of oil spills could potentially be inferred using the fluorescence spectroscopy method. The excitation-emission matrix (EEM) includes additional fluorescence information in the dimension of excitation wavelength, which could be useful to identify oil species. This study proposed an oil species identification model using transformer network. The EEMs of oil pollutants are reconstructed into sequenced patch input that consists of the fluorometric spectra obtained under the different excitation wavelengths. The comparative experiments show that the proposed model can reduce the incorrect predictions and achieve higher identification accuracies than the regular convolutional neural networks that have been used in the previous studies. According to the structure of transformer network, an ablation experiment is also designed to evaluate the contributions of different input patches and seek for the optimal excitation wavelengths for oil species identification. The proposed model is expected to identify oil species, and even other fluorescent materials, based on the fluorometric spectra collected under multiple excitation wavelengths.

Shaped patch based nonparametric discriminant analysis for hyperspectral image classification through the CNN model

ABSTRACT Convolutional neural network (CNN) has shown superior classification results for hyperspectral image classification. Due to high dimensionality of the hyperspectral image, usually a feature reduction method is applied to the hyperspectral data before feeding it to the CNN model. Principal component analysis is the most widely used approach to this end. However, it has two main disadvantages. It uses the mean square error measure and ignores the class separability. Moreover, it is a pixel-based method, which does not utilize the spatial information. To deal with these weaknesses, the patch-based nonparametric discriminant analysis method is proposed for hyperspectral image feature reduction in this work. However, due to the complex and heterogeneous nature of the hyperspectral images, the use of fixed square patches is not so appropriate. To deal with this difficulty, a shaping approach is introduced for the generation of adaptive patches with consistent shapes in each local region. The input patches of the CNN model are also weighted according to the spectral similarity and class boundaries information. Moreover, a simple residual CNN model is suggested for the classification of the extracted shaped input patches. The efficiency of the proposed framework is assessed in different cases using various evaluation measures on six real hyperspectral images.

Improved Image Quality Assessment by Utilizing Pre-Trained Architecture Features with Unified Learning Mechanism

The purpose of the no-reference image quality assessment (NR-IQA) is to measure perceived image quality based on subjective judgments; however, due to the lack of a clean reference image, this is a complicated and unresolved challenge. Massive new IQA datasets have facilitated the creation of deep learning-based image quality measurements. We present a unique model to handle the NR-IQA challenge in this research by employing a hybrid strategy that leverages from pre-trained CNN model and the unified learning mechanism that extracts both local and non-local characteristics from the input patch. The deep analysis of the proposed framework shows that the model uses features and a mechanism that improves the monotonicity relationship between objective and subjective ratings. The intermediary goal was mapped to a quality score using a regression architecture. To extract various feature maps, a deep architecture with an adaptive receptive field was used. Analyses of this biggest NR-IQA benchmark datasets demonstrate that the suggested technique outperforms current state-of-the-art NR-IQA measures.

StereoPairFree: Self-Constructed Stereo Correspondence Network From Natural Images

This article proposes a variant of a convolutional neural network-based stereo matching method, StereoPairFree, that requires only normal and not left or right stereo images. The corresponding image patches for training stereo matching cost networks look similar, apart from some properties, such as different illumination, occlusion, distortion, and foreshortening. We propose a method for generating synthesized pairs of corresponding image patches for a given image patch. We also propose a multipatch matching cost network that exploits various input patch sizes. The proposed matching cost network is optimized by cross-based cost aggregation and semiglobal matching, followed by consistency checks and bilateral filtering. Hence, StereoPairFree does not require a single stereo pair but standard images to build a deep learning stereo matching method. It is the first stereo matching method constructed from normal images and significantly outperforms monocular stereo matching approaches (Hur and Roth, 2020), (Hur and Roth, 2021), (Schuster et al., 2020). We evaluated StereoPairFree with the KITTI-2012, KITTI-2015, and Middlebury datasets. It significantly outperformed the baseline and several deep learning-based methods using these datasets.

Context-Patch Representation Learning With Adaptive Neighbor Embedding for Robust Face Image Super-Resolution

Representation learning steered robust face image super-resolution (FSR) methods have attracted extensive attention in the past few decades. Most previous methods were devoted to exploiting the local position patches in the training set for FSR. However, they usually overlooked the sufficient usage of the contextual information around the testing patches, which are useful for stable representation learning. In this paper, we attempt to utilize the context-patch around the testing patch and propose a method named context-patch representation learning with adaptive neighbor embedding (CRL-ANE) for FSR. On one hand, we simultaneously use the testing position patch and its adjacent ones for stable representation weight learning. This contextual information can compensate for recovering missing details in the target patch. On the other hand, for each input patch set, due to its inherent facial structural properties, we design an adaptive neighbor embedding strategy to elaborately and adaptively choose primary candidates for more accurate reconstruction. These two improvements enable the proposed method to achieve better SR performance than some of the other methods. Qualitative and quantitative experiments on some benchmarks have validated the superiority of the proposed method over some state-of-the-art methods.

Cross-Scale KNN Image Transformer for Image Restoration

Numerous image restoration approaches have been proposed based on attention mechanism, achieving superior performance to convolutional neural networks (CNNs) based counterparts. However, they do not leverage the attention model in a form fully suited to the image restoration tasks. In this paper, we propose an image restoration network with a novel attention mechanism, called cross-scale <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -NN image Transformer (CS-KiT), that effectively considers several factors such as locality, non-locality, and cross-scale aggregation, which are essential to the image restoration. To achieve locality and non-locality, the CS-KiT builds <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -nearest neighbor relation of local patches and aggregates similar patches through a local attention. To induce cross-scale aggregation, we ensure that each local patch embraces different scale information with scale-aware patch embedding (SPE) which predicts an input patch scale through a combination of multi-scale convolution branches. We show the effectiveness of the CS-KiT with experimental results, outperforming state-of-the-art restoration approaches on image denoising, deblurring, and deraining benchmarks.

TriTF: A Triplet Transformer Framework Based on Parents and Brother Attention for Hyperspectral Image Change Detection

Hyperspectral image (HSI) change detection (CD) is a technique to accurately detect land cover changes by using HSIs with rich spatial-spectral information. In recent years, the HSI-CD methods based on convolutional neural networks (CNNs) have achieved great success because of their flexible and effective feature extraction ability. However, these methods often take the HSI patches as the input of the networks, which undoubtedly hinders the overall perception of the HSIs. Meanwhile, the valuable temporal information in HSIs is often underutilized. For this end, a triplet transformer framework (TriTF) based on parents-temporal attention and brother-spatial attention is proposed for HSI-CD. The proposed framework mainly contains the following three parts: 1) Transformer-based network backbone, which uses the self-attention to capture the correlation between arbitrarily two pixels in the same patch and extracts the global spatial correlation in the unit of encoded input patches; 2) parents-temporal attention (PTA) branch. Unlike the previous cross-temporal attention mechanisms of the “T1↔T2” mode which only consider the interaction between bi-temporal HSIs, this paper constructs a novel PTA of the “T1→T3←T2” mode which takes the difference-temporal image T3 as the core. The impact of bi-temporal HSIs on the land cover changes is more concerned in the PTA; 3) brother-spatial attention (BSA) branch. The most similar patch in the current training batch of each patch is defined as its brother patch. Furthermore, cross-spatial attention is applied to propagate the features of the brother patch to the current patch. Thus, the middle- and long-range dependencies can be utilized and the scope of feature propagation can be extended. In this paper, the experiments under low and high sampling rates are conducted and proved the outstanding change detection performance of the proposed TriTF when compared with abundant state-of-the-art (SOTA) CD algorithms. The source code of this paper will be released at https://github.com/zkylnnu/TriTF.

NIMG-61. IMPROVED GENERALIZABILITY OF RADIOPATHOMIC PROBABILISTIC MAPPING OF TREATMENT-INDUCED EFFECTS WITH PHYSIOLOGIC MR IMAGING AND DEEP LEARNING IN PATIENTS WITH RECURRENT GLIOBLASTOMA

Abstract Although physiologic (diffusion-weighted and perfusion-weighted) MRI has shown promise in identifying regions of recurrent tumor (rTumor) in patients with glioblastoma suspected of progression, distinguishing treatment-induced effects (TxE) from rTumor on anatomical MRI remains a challenge. Whereas prior larger-scale machine learning (ML)-based studies mostly utilize anatomical imaging alone and/or perform lesion-level predictions, this study aimed to develop a non-invasive, radiopathomic tool for regional probabilistic mapping of TxE using 208 tissue-samples (55 pathologically-confirmed TxE, 153 recurrent glioblastoma) acquired from 107 patients with known spatial coordinates on pre-surgical MRI. We tested the hypothesis that applying a deep-learning (DL) model that included physiological MRI can: 1) more accurately identify areas of TxE that mimic rTumor on anatomical MRI and 2) better generalize to an independent test set than ML-models or a DL-model that uses anatomical MRI alone. An 80/20 split for training/validation was used after 1/3 of the patients were withheld for testing. Oversampling of TxE samples was employed to address class imbalance and an equal proportion of TxE samples was maintained across all datasets. Three ML-models, their ensemble, and a deep 4D-convolutional-neural-network were trained based on normalized anatomical (post-contrast T1, T2-FLAIR), diffusion-weighted (ADC, FA), and DSC perfusion-weighted (PeakHeight, %recovery) images cropped to 10mm-cubic patches centered on the coordinates from where tissue was obtained. Although Random Forest and voting-ensembled ML-models using all imaging and the anatomical DL-model had the best validation performance (AUC=0.81-0.82), these models did not generalize (test AUC=0.58-0.59). The DL-model including physiologic images had slightly lower validation AUC (0.78) but the best overall test AUC (0.795), indicating superior generalizability. Elevated blood volume (nPeakHeight) was the most important feature. Our DL-model’s interpretability was also demonstrated by disrupting class separation after shuffling voxels within each input patch. These results suggest that using deep-learning with physiologic MRI can improve intratumoral classification of TxE from rTumor.

A multi‐data training method for a deep neural network to improve the separation effect of simultaneous‐source data

AbstractWithin the field of seismic data acquisition with active sources, the technique of acquiring simultaneous data, also known as blended data, offers operational advantages. The preferred processing of blended data starts with a step of deblending, that is separation of the data acquired by the different sources, to produce data that mimic data from a conventional seismic acquisition and can be effectively processed by standard methods. Recently, deep learning methods based on the deep neural network have been applied to the deblending task with promising results, in particular using an iterative approach. We propose an enhancement to deblending with an iterative deep neural network, whereby we modify the training stage of the deep neural network in order to achieve better performance through the iterations. We refer to the method that only uses the blended data as the input data as the general training method. Our new multi‐data training method allows the deep neural network to be trained by the data set with the input patches composed of blended data, noisy data with low amplitude crosstalk noise, and unblended data, which can improve the ability of the deep neural network to remove crosstalk noise and protect weak signal. Based on such an extended training data set, the multi‐data training method embedded in the iterative separation framework can result in different outputs at different iterations and converge to the best result in a shorter iteration number. Transfer learning can further improve the generalization and separation efficacy of our proposed method to deblend the simultaneous‐source data. Our proposed method is tested on two synthetic data and two field data to prove the effectiveness and superiority in the deblending of the simultaneous‐source data compared with the general training method, generic noise attenuation network and low‐rank matrix factorization methods.

Mixture 2D Convolutions for 3D Medical Image Segmentation.

Three-dimensional (3D) medical image segmentation plays a crucial role in medical care applications. Although various two-dimensional (2D) and 3D neural network models have been applied to 3D medical image segmentation and achieved impressive results, a trade-off remains between efficiency and accuracy. To address this issue, a novel mixture convolutional network (MixConvNet) is proposed, in which traditional 2D/3D convolutional blocks are replaced with novel MixConv blocks. In the MixConv block, 3D convolution is decomposed into a mixture of 2D convolutions from different views. Therefore, the MixConv block fully utilizes the advantages of 2D convolution and maintains the learning ability of 3D convolution. It acts as 3D convolutions and thus can process volumetric input directly and learn intra-slice features, which are absent in the traditional 2D convolutional block. By contrast, the proposed MixConv block only contains 2D convolutions; hence, it has significantly fewer trainable parameters and less computation budget than a block containing 3D convolutions. Furthermore, the proposed MixConvNet is pre-trained with small input patches and fine-tuned with large input patches to improve segmentation performance further. In experiments on the Decathlon Heart dataset and Sliver07 dataset, the proposed MixConvNet outperformed the state-of-the-art methods such as UNet3D, VNet, and nnUnet.