Anisotropic Convolution Research Articles

A mixed Mamba U-net for prostate segmentation in MR images

The diagnosis of early prostate cancer depends on the accurate segmentation of prostate regions in magnetic resonance imaging (MRI). However, this segmentation task is challenging due to the particularities of prostate MR images themselves and the limitations of existing methods. To address these issues, we propose a U-shaped encoder-decoder network MM-UNet based on Mamba and CNN for prostate segmentation in MR images. Specifically, we first proposed an adaptive feature fusion module based on channel attention guidance to achieve effective fusion between adjacent hierarchical features and suppress the interference of background noise. Secondly, we propose a global context-aware module based on Mamba, which has strong long-range modeling capabilities and linear complexity, to capture global context information in images. Finally, we propose a multi-scale anisotropic convolution module based on the principle of parallel multi-scale anisotropic convolution blocks and 3D convolution decomposition. Experimental results on two public prostate MR image segmentation datasets demonstrate that the proposed method outperforms competing models in terms of prostate segmentation performance and achieves state-of-the-art performance. In future work, we intend to enhance the model's robustness and extend its applicability to additional medical image segmentation tasks.

An optical flow estimation method based on multiscale anisotropic convolution

An optical flow estimation method based on multiscale anisotropic convolution

MDM-U-Net: A novel network for renal cancer structure segmentation.

Accurate segmentation of the renal cancer structure, including the kidney, renal tumors, veins, and arteries, has great clinical significance, which can assist clinicians in diagnosing and treating renal cancer. For accurate segmentation of the renal cancer structure in contrast-enhanced computed tomography (CT) images, we proposed a novel encoder-decoder structure segmentation network named MDM-U-Net comprising a multi-scale anisotropic convolution block, dual activation attention block, and multi-scale deep supervision mechanism. The multi-scale anisotropic convolution block was used to improve the feature extraction ability of the network, the dual activation attention block as a channel-wise mechanism was used to guide the network to exploit important information, and the multi-scale deep supervision mechanism was used to supervise the layers of the decoder part for improving segmentation performance. In this study, we developed a feasible and generalizable MDM-U-Net model for renal cancer structure segmentation, trained the model from the public KiPA22 dataset, and tested it on the KiPA22 dataset and an in-house dataset. For the KiPA22 dataset, our method ranked first in renal cancer structure segmentation, achieving state-of-the-art (SOTA) performance in terms of 6 of 12 evaluation metrics (3 metrics per structure). For the in-house dataset, our method achieves SOTA performance in terms of 9 of 12 evaluation metrics (3 metrics per structure), demonstrating its superiority and generalization ability over the compared networks in renal structure segmentation from contrast-enhanced CT scans.

FV-Seg-Net: Fully Volumetric Network for Accurate Segmentation of COVID-19 Lesions From Chest CT Scans

Automated and precise pneumonia segmentation of COVID-19 extends the view of medical supply chains and offers crucial medical supplies to fight the COVID-19 pandemic. Deep learning plays a vital role in improving the COVID-19 segmentation from computed tomography (CT) scans. However, the literature lacks a precise segmentation approach on small-size lesions because they often split the CT scan into 2-D slices or 3-D patches, leading to the loss of contextual and/or global information. In order to address this, this article proposes a novel fully volumetric segmentation network, called FV-Seg-Net, that effectively exploits the local and global spatial information and enables the entire CT volume processing at once. The decoder is designed using a computationally efficient recalibrated anisotropic convolution module that can acquire the 3-D semantic representation of the CT volumes with anisotropic resolution. To avoid losing information during down-sampling, we reconstruct the skip-connection using a multilevel multiscale pyramid aggregation module and ensure more effective context fusion that improves the reconstruction capability of the decoder. Finally, stacked data augmentation (StackAug) is presented to magnify the training data and improve the generalizability of FV-Seg-Net. Proof of concept experiments on two public datasets demonstrates that the FV-Seg-Net achieves excellent segmentation performance (Dice score: 85.69 and a surface-dice: 84.79%), outperforming the current cutting-edge studies.

Nonlocal Traffic Models with General Kernels: Singular Limit, Entropy Admissibility, and Convergence Rate

Nonlocal conservation laws (the signature feature being that the flux function depends on the solution through the convolution with a given kernel) are extensively used in the modeling of vehicular traffic. In this work we discuss the singular local limit, namely the convergence of the nonlocal solutions to the entropy admissible solution of the conservation law obtained by replacing the convolution kernel with a Dirac delta. Albeit recent counter-examples rule out convergence in the general case, in the specific framework of traffic models (with anisotropic convolution kernels) the singular limit has been established under rigid assumptions, i.e. in the case of the exponential kernel (which entails algebraic identities between the kernel and its derivatives) or under fairly restrictive requirements on the initial datum. In this work we obtain general convergence results under assumptions that are entirely natural in view of applications to traffic models, plus a convexity requirement on the convolution kernels. We then provide a general criterion for entropy admissibility of the limit and a convergence rate. We also exhibit a counter-example showing that the convexity assumption is necessary for our main compactness estimate.

Contour-Aware Equipotential Learning for Semantic Segmentation

With increasing demands for high-quality semantic segmentation in the industry, hard-distinguishing semantic boundaries have posed a significant threat to existing solutions. Inspired by real-life experience, i.e., combining varied observations contributes to higher visual recognition confidence, we present the equipotential learning (EPL) method. This novel module transfers the predicted/ground-truth semantic labels to a self-defined potential domain to learn and infer decision boundaries along customized directions. The conversion to the potential domain is implemented via a lightweight differentiable anisotropic convolution that neither incurs the parameter overhead of the neural networks nor changes to the architectures. Besides, the designed two loss functions, the point loss and the equipotential line loss implement anisotropic field regression and category-level contour learning, respectively, enhancing prediction consistencies in the inter/intra-class boundary areas. More importantly, EPL is agnostic to network architectures, and thus it can be plugged into most existing segmentation models. This paper is the first attempt to address the boundary segmentation problem with field regression and contour learning. Meaningful performance improvements on Pascal Voc 2012 and Cityscapes demonstrate that the proposed EPL module can benefit the off-the-shelf fully convolutional network models when recognizing semantic boundary areas. Besides, intensive comparisons and analysis show the favorable merits of EPL for distinguishing semantically-similar and irregular-shaped categories.

DeltaConv

Learning from 3D point-cloud data has rapidly gained momentum, motivated by the success of deep learning on images and the increased availability of 3D data. In this paper, we aim to construct anisotropic convolution layers that work directly on the surface derived from a point cloud. This is challenging because of the lack of a global coordinate system for tangential directions on surfaces. We introduce DeltaConv, a convolution layer that combines geometric operators from vector calculus to enable the construction of anisotropic filters on point clouds. Because these operators are defined on scalar- and vector-fields, we separate the network into a scalar- and a vector-stream, which are connected by the operators. The vector stream enables the network to explicitly represent, evaluate, and process directional information. Our convolutions are robust and simple to implement and match or improve on state-of-the-art approaches on several benchmarks, while also speeding up training and inference.

High-resolution synthetic aperture radar image classification using multi-scale anisotropic convolutional sparse coding

The classification of high-resolution (HR) synthetic aperture radar (SAR) image is of great significance to SAR scene interpretation and understanding. However, the HR SAR image contains rich ground information and complex spatial structural features. The extraction of effective distinguishing features under the influence of coherent speckle is still a challenging task. A multi-scale anisotropic convolution sparse coding (MACSC) algorithm is proposed for HR SAR image classification. First, the low-order statistical features are extracted to improve the richness of SAR image. Then an unsupervised MACSC model is utilized to learn a set of sparse feature maps and convolution filters for the above statistical features by solving the designed objective function. In MACSC, the multi-scale anisotropic Gaussian kernels are utilized to initialize the dictionary of MACSC. Compared with the isotropic Gaussian kernel of the original CSC, these kernels can more accurately describe the details of different directions and scales in the SAR scene. Meanwhile, the adaptive sparse control factor is introduced into the MACSC model, which can make the learned sparse feature maps suppress the speckle and capture the abundant edge and texture information of the SAR image. After that, the aggregation operation is conducted on multi-scale and multi-direction sparse feature maps to integrate the neighbor pixels and reduce the computing burden. Finally, the obtained feature vector is input into the support vector machine classifier to realize classification. Experiments on three challenging SAR data sets demonstrate that MACSC achieves better classification performance than related sparse representation methods.

Anisotropic Convolutional Neural Networks for RGB-D Based Semantic Scene Completion.

Semantic scene completion (SSC) is a computer vision task aiming to simultaneously infer the occupancy and semantic labels for each voxel in a scene from partial information consisting of a depth image and/or a RGB image. As a voxel-wise labeling task, the key for SSC is how to effectively model the visual and geometrical variations to complete the scene. To this end, we propose the Anisotropic Network (AIC-Net), with novel convolutional modules that can model varying anisotropic receptive fields voxel-wisely in a computationally efficient manner. The basic idea to achieve such anisotropy is to decompose 3D convolution into three consecutive dimensional convolutions, and determine the dimension-wise kernels on the fly. One module, termed kernel-selection anisotropic (KSA) convolution, adaptively selects the optimal kernel sizes for each dimensional convolution from a set of candidate kernels, and the other module, termed kernel-modulation anisotropic (KMA) convolution, directly modulates a single convolutional kernel for each dimension to derive more flexible receptive field. By stacking multiple such anisotropic modules, the 3D context modeling capability and flexibility can be further enhanced. Moreover, we present a new end-to-end trainable framework to approach the SSC task avoiding the expensive TSDF pre-processing as in many existing methods. Extensive experiments on SSC benchmarks show the advantage of the proposed methods.

Anisotropic Convolution for Image Classification.

Convolutional neural networks are built upon simple but useful convolution modules. The traditional convolution has a limitation on feature extraction and object localization due to its fixed scale and geometric structure. Besides, the loss of spatial information also restricts the networks' performance and depth. To overcome these limitations, this paper proposes a novel anisotropic convolution by adding a scale factor and a shape factor into the traditional convolution. The anisotropic convolution augments the receptive fields flexibly and dynamically depending on the valid sizes of objects. In addition, the anisotropic convolution is a generalized convolution. The traditional convolution, dilated convolution and deformable convolution can be viewed as its special cases. Furthermore, in order to improve the training efficiency and avoid falling into a local optimum, this paper introduces a simplified implementation of the anisotropic convolution. The anisotropic convolution can be applied to arbitrary convolutional networks and the enhanced networks are called ACNs (anisotropic convolutional networks). Experimental results show that ACNs achieve better performance than many state-of-the-art methods and the baseline networks in tasks of image classification and object localization, especially in classification task of tiny images.

Outline Feature Extraction of Positron Image Based on a 3D Anisotropic Convolution Operator

This study investigates the application of positron annihilation techniques to the interior of dense metal cavities for Three-dimensional (3D) imaging and extraction of contour features inside the cavities. A feature extraction algorithm based on a 3D anisotropic convolutional operator is proposed for profile feature extraction in low-resolution, low-contrast, and low-signal-to-noise positron image. First, aiming at the problem of positron image noise caused by inconsistent detectors and metal scattering effects, an image preprocessing algorithm combining filtering and full pixel correction is proposed. A 3D anisotropic convolution operator is then designed to extract contour features. To solve the contour feature discontinuity in the extracted contour feature, a 3D path search algorithm is proposed to obtain the centroid coordinate set of the contour feature, and then the centroid coordinate set is subjected to 3D curve fitting to obtain a smooth and continuous contour feature. The study is carried out on the raw 16-bit Digital Imaging and Communications in Medicine (DICOM) data of the positron image, and the data are processed from a 3D perspective, taking full advantage of the correlation between slices in the 3D positron image. In the actual testing, positron images with different kinds of foreign objects in the cavity are extracted using a 3D anisotropic convolution operator, and the contour feature extraction resolution reaches 2 mm.

DeepBranch: Deep Neural Networks for Branch Point Detection in Biomedical Images.

Morphology reconstruction of tree-like structures in volumetric images, such as neurons, retinal blood vessels, and bronchi, is of fundamental interest for biomedical research. 3D branch points play an important role in many reconstruction applications, especially for graph-based or seed-based reconstruction methods and can help to visualize the morphology structures. There are a few hand-crafted models proposed to detect the branch points. However, they are highly dependent on the empirical setting of the parameters for different images. In this paper, we propose a DeepBranch model for branch point detection with two-level designed convolutional networks, a candidate region segmenter and a false positive reducer. On the first level, an improved 3D U-Net model with anisotropic convolution kernels is employed to detect initial candidates. Compared with the traditional sliding window strategy, the improved 3D U-Net can avoid massive redundant computations and dramatically speed up the detection process by employing dense-inference with fully convolutional neural networks (FCN). On the second level, a method based on multi-scale multi-view convolutional neural networks (MSMV-Net) is proposed for false positive reduction by feeding multi-scale views of 3D volumes into multiple streams of 2D convolution neural networks (CNNs), which can take full advantage of spatial contextual information as well as fit different sizes. Experiments on multiple 3D biomedical images of neurons, retinal blood vessels and bronchi confirm that the proposed 3D branch point detection method outperforms other state-of-the-art detection methods, and is helpful for graph-based or seed-based reconstruction methods.

B-separable anisotropic convolution operators and applications

In this paper, the B-separability properties of anisotropic convolution-operators are investigated. In particular, the sharp estimates for the resolvent are established. Thus these operators are positive and also are generators of analytic semigroups. Moreover, we find sufficient conditions that guarantee the maximal B-regularity of the Cauchy problem for a parabolic convolution equation. Finally, the convolution equation is applied to establish maximal regularity properties for an anisotropic integro-differential equations and their infinite systems.

Fast anisotropic gauss filtering

We derive the decomposition of the anisotropic Gaussian in a one-dimensional (1-D) Gauss filter in the x-direction followed by a 1-D filter in a nonorthogonal direction phi. So also the anisotropic Gaussian can be decomposed by dimension. This appears to be extremely efficient from a computing perspective. An implementation scheme for normal convolution and for recursive filtering is proposed. Also directed derivative filters are demonstrated. For the recursive implementation, filtering an 512 x 512 image is performed within 40 msec on a current state of the art PC, gaining over 3 times in performance for a typical filter, independent of the standard deviations and orientation of the filter. Accuracy of the filters is still reasonable when compared to truncation error or recursive approximation error. The anisotropic Gaussian filtering method allows fast calculation of edge and ridge maps, with high spatial and angular accuracy. For tracking applications, the normal anisotropic convolution scheme is more advantageous, with applications in the detection of dashed lines in engineering drawings. The recursive implementation is more attractive in feature detection applications, for instance in affine invariant edge and ridge detection in computer vision. The proposed computational filtering method enables the practical applicability of orientation scale-space analysis.