Adaptive Convolution Research Articles

VCU-Net: a vascular convolutional network with feature splicing for cerebrovascular image segmentation.

Cerebrovascular image segmentation is one of the crucial tasks in the field of biomedical image processing. Due to the variable morphology of cerebral blood vessels, the traditional convolutional kernel is weak in perceiving the structure of elongated blood vessels in the brain, and it is easy to lose the feature information of the elongated blood vessels during the network training process. In this paper, a vascular convolutional U-network (VCU-Net) is proposed to address these problems. This network utilizes a new convolution (vascular convolution) instead of the traditional convolution kernel, to extract features of elongated blood vessels in the brain with different morphologies and orientations by adaptive convolution. In the network encoding stage, a new feature splicing method is used to combine the feature tensor obtained through vascular convolution with the original tensor to provide richer feature information. Experiments show that the DSC and IOU of the proposed method are 53.57% and 69.74%, which are improved by 2.11% and 2.01% over the best performance of the GVC-Net among several typical models. In image visualization, the proposed network has better segmentation performance for complex cerebrovascular structures, especially in dealing with elongated blood vessels in the brain, which shows better integrity and continuity.

A novel convolution transformer-based network for histopathology-image classification using adaptive convolution and dynamic attention

Renal cell carcinoma (RCC), which is the primary subtype of kidney cancer, is among the leading causes of cancer. Recent breakthroughs in computer vision, particularly deep learning, have revolutionized the analysis of histopathology images, thus providing potential solutions for tasks such as the grading of renal cell carcinoma. Nevertheless, the multitude of available neural network architectures and the absence of systematic evaluations render it challenging to identify optimal models and training configurations for distinct histopathology classification tasks. Hence, we propose a novel hybrid model that effectively combines the advantages of vision transformers and convolutional neural networks. The proposed method, which is named the renal cancer grading network, comprises two essential components: an adaptive convolution (AC) block and a dynamic attention (DA) block. The AC block emphasizes efficient feature extraction and spatial representation learning via intelligently designed convolutional operations. The DA block, which is constructed on the features of the AC block, is a crucial module for histopathology-image classification. It introduces a dynamic attention mechanism and employs a transformer encoder to refine learned representations. Experiments were conducted on four publicly available histopathology datasets: RCC dataset of Kasturba medical college (KMC), colorectal cancer histology (CRCH), break cancer histology (BreakHis) and colon cancer histopathology dataset (CCH). The proposed method demonstrated an accuracy of 90.62%, precision of 91.23%, recall of 90.63%, and a weighted harmonic mean of precision and recall (F1-score) of 90.92 on the KMC dataset. Similarly, the proposed method demonstrates consistent accuracy (weighted average F1-score of 99%) on the CRCH dataset, recognition rate of 88.30% on the BreakHis dataset, and an accuracy of 99.7% on CCH dataset. These results confirm that our method outperforms the state-of-the-art methods, thus demonstrating its effectiveness and robustness across various datasets.

TIST-Net: style transfer in dynamic contrast enhanced MRI using spatial and temporal information

Objective. Training deep learning models for image registration or segmentation of dynamic contrast enhanced (DCE) MRI data is challenging. This is mainly due to the wide variations in contrast enhancement within and between patients. To train a model effectively, a large dataset is needed, but acquiring it is expensive and time consuming. Instead, style transfer can be used to generate new images from existing images. In this study, our objective is to develop a style transfer method that incorporates spatio-temporal information to either add or remove contrast enhancement from an existing image. Approach. We propose a temporal image-to-image style transfer network (TIST-Net), consisting of an auto-encoder combined with convolutional long short-term memory networks. This enables disentanglement of the content and style latent spaces of the time series data, using spatio-temporal information to learn and predict key structures. To generate new images, we use deformable and adaptive convolutions which allow fine grained control over the combination of the content and style latent spaces. We evaluate our method, using popular metrics and a previously proposed contrast weighted structural similarity index measure. We also perform a clinical evaluation, where experts are asked to rank images generated by multiple methods. Main Results. Our model achieves state-of-the-art performance on three datasets (kidney, prostate and uterus) achieving an SSIM of 0.91 ± 0.03, 0.73 ± 0.04, 0.88 ± 0.04 respectively when performing style transfer between a non-enhanced image and a contrast-enhanced image. Similarly, SSIM results for style transfer from a contrast-enhanced image to a non-enhanced image were 0.89 ± 0.03, 0.82 ± 0.03, 0.87 ± 0.03. In the clinical evaluation, our method was ranked consistently higher than other approaches. Significance. TIST-Net can be used to generate new DCE-MRI data from existing images. In future, this may improve models for tasks such as image registration or segmentation by allowing small training datasets to be expanded.

Internet of Things intrusion detection: Research and practice of NSENet and LSTM fusion models

To address the problems of complex environment, limited device computational resources and limited memory resources in the existing IoT, SELSTM, an intrusion detection system composed of NSENet and LSTM fusion based on SENet, is investigated. The NSENet part of the SELSTM system is based on the squeeze-and-excitation network (SENet). The lightweight computational modules NonLocal, SKConv and inverted residuals are fused into SE blocks, and self-attention of Nonlocal is used to improve the local receptive field of feature extraction. The channel attention and spatial attention of each part of the data are strengthened by the use of SKConv To enhance the adaptive convolution ability of the model and ensure the completeness of the information, the properties of the inverted residual structure are used to ensure that the gradient of the model decreases steadily without gradient explosion or disappearance. For the problem of data imbalance, the dataset is randomly resampled using the weight resampling technique to improve the balance of the dataset to ensure that the final detection effect of the model is more effective and generalized, while the data flow is divided into two parts for processing, and the model parameters are optimized using the model gradient optimizer consisting of the optimizer Lion and the optimization function Lookahead. The model extracts the spatial and temporal features of the data through multidimensional extraction to ensure the completeness of the data feature information in multiple dimensions, thus obtaining better detection results. The results of the experiments comparing the SELSTM model with other models on the intrusion dataset show that the intrusion detection model has a higher detection precision and accuracy than the traditional deep learning intrusion detection model, which indicates that the SELSTM has better detection performance properties and better practicality and effectiveness on IoT devices.

FACNN: fuzzy-based adaptive convolution neural network for classifying COVID-19 in noisy CXR images.

COVID-19 detection using chest X-rays (CXR) has evolved as a significant method for early diagnosis of the pandemic disease. Clinical trials and methods utilize X-ray images with computer and intelligent algorithms to improve detection and classification precision. This article thus proposes a fuzzy-based adaptive convolution neural network (FACNN) model to improve the detection precision by confining the false rates. The feature extraction process between the successive regions is validated using a fuzzy process that classifies labeled and unknown pixels. The membership functions are derived based on high precision features for detection and false rate suppression process. The convolution neural network process is responsible for increasing detection precision through recurrent training based on feature availability. This availability analysis is verified using fuzzy derivatives under local variances. Based on variance-reduced features, the appropriate regions with labeled and unknown features are used for normal or infected classification. Thus, the proposed FACNN improves accuracy, precision, and feature extraction by 14.36%, 8.74%, and 12.35%, respectively. This model reduces the false rate and extraction time by 10.35% and 10.66%, respectively.

Key‐point‐guided adaptive convolution and instance normalization for continuous transitive face reenactment of any person

AbstractFace reenactment technology is widely applied in various applications. However, the reconstruction effects of existing methods are often not quite realistic enough. Thus, this paper proposes a progressive face reenactment method. First, to make full use of the key information, we propose adaptive convolution and instance normalization to encode the key information into all learnable parameters in the network, including the weights of the convolution kernels and the means and variances in the normalization layer. Second, we present continuous transitive facial expression generation according to all the weights of the network generated by the key points, resulting in the continuous change of the image generated by the network. Third, in contrast to classical convolution, we apply the combination of depth‐ and point‐wise convolutions, which can greatly reduce the number of weights and improve the efficiency of training. Finally, we extend the proposed face reenactment method to the face editing application. Comprehensive experiments demonstrate the effectiveness of the proposed method, which can generate a clearer and more realistic face from any person and is more generic and applicable than other methods.

Deep learning with adaptive convolutions for classification of retinal diseases via optical coherence tomography

Optical coherence tomography (OCT) uses interferometry to capture high-resolution cross-sectional images of the retina to diagnose retinal diseases. Convolutional neural networks (CNNs) have become essential for developing efficient computer-aided diagnostic algorithms, but noisy images can hinder their performance. This study introduces an innovative image preprocessing strategy that involves a new method of representing images to reduce image noise and a new adaptive convolution layer. The adaptive convolution layer aims to replace traditional convolution layers for OCT image classification by relying on local’Feature Content’. The proposed image representation is based on Zeckendorf's theorem, which states that every positive integer may be split into a unique sum of distinct, non-adjacent Fibonacci numbers. The proposed approach enables the generation of two separate images, known as the ‘base’ and ‘fine,’ where the ‘base’ image is the denoised image. We assessed our methodology by evaluating against ten filters, comprising a Low-pass filter, Gaussian filter, Wiener filter, Wavelet filter, Guided filter, Lee filter, Frost filter, Kuan filter, Detail Preserving Anisotropic Diffusion (DPAD) filter, and Non-Local Means (NLM) filter. In our study, we found that the proposed filter produced the most favorable results in five of the six no-reference parameters (Blur Percent (BP), Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE), Naturalness Image Quality Evaluator (NIQE), Wavelet Variance (WAVV), Wavelet Variance Radial (WAVR)) used to assess the effectiveness of the proposed image enhancement technique. The OCT dataset utilized in the study was compiled by the University of California San Diego. The proposed adaptive convolution layer and its accompanying activation function were tested using seven OCT image classification CNN architectures. The test architectures comprise OctNET, NT-CNN, AOCT-NET, M-CNN, LightOCT, RetiNet, and DeepOCT. Experiments were conducted to assess the impact of the new preprocessing algorithm and the placement of the adaptive convolution layer as a substitute for the conventional convolution layer. Implementing the proposed approaches resulted in accuracy improvements ranging from 0.44% to 2.44% across architectures. Our findings highlight the efficacy of the proposed indirect noise reduction technique and a texture-sensitive adaptive convolution layer.

Spectral-Spatial-Sensorial Attention Network with Controllable Factors for Hyperspectral Image Classification

Hyperspectral image (HSI) classification aims to recognize categories of objects based on spectral–spatial features and has been used in a wide range of real-world application areas. Attention mechanisms are widely used in HSI classification for their ability to focus on important information in images automatically. However, due to the approximate spectral–spatial features in HSI, mainstream attention mechanisms are difficult to accurately distinguish the small difference, which limits the classification accuracy. To overcome this problem, a spectral–spatial-sensorial attention network (S3AN) with controllable factors is proposed to efficiently recognize different objects. Specifically, two controllable factors, dynamic exponential pooling (DE-Pooling) and adaptive convolution (Adapt-Conv), are designed to enlarge the difference in approximate features and enhance the attention weight interaction. Then, attention mechanisms with controllable factors are utilized to build the redundancy reduction module (RRM), feature learning module (FLM), and label prediction module (LPM) to process HSI spectral–spatial features. The RRM utilizes the spectral attention mechanism to select representative band combinations, and the FLM introduces the spatial attention mechanism to highlight important objects. Furthermore, the sensorial attention mechanism extracts location and category information in a pseudo label to guide the LPM for label prediction and avoid details from being ignored. Experimental results on three public HSI datasets show that the proposed method is able to accurately recognize different objects with an overall accuracy (OA) of 98.69%, 98.89%, and 97.56%, respectively.

An adaptive fully convolutional network for bearing fault diagnosis under noisy environments.

Intelligent diagnostic algorithms based on convolutional neural networks (CNNs) have shown great potential in diagnosing various conditions. However, accurately and robustly diagnosing faults in noisy situations remains challenging. This study presents an adaptive fully convolutional network (AFCN) for identifying bearing defects in noisy environments. First, we use a novel large kernel convolution method for high-frequency noise reduction and wide-area temporal feature extraction. By utilizing a sequence of stacked residual adaptive convolution blocks, the AFCN achieves a selective emphasis on significant features and adaptive adjustment of feature weights at various convolution scales. The experimental results have shown that the AFCN achieves a diagnostic accuracy of over 90% for the faults in the CWRU dataset under the -8 dB noise and over 77% for the PU dataset in the case of -6 dB noise. The comparison results with five advanced baseline models have demonstrated the superiority of the AFCN in feature extraction, noise immunity, and robustness to the noise environment. The AFCN provides a better adaption to noise interference than conventional CNNs and other advanced adaptive networks.

MM-TTS: Multi-Modal Prompt Based Style Transfer for Expressive Text-to-Speech Synthesis

The style transfer task in Text-to-Speech (TTS) refers to the process of transferring style information into text content to generate corresponding speech with a specific style. However, most existing style transfer approaches are either based on fixed emotional labels or reference speech clips, which cannot achieve flexible style transfer. Recently, some methods have adopted text descriptions to guide style transfer. In this paper, we propose a more flexible multi-modal and style controllable TTS framework named MM-TTS. It can utilize any modality as the prompt in unified multi-modal prompt space, including reference speech, emotional facial images, and text descriptions, to control the style of the generated speech in a system. The challenges of modeling such a multi-modal style controllable TTS mainly lie in two aspects: 1) aligning the multi-modal information into a unified style space to enable the input of arbitrary modality as the style prompt in a single system, and 2) efficiently transferring the unified style representation into the given text content, thereby empowering the ability to generate prompt style-related voice. To address these problems, we propose an aligned multi-modal prompt encoder that embeds different modalities into a unified style space, supporting style transfer for different modalities. Additionally, we present a new adaptive style transfer method named Style Adaptive Convolutions (SAConv) to achieve a better style representation. Furthermore, we design a Rectified Flow based Refiner to solve the problem of over-smoothing Mel-spectrogram and generate audio of higher fidelity. Since there is no public dataset for multi-modal TTS, we construct a dataset named MEAD-TTS, which is related to the field of expressive talking head. Our experiments on the MEAD-TTS dataset and out-of-domain datasets demonstrate that MM-TTS can achieve satisfactory results based on multi-modal prompts. The audio samples and constructed dataset are available at https://multimodal-tts.github.io.

Incorporating a-priori information in deep learning models for quantitative susceptibility mapping via adaptive convolution.

Quantitative susceptibility mapping (QSM) has attracted considerable interest for tissue characterization (e.g., iron and calcium accumulation, myelination, venous vasculature) in the human brain and relies on extensive data processing of gradient-echo MRI phase images. While deep learning-based field-to-susceptibility inversion has shown great potential, the acquisition parameters applied in clinical settings such as image resolution or image orientation with respect to the magnetic field have not been fully accounted for. Furthermore, the lack of comprehensive training data covering a wide range of acquisition parameters further limits the current QSM deep learning approaches. Here, we propose the integration of a priori information of imaging parameters into convolutional neural networks with our approach, adaptive convolution, that learns the mapping between the additional presented information (acquisition parameters) and the changes in the phase images associated with these varying acquisition parameters. By associating a-priori information with the network parameters itself, the optimal set of convolution weights is selected based on data-specific attributes, leading to generalizability towards changes in acquisition parameters. Moreover, we demonstrate the feasibility of pre-training on synthetic data and transfer learning to clinical brain data to achieve substantial improvements in the computation of susceptibility maps. The adaptive convolution 3D U-Net demonstrated generalizability in acquisition parameters on synthetic and in-vivo data and outperformed models lacking adaptive convolution or transfer learning. Further experiments demonstrate the impact of the side information on the adaptive model and assessed susceptibility map computation on simulated pathologic data sets and measured phase data.

STANet: Spatio-Temporal Adaptive Network and Clinical Prior Embedding Learning for 3D+T CMR Segmentation.

The segmentation of cardiac structure in magnetic resonance images (CMR) is paramount in diagnosing and managing cardiovascular illnesses, given its 3D+Time (3D+T) sequence. The existing deep learning methods are constrained in their ability to 3D+T CMR segmentation, due to: (1) Limited motion perception. The complexity of heart beating renders the motion perception in 3D+T CMR, including the long-range and cross-slice motions. The existing methods' local perception and slice-fixed perception directly limit the performance of 3D+T CMR perception. (2) Lack of labels. Due to the expensive labeling cost of the 3D+T CMR sequence, the labels of 3D+T CMR only contain the end-diastolic and end-systolic frames. The incomplete labeling scheme causes inefficient supervision. Hence, we propose a novel spatio-temporal adaptation network with clinical prior embedding learning (STANet) to ensure efficient spatio-temporal perception and optimization on 3D+T CMR segmentation. (1) A spatio-temporal adaptive convolution (STAC) treats the 3D+T CMR sequence as a whole for perception. The long-distance motion correlation is embedded into the structural perception by learnable weight regularization to balance long-range motion perception. The structural similarity is measured by cross-attention to adaptively correlate the cross-slice motion. (2) A clinical prior embedding learning strategy (CPE) is proposed to optimize the partially labeled 3D+T CMR segmentation dynamically by embedding clinical priors into optimization. STANet achieves outstanding performance with Dice of 0.917 and 0.94 on two public datasets (ACDC and STACOM), which indicates STANet has the potential to be incorporated into computer-aided diagnosis tools for clinical application.

Design of multispectral array imaging system based on depth-guided network

Imaging techniques using multispectral filter arrays (MSFA)have become a research hotspot with the rapid development of spectroscopic techniques. Among them, exploiting the correlation of color channels in the raw data and reconstructing raw images with high sparsity is a bottleneck and constraint in multi-band MSFA imaging systems. Therefore, this paper proposes a 4 × 4 eight-band MSFA imaging system containing a high sampling rate all-pass band. The all-pass band with a 1/2 high sampling rate contains rich color texture information to provide more features. A depth-guided reconstruction network (DGRN), including a depth-guided model (DGM) and a channel adaptive convolution model (CACM), is established to reconstruct the original spectral images. DGM extracts the color texture information of all-pass band images as the guide feature, which is combined with the initially processed eight-band shallow features to be the input of CACM to assign different guide features to different bands adaptively for learning and aggregation. The spatial correlation and spectral correlation of multiple bands are jointly learned using spectral and spatial properties to make the network flexible for MSFA imaging systems. The experimental results show that the method can effectively remove the artifacts of reconstructed images and improve the edge texture clarity.

Hourly load prediction based feature selection scheme and hybrid CNN‐LSTM method for building's smart solar microgrid

AbstractThe short‐term load prediction is the critical operation in the peak demand administration and power generation scheduling of buildings that integrated the smart solar microgrid (SSM). Many research studies have proved that hybrid deep learning strategies achieve more accuracy and feasibility in practical applications than individual algorithms. Moreover, many buildings have integrated the SSM on the rooftop with the battery management system (BMS) to enhance energy efficiency management. However, the traditional methodologies only processed the weather parameters and power demand information for short‐term load prediction, ignoring the collected data from SSM and BMS by the advanced metering infrastructures (AMI), which probably improved prediction accuracy. In this research, many accumulated data of building and SSM are collected before methodology implementation. Considering the diversities of accumulated parameters from SSM and BMS, an adaptive convolution neural network long short‐term memory (CNN‐LSTM) is proposed for hourly electrical load prediction. The CNN could extract the critical large‐scale input feature, while the LSTM could achieve better accurate forecasts. The Pearson correlation matrix is calculated for the feature selection scheme from different data units. The hyperparameter tuning is utilized for obtaining the optimized hybrid CNN‐LSTM algorithm. The K‐fold cross‐validation is employed for deep learning algorithm verification, which includes LSTM, GRU, CNN, and Bi‐LSTM methodologies. The results prove that the hybrid CNN‐LSTM achieved outperformed improvements, which are 20.57%, 29.63%, 19.06% in MSE, MAE, MAPE, and 21.24%, 22.02%, 3.82% in validating MSE, MAE, MAPE, respectively. The hybrid CNN and LSTM combined with the feature selection scheme achieve superior predicting accuracies, proving the adaptability ability for integrating into the energy management system (EMS) of the building's SSM.

Multiple Adverse Weather Removal Using Masked-Based Pre-Training and Dual-Pooling Adaptive Convolution

Multiple Adverse Weather Removal Using Masked-Based Pre-Training and Dual-Pooling Adaptive Convolution

Reliable and Lightweight Adaptive Convolution Network for PCB Surface Defect Detection

Reliable and Lightweight Adaptive Convolution Network for PCB Surface Defect Detection

ACPP-Net: Enhancing Strip Steel Surface Defect Detection With Efficient Adaptive Convolution and Channel-Spatial Pyramid Pooling

ACPP-Net: Enhancing Strip Steel Surface Defect Detection With Efficient Adaptive Convolution and Channel-Spatial Pyramid Pooling

SSCAConv: Self-Guided Spatial-Channel Adaptive Convolution for Image Fusion

SSCAConv: Self-Guided Spatial-Channel Adaptive Convolution for Image Fusion

Attention for Adaptive Convolution in Convolutional Neural Networks for Single Image Super-Resolution

Attention for Adaptive Convolution in Convolutional Neural Networks for Single Image Super-Resolution

Adaptive Convolution Sparse Filtering Method for the Fault Diagnosis of an Engine Timing Gearbox.

Due to the superior robustness of outlier signals and the unique advantage of not relying on a priori knowledge, Convolution Sparse Filtering (CSF) is drawing more and more attention. However, the excellent properties of CSF is limited by its inappropriate selection of the number and length of its filters. Therefore, the Adaptive Convolution Sparse Filtering (ACSF) method is proposed in this paper to implement an end-to-end health monitoring and fault diagnostic model. Firstly, a novel metric entropy-time function (He-T) is proposed to measure the accuracy and efficiency of signals filtered by the CSF. Then, the filtered signal with the minimum He-T is detected with particle swarm optimization. Finally, the failure mode is diagnosed according to the envelope spectrum of the signal with minimum He-T. The effectiveness and efficiency of the ACSF is demonstrated through the experiment. The results indicate the ACSF can extract the failure characteristic of the gearbox.