Residual Block Research Articles

US-Net: U-shaped network with Convolutional Attention Mechanism for ultrasound medical images

Ultrasound imaging, characterized by low contrast, high noise, and interference from surrounding tissues, poses significant challenges in lesion segmentation. To tackle these issues, we introduce an enhanced U-shaped network that incorporates several novel features for precise, automated segmentation. Firstly, our model utilizes a convolution-based self-attention mechanism to establish long-range dependencies in feature maps, crucial for small dataset applications, accompanied by a soft thresholding method for noise reduction. Secondly, we employ multi-sized convolutional kernels to enrich feature processing, coupled with curvature calculations to accentuate edge details via a soft-attention approach. Thirdly, an advanced skip connection strategy is implemented in the UNet architecture, integrating information entropy to assess and utilize texture-rich channels, thereby improving semantic detail in the encoder before merging with decoder outputs. We validated our approach using a newly curated dataset, VPUSI (Vascular Plaques Ultrasound Images), alongside the established datasets, BUSI, TN3K and DDTI. Comparative experiments on these datasets show that our model outperforms existing state-of-the-art techniques in segmentation accuracy.

Dynamic attention aggregated missing spatial–temporal data imputation for traffic speed prediction

Traffic speed forecasting is indispensable in Intelligent Traffic Systems (ITS). The missing traffic speed due to sensors’ failure may hamper the accurate prediction of traffic speed. Different kinds of imputation techniques were employed to replace the missing values considering the spatial and temporal information. However, previous techniques did not consider the dynamic contribution of the neighboring nodes and failed to capture the complex properties of traffic speed data precisely. The architecture of the previous imputation models was infeasible in real-time applications in terms of scalability. Therefore, a novel machine learning-based imputation technique is proposed to generate the missing traffic speed using the spatial–temporal information and dynamic contribution of neighboring nodes. A new dataset considering the characteristics of the traffic speed data is generated from the existing dataset by exploiting the non-missing traffic speed of neighboring nodes. The newly formed dataset helps in the estimation of the dynamic contribution of the neighboring nodes using the linear regression algorithm. The non-missing speeds and estimated dynamic contribution of neighboring nodes assist in calculating the missing speeds of a node. Furthermore, a novel deep learning-based prediction model is introduced to forecast traffic speed accurately. The prediction model contains graph structure learning, a deep graph neural network with skip connections, GRU, multi-head soft attention, and a fully connected neural network. Graph structure learning is proposed based on the distribution of the congestion to represent the graph in an efficient way. The deep graph neural network with skip connections and GRU help in capturing topological and temporal information, respectively. The multi-head soft attention is designed to focus on every relevant temporal information at each time step for capturing global traffic information. Finally, the extracted spatial–temporal features are fed into the fully connected neural network to predict the traffic speed. The proposed imputation and prediction models are evaluated on two real-time datasets, METR-LA and PeMSD7 datasets, under different missing rates of different missing patterns. The proposed framework generates possible spatial–temporal information for efficient traffic congestion management and evicts erroneous issues in real-time.

On the application of hybrid deep 3D convolutional neural network algorithms for predicting the micromechanics of brain white matter

Background:Material characterization of brain white matter (BWM) is difficult due to the anisotropy inherent to the three-dimensional microstructure and the various interactions between heterogeneous brain-tissue (axon, myelin, and glia). Developing full scale finite element models that accurately represent the relationship between the micro and macroscale BWM is however extremely challenging and computationally expensive. The anisotropic properties of the microstructure of BWM computed by building unit cells under frequency domain viscoelasticity comprises of 36 individual constants each, for the loss and storage moduli. Furthermore, the architecture of each unit cell is arbitrary in an infinite dataset. Methods:In this study, we extend our previous work on developing representative volume elements (RVE) of the microstructure of the BWM in the frequency domain to develop 3D deep learning algorithms that can predict the anisotropic composite properties. The deep 3D convolutional neural network (CNN) algorithms utilizes a voxelization method to obtain geometry information from 3D RVEs. The architecture information encoded in the voxelized location is employed as input data while cross-referencing the RVEs’ material properties (output data). We further develop methods by incorporating parallel pathways, Residual Neural Networks and inception modulus that improve the efficiency of deep learning algorithms. Results:This paper presents different CNN algorithms in predicting the anisotropic composite properties of BWM. A quantitative analysis of the individual algorithms is presented with the view of identifying optimal strategies to interpret the combined measurements of brain MRE and DTI. Significance:The proposed Multiscale 3D ResNet (M3DR) algorithm demonstrates high learning ability and performance over baseline CNN algorithms in predicting BWM tissue properties. The hybrid M3DR framework also overcomes the significant limitations encountered in modeling brain tissue using finite elements alone including those such as high computational cost, mesh and simulation failure. The proposed framework also provides an efficient and streamlined platform for implementing complex boundary conditions, modeling intrinsic material properties and imparting interfacial architecture information.

Enhancing severe weather predictions with the I-ConvGRU model: An iterative approach for radar echo time series through ConvGRU and RainNet integration

ABSTRACT Precipitation nowcasting plays a crucial role in disaster prevention and mitigation. Existing forecasting models often underutilize output data, leading to suboptimal forecasting performance. To tackle this issue, we introduce the I-ConvGRU model, a novel radar echo timing prediction model that synergizes the temporal dynamics optimization of ConvGRU with the spatial feature enhancement capabilities of RainNet. The model forecasts future scenarios by processing 10 sequential time-series images as input while employing skip connections to boost its spatial feature representation further. Evaluation of the radar echo data set from the Hong Kong Hydrological and Meteorological Bureau spanning from 2009 to 2015 demonstrates the I-ConvGRU model's superiority, with reductions of 17(3.8%) and 49(3.2%) in MSE and MAE metrics, respectively, compared with the TrajGRU model; meanwhile, the I-ConvGRU model had 52(5.8%) and 144(3.8%) lower values on the B-MSE and B-MAE metrics, respectively, than the slightly better performing TrajGRU model. Notably, it significantly improves the prediction of severe precipitation events, with the CSI and HSS metrics increasing by 0.0251(9.6%) and 0.0277(6.8%). These results affirm the model's enhanced effectiveness in radar echo forecasting, particularly in predicting heavy rainfall events.

Breaking through clouds: A hierarchical fusion network empowered by dual-domain cross-modality interactive attention for cloud-free image reconstruction

Cloud obscuration undermines the availability of optical images for continuous monitoring in earth observation. Fusing features from synthetic aperture radar (SAR) has been recognized as a feasible strategy to guide the reconstruction of corrupted signals in cloud-contaminated regions. However, due to the different imaging mechanisms and reflection characteristics, the substantial domain gap between SAR images and optical images makes it a challenging problem to effectively perform cross-modality feature fusion. Although several SAR-assisted cloud removal methods have been proposed, most of them are often unable to achieve adequate information interaction between different modalities, which greatly limits the effectiveness and reliability of cross-modality fusion. In this paper, we proposed a novel hierarchical framework for cloud-free multispectral image reconstruction, which effectively integrates SAR and optical data by a dual-domain interactive attention mechanism. The overall encoder–decoder network is a W-shaped asymmetric structure with a two-branch encoder and one decoder. The encoder branches extract features from SAR images and optical images separately, while the decoder exploits multiscale residual block groups to expand the receptive field and multi-output strategy for reducing the training difficulty. The core cross-modality feature fusion module at the bottleneck adopts the dual-domain interactive attention (DDIA) mechanism which enhances the reciprocal infusion of SAR and optical features to encourage the reconstruction of spectral and structural information. Furthermore, features in the spatial and frequency domains are integrated to improve the effectiveness of the fusion process. To echo the overall network structure, the loss function is designed as a multiscale loss in dual domains. The proposed method can realize sufficient information communication and effective cross-modality fusion between SAR features and optical features. Extensive experiments on the SMILE-CR dataset and SEN12MS-CR dataset demonstrated that the proposed method can outperform the seven representative deep-learning comparative methods in terms of visual performance and quantitative accuracy.

Hierarchical cavitation intensity recognition using Sub-Master Transition Network-based acoustic signals in pipeline systems

Deep learning (DL) has emerged as the most effective approach for cavitation intensity recognition. However, current DL-based models face limitations due to the inherent assumption that all target classes are treated equally. In particular, when certain classes are more difficult to distinguish than others, or when classes can be structured into a hierarchy of categories, DL-based cavitation recognition models struggle to achieve optimal performance. In this study, we propose a hierarchical cavitation intensity recognition framework using the Sub-Master Transition Network (SMTNet) to improve the performance of cavitation intensity recognition of acoustic signal valves. The SMTNet model achieves cavitation intensity recognition from coarse to fine according to the hierarchical properties of cavitation states, overcoming the limitations of DL methods. We also present a global filter layer to address for the long-term dependence of the acoustic signals. In addition, we introduce a 1D double hierarchical residual block to capture sensitive features of acoustic signals. Moreover, a hierarchical multi-label tree is embedded in the SMTNet to map the semantic structure of cavitation states. The proposed model has been evaluated on three real-world cavitation datasets from SAMSON AG, all showing superior results and outperforming SOTA DL methods.

HEDehazeNet: Unpaired image dehazing via enhanced haze generation

Unpaired image dehazing models based on Cycle-Consistent Adversarial Networks (CycleGAN) typically consist of two cycle branches: dehazing-rehazing branch and hazing-dehazing branch. In these two branches, there is an asymmetry of information in the mutual transformation process between haze images and haze-free images. Previous models tended to focus more on the transformation process from haze images to haze-free images within the dehazing-rehazing branch, overlooking the provision of effective information for the formation of haze images in the hazing-dehazing branch. This oversight results in the production of haze patterns that are both monotonous and simplistic, ultimately impeding the overall performance and generalization capabilities of dehazing networks. In light of this, this paper proposes a novel model called HEDehazeNet (Dehazing Net based on Haze Generation Enhancement), which provides crucial information for the generation process of haze images through a dedicated haze generation enhancement module. This module is capable of producing three distinct modes of transmission maps - random transmission map, simulated transmission map, and mixed transmission maps combining both. Employing these transmission maps to generate hazing images with varying density and patterns provides the dehazing network with a more diverse and dynamically complex set of training samples, thereby enhancing its capacity to handle intricate scenes. Additionally, we made minor modifications to the U-Net, replacing residual blocks with multi-scale parallel convolutional blocks and channel self-attention, to further enhance the network's performance. Experiments were conducted on both synthetic and real-world datasets, substantiating the superiority of HEDehazeNet over the current state-of-the-art unpaired dehazing models.

Ancient Painting Inpainting Based on Multi-Layer Feature Enhancement and Frequency Perception

Image inpainting aims to restore the damaged information in images, enhancing their readability and usability. Ancient paintings, as a vital component of traditional art, convey profound cultural and artistic value, yet often suffer from various forms of damage over time. Existing ancient painting inpainting methods are insufficient in extracting deep semantic information, resulting in the loss of high-frequency detail features of the reconstructed image and inconsistency between global and local semantic information. To address these issues, this paper proposes a Generative Adversarial Network (GAN)-based ancient painting inpainting method using multi-layer feature enhancement and frequency perception, named MFGAN. Firstly, we design a Residual Pyramid Encoder (RPE), which fully extracts the deep semantic features of ancient painting images and strengthens the processing of image details by effectively combining the deep feature extraction module and channel attention. Secondly, we propose a Frequency-Aware Mechanism (FAM) to obtain the high-frequency perceptual features by using the frequency attention module, which captures the high-frequency details and texture features of the ancient paintings by increasing the skip connections between the low-frequency and the high-frequency features, and provides more frequency perception information. Thirdly, a Dual Discriminator (DD) is designed to ensure the consistency of semantic information between global and local region images, while reducing the discontinuity and blurring differences at the boundary during image inpainting. Finally, extensive experiments on the proposed ancient painting and Huaniao datasets show that our proposed method outperforms competitive image inpainting methods and exhibits robust generalization capabilities.

Comparison of super-resolution deep learning models for flow imaging

The primary goal of this study is to introduce deep learning (DL) methods as a cost-effective alternative to the computationally intensive Direct Numerical Simulation (DNS) simulations. We show that one can obtain a parametric field from a low-resolution input and map it to a fine grid output, significantly reducing the computational burden. We assess five super-resolution models for up-scaling low-resolution flow data into fine-grid numerical simulations’ output for accuracy and efficiency. The proposed architectures employ convolutional neural networks interconnected in encoder/decoder branches. We investigate these models using turbulent velocity fields inside a suddenly expanded channel characterized by complex features, including turbulence, instabilities, asymmetries, separation, and reattachment. Our results reveal that an encoder/decoder model with residual connections delivers the fastest results, a U-Net-based model with skip connections excels at producing sharper edges in regions prone to blurring, while deeper models incorporating maximum and average pooling layers show superior performance in reconstructing velocity profiles. These findings significantly contribute to our understanding of the potential of deep learning in fluid mechanics. The models presented in this study are trained and validated on standard computer hardware and can be easily adapted to other problems. The findings are promising for discovering and analyzing flow physics, highlighting the potential for DL techniques to improve the accuracy of the available fluid mechanics computational tools.

A Novel Transformer Network Based on Cross–Spatial Learning and Deformable Attention for Composite Fault Diagnosis of Agricultural Machinery Bearings

Diagnosing agricultural machinery faults is critical to agricultural automation, and identifying vibration signals from faulty bearings is important for agricultural machinery fault diagnosis and predictive maintenance. In recent years, data–driven methods based on deep learning have received much attention. Considering the roughness of the attention receptive fields in Vision Transformer and Swin Transformer, this paper proposes a Shift–Deformable Transformer (S–DT) network model with multi–attention fusion to achieve accurate diagnosis of composite faults. In this method, the vibration signal is first transformed into a time–frequency graph representation through continuous wavelet transform (CWT); secondly, dilated convolutional residual blocks and efficient attention for cross–spatial learning are used for low–level local feature enhancement. Then, the shift window and deformable attention are fused into S–D Attention, which has a more focused receptive field to learn global features accurately. Finally, the diagnosis result is obtained through the classifier. Experiments were conducted on self–collected datasets and public datasets. The results show that the proposed S–DT network performs excellently in all cases. With a slight decrease in the number of parameters, the validation accuracy improves by more than 2%, and the training network has a fast convergence period. This provides an effective solution for monitoring the efficient and stable operation of agricultural automation machinery and equipment.

Smartphone video imaging: A versatile, low-cost technology for food authentication

This study presents a low-cost smartphone-based imaging technique called smartphone video imaging (SVI) to capture short videos of samples that are illuminated by a colour-changing screen. Assisted by artificial intelligence, the study develops new capabilities to make SVI a versatile imaging technique such as the hyperspectral imaging (HSI). SVI enables classification of samples with heterogeneous contents, spatial representation of analyte contents and reconstruction of hyperspectral images from videos. When integrated with a residual neural network, SVI outperforms traditional computer vision methods for ginseng classification. Moreover, the technique effectively maps the spatial distribution of saffron purity in powder mixtures with predictive performance that is comparable to that of HSI. In addition, SVI combined with the U-Net deep learning module can produce high-quality images that closely resemble the target images acquired by HSI. These results suggest that SVI can serve as a consumer-oriented solution for food authentication.

Rs-net: Residual Sharp U-Net architecture for pavement crack segmentation and severity assessment

U-net, a fully convolutional network-based image segmentation method, has demonstrated widespread adaptability in the crack segmentation task. The combination of the semantically dissimilar features of the encoder (shallow layers) and the decoder (deep layers) in the skip connections leads to blurry features map and leads to undesirable over- or under-segmentation of target regions. Additionally, the shallow architecture of the U-Net model prevents the extraction of more discriminatory information from input images. This paper proposes a Residual Sharp U-Net (RS-Net) architecture for crack segmentation and severity assessment in pavement surfaces to address these limitations. The proposed architecture uses residual block in the U-Net model to extract a more insightful representation of features. In addition to that, a sharpening kernel filter is used instead of plain skip connections to generate a fine-tuned encoder features map before combining it with decoder features maps to reduce the dissimilarity between them and smoothes artifacts in the network layers during early training. The proposed architecture is also integrated with various morphological operations to assess the severity of cracks and categorize them into hairline, medium, and severe labels. Experiments results demonstrated that the RS-Net model has promising segmentation performance, outperforming earlier U-Net variations on testing data for crack segmentation and severity assessment, with a promising accuracy (>0.97)

Dynamic obstacle avoidance model of autonomous driving with attention mechanism and temporal residual block

Dynamic obstacle avoidance is crucial in autonomous driving, ensuring vehicle safety by preventing collisions and enhancing driving efficiency. Dynamic obstacle avoidance algorithms have made significant progress due to deep learning. However, video-based target detection methods can suffer from missed or false detections when processing consecutive frames, especially for high-speed moving targets or in complex dynamic scenes. Multi-target tracking methods require intricate algorithm designs for target initialization and occluded object recovery, which can be compromised by tracker performance, leading to unstable tracking or target loss. To address the issue of target loss in multi-target tracking, we designed the novel YTCN model that infuses time series information through temporal convolution and enhances the sensitivity of the receptive field with Spatial Pyramid Pooling, Feature Concatenate and Spatial Convolution(SPPFCSPC), enhancing the model’s feature extraction capability. Simultaneously, we developed the novel Global Attention Mechanism(GAM) and Double Attention(DA) mechanisms that merge channel and spatial features to enhance feature representation. Finally we design the Temporal Residual Block(TRB) to model the temporal obstacles. Experimental results show that our method achieves 82.4 % of mAP_0.5 on the BDD 100 K dataset, which gets 1.3 % higher than the previous methods.

A new strong convective precipitation forecasting method based on attention mechanism and spatio-temporal reasoning

Radar observation variables reflect the precipitation amount of strong convective precipitation processes, which accurate forecast is an important difficulty in weather forecasting. Current forecasting methods are mostly based on radar echo extrapolation, which has the insufficiency of input information and the ineffectiveness of model architecture. This paper presents a Bidirectional Long Short-Term Memory forecasting method for strong convective precipitation based on the attention mechanism and residual neural network (ResNet-Attention-BiLSTM). First, this paper uses ResNet to effectively extract the key information of extreme weather and solves the problem of regression to the mean of the prediction model by learning the residuals of the radar observation data. Second, this paper uses the attention mechanism to adaptively weight the fusion of the features to enhance the extraction of the important features of the precipitation image data. On this basis, this paper presents a novel spatio-temporal reasoning method for radar observations and establishes a precipitation forecasting model, which captures the past and future time-order relationship of the sequence data. Finally, this paper conducts experiments based on the real collected data of a strong convective precipitation process and compares its performance with the existing models, the mean absolute percentage error of this model was reduced by 15.94% (1 km), 18.72% (3 km), and 14.91% (7 km), and the coefficient of determination (R2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$R^{2}$$\\end{document}) was increased by 10.89% (1 km), 9.61% (3 km), and 9.29% (7 km), which proves the state of the art and effectiveness of this forecasting model.

NeuroPred-ResSE: Predicting neuropeptides by integrating residual block and squeeze-excitation attention mechanism

Neuropeptides play crucial roles in regulating neurological function acting as signaling molecules, which provide new opportunity for developing drugs for the treatment of neurological diseases. Therefore, it is very necessary to develop a rapid and accurate prediction model for neuropeptides. Although a few prediction tools have been developed, there is room for improvement in prediction accuracy by using deep learning approach. In this paper, we establish the NeuroPred-ResSE model based on residual block and squeeze-excitation attention mechanism. Firstly, we extract multi-features by using one-hot coding based on the NT5CT5 sequence, dipeptide deviation from expected mean and natural vector. Then, we integrate residual block and squeeze-excitation attention mechanism, which can capture and identify the most relevant attribute features. Finally, the accuracies of the training set and test set are 97.16 % and 96.60 % based on the 5-fold cross-validation and independent test, respectively, and other evaluation metrics have also obtained satisfactory results. The experimental results show that the performance of the NeuroPred-ResSE model outperforms those of existing state-of-the-art models, and our model is an effective, intelligent and robust prediction tool. The datasets and source codes are available at https://github.com/yunyunliang88/NeuroPred-ResSE.

LFDT-Fusion: A latent feature-guided diffusion Transformer model for general image fusion

For image fusion tasks, it is inefficient for the diffusion model to iterate multiple times on the original resolution image for feature mapping. To address this issue, this paper proposes an efficient latent feature-guided diffusion model for general image fusion. The model consists of a pixel space autoencoder and a compact Transformer-based diffusion network. Specifically, the pixel space autoencoder is a novel UNet-based latent diffusion strategy that compresses inputs into a low-resolution latent space through downsampling. Simultaneously, skip connections transfer multiscale intermediate features from the encoder to the decoder for decoding, preserving the high-resolution information of the original input. Compared to the existing VAE-GAN-based latent diffusion strategy, the proposed UNet-based strategy is significantly more stable and generates highly detailed images without adversarial optimization. The Transformer-based diffusion network consists of a denoising network and a fusion head. The former captures long-range diffusion dependencies and learns hierarchical diffusion representations, while the latter facilitates diffusion feature interactions to comprehend complex cross-domain information. Moreover, improvements to the diffusion model in noise level, denoising steps, and sampler selection have yielded superior fusion performance across six image fusion tasks. The proposed method illustrates qualitative and quantitative advantages, as evidenced by experimental results in both public datasets and industrial environments. The code is available at: https://github.com/BOYang-pro/LFDT-Fusion.

Lightweight pyramid attention residual network for intelligent fault diagnosis of machine under sharp speed variation

Deep learning based diagnostic methods have obtained great success in intelligent fault diagnosis of machines. However, most of the existing methods are developed for fault diagnosis task under stable working conditions, and cannot handle the data generated under sharp speed variation favorably. Besides, the large model parameters and high computing costs of these methods fail to meet the requirement of small memory and fast-precise prediction in practical applications. To address these challenges, a lightweight pyramid attention residual network (PARNet) is proposed in this paper. First, an efficient pyramid squeeze convolution module is proposed to capture multi-scale information from raw signals. Then, a soft squeeze-and-excitation attention is designed to establish long-range channel dependency, and fuse features of different scales. Based on the above improvements, a pyramid attention residual block is proposed as basic feature learning unit, and a concise CNN-based model PARNet is built using PAR block. Extensive experiments in two case studies are conducted to validate the effectiveness and adaptability of PARNet under different speed variation conditions. The test results demonstrate the superiority of the proposed method for machine fault diagnosis and reliability assessment in terms of recognition accuracy and model parameter complexity, compared with other advanced diagnostic methods.

Spatial attention guided cGAN for improved salient object detection

Recent research shows that Conditional Generative Adversarial Networks (cGANs) are effective for Salient Object Detection (SOD), a challenging computer vision task that mimics the way human vision focuses on important parts of an image. However, implementing cGANs for this task has presented several complexities, including instability during training with skip connections, weak generators, and difficulty in capturing context information for challenging images. These challenges are particularly evident when dealing with input images containing small salient objects against complex backgrounds, underscoring the need for careful design and tuning of cGANs to ensure accurate segmentation and detection of salient objects. To address these issues, we propose an innovative method for SOD using a cGAN framework. Our method utilizes encoder-decoder framework as the generator component for cGAN, enhancing the feature extraction process and facilitating accurate segmentation of the salient objects. We incorporate Wasserstein-1 distance within the cGAN training process to improve the accuracy of finding the salient objects and stabilize the training process. Additionally, our enhanced model efficiently captures intricate saliency cues by leveraging the spatial attention gate with global average pooling and regularization. The introduction of global average pooling layers in the encoder and decoder paths enhances the network's global perception and fine-grained detail capture, while the channel attention mechanism, facilitated by dense layers, dynamically modulates feature maps to amplify saliency cues. The generated saliency maps are evaluated by the discriminator for authenticity and gives feedback to enhance the generator's ability to generate high-resolution saliency maps. By iteratively training the discriminator and generator networks, the model achieves improved results in finding the salient object. We trained and validated our model using large-scale benchmark datasets commonly used for salient object detection, namely DUTS, ECSSD, and DUT-OMRON. Our approach was evaluated using standard performance metrics on these datasets. Precision, recall, MAE and Fβ score metrics are used to evaluate performance. Our method achieved the lowest MAE values: 0.0292 on the ECSSD dataset, 0.033 on the DUTS-TE dataset, and 0.0439 on the challenging and complex DUT-OMRON dataset, compared to other state-of-the-art methods. Our proposed method demonstrates significant improvements in salient object detection, highlighting its potential benefits for real-life applications.

DERE-Net: A dual-encoder residual enhanced U-Net for muscle fiber segmentation of H&E images

Accurate segmentation of hematoxylin-eosin (H&E) muscle fiber images is crucial for the diagnosis of weightless muscle atrophy. However, uneven contrast, blurred fiber boundaries and adhesions in H&E images are still challenges for accurate segment complete muscle fiber. Although the existing single-encoder based U-shaped networks can extract local information about the target in H&E images, ignores remote dependencies between muscle fibers and low-level detail features, resulting in poor segmentation accuracy when facing H&E images with complex contrast. Therefore, we propose a dual-encoder residual enhanced U-Net segmentation model (DERE-Net) for effective segment muscle fibers in H&E images. DERE-Net uses Transformer and residual CNN to extract global context information and local features of muscle fibers in parallel, and fuses these two features to enable the model to accurately identify the muscle fibers. The multi-scale semantic information fusion (MSIF) module also utilizes the differences between multi-scale features to recover undetected muscle fiber, thus improving the network’s ability to recognize the object regions. In addition, the attention module is added to the skip connection, making the muscle fiber regions information more prominent and improving the robustness of the model. To demonstrate the DERE-Net effect, the comparative experiments are conducted on our own muscle fiber H&E image dataset, GLAS and MoNuSeg. DERE-Net achieved excellent performance on the muscle fiber H&E dataset (DSC 0.919), GLAS dataset (DSC 0.912), and MoNuSeg dataset (DSC 0.821). These results indicate that DERE-Net can accurately predict muscle fiber regions, providing a new method for accurate diagnosis of muscle atrophy in the future.

Hierarchical multi-level dynamic hyperparameter deformable image registration with convolutional neural network

Objective. To enable the registration network to be trained only once, achieving fast regularization hyperparameter selection during the inference phase, and to improve registration accuracy and deformation field regularity. Approach. Hyperparameter tuning is an essential process for deep learning deformable image registration (DLDIR). Most DLDIR methods usually perform a large number of independent experiments to select the appropriate regularization hyperparameters, which are time-consuming and resource-consuming. To address this issue, we propose a novel dynamic hyperparameter block, which comprises a distributed mapping network, dynamic convolution, attention feature extraction layer, and instance normalization layer. The dynamic hyperparameter block encodes the input feature vectors and regularization hyperparameters into learnable feature variables and dynamic convolution parameters which changes the feature statistics of the high-dimensional features layer feature variables, respectively. In addition, the proposed method replaced the single-level structure residual blocks in LapIRN with a hierarchical multi-level architecture for the dynamic hyperparameter block in order to improve registration performance. Main results. On the OASIS dataset, the proposed method reduced the percentage of |Jϕ|⩽0 by 28.01 % , 9.78 % and improved Dice similarity coefficient by 1.17 % , 1.17 % , compared with LapIRN and CIR, respectively. On the DIR-Lab dataset, the proposed method reduced the percentage of |Jϕ|⩽0 by 10.00 % , 5.70 % and reduced target registration error by 10.84 % , 10.05 % , compared with LapIRN and CIR, respectively. Significance. The proposed method can fast achieve the corresponding registration deformation field for arbitrary hyperparameter value during the inference phase. Extensive experiments demonstrate that the proposed method reduces training time compared to DLDIR with fixed regularization hyperparameters while outperforming the state-of-the-art registration methods concerning registration accuracy and deformation smoothness on brain dataset OASIS and lung dataset DIR-Lab.