Network Depth Research Articles

Application of ResUNet-CBAM in Thin-Section Image Segmentation of Rocks

The convolutional neural network (CNN) algorithm in deep learning has been widely applied in petroleum geology research both domestically and internationally. Automated and accurate segmentation of thin-section images of rocks is foundational for in-depth analysis. However, traditional segmentation methods for reservoir rock thin sections often suffer from low accuracy and high cost. To address these issues, this paper proposes a novel segmentation algorithm based on an improved UNet network, integrating residual networks and the CBAM attention mechanism. By incorporating residual modules, the network depth is expanded, and the CBAM attention mechanism enhances the feature weighting capability during learning. Experimental results demonstrate that this method outperforms traditional approaches in both segmentation accuracy and efficiency, representing significant advancements in reservoir rock thin-section image segmentation.

Wetland classification based on depth-adaptive convolutional neural networks using leaf-off SAR imagery

The recent development of deep learning (DL) techniques has created opportunities for classifying wetlands from remote sensing data (mainly optical data). However, the methods for accurately and efficiently classifying large-scale wetlands using DL and radar data that can be more effective than optical data still needs evaluation. In this study, we developed an end-to-end depth-adaptive convolutional neural network (CNN) for mapping wetlands using leaf-off time-series Sentinel-1 Synthetic Aperture Radar (SAR) imagery along with ancillary data. We examined the inclusion of multi-land cover proximity information and a CNN-based self-supervised SAR denoising procedure for enhancing wetland classification accuracy. The depth-adaptive CNN based on U-Net architecture was designed to classify wetland classes (emergent wetland, scrub-shrub wetland, forested wetland, and open water) in Delaware, U.S. while achieving optimization between model complexity (network depths) and accuracy. Results show that our proposed DL method (OA = 0.93, MIoU = 0.60) not only produced a higher classification accuracy than the traditional RF method (OA = 0.89, MIoU = 0.18) but also had a significantly reduced computational cost compared to established state-of-the-art CNNs (e.g., DeepLabV3+ and DANet) without loss of accuracy. The inclusion of multi-land cover proximity information (especially distances to forest and water) and the CNN-based self-supervised SAR denoising procedure can both enhance wetland classification accuracy, especially for forested wetland using traditional RF methods. These results demonstrated the novelty and efficiency of our proposed DL method for classifying wetlands by combing denoised SAR imagery and ancillary information, which provides insights on integration of DL approach and radar data for supporting operational wetland mapping at large spatial scales.

Latent Graph Attention for Spatial Context in Light-Weight Networks: Multi-Domain Applications in Visual Perception Tasks

Global contexts in images are quite valuable in image-to-image translation problems. Conventional attention-based and graph-based models capture the global context to a large extent; however, these are computationally expensive. Moreover, existing approaches are limited to only learning the pairwise semantic relation between any two points in the image. In this paper, we present Latent Graph Attention (LGA), a computationally inexpensive (linear to the number of nodes) and stable modular framework for incorporating the global context in existing architectures. This framework particularly empowers small-scale architectures to achieve performance closer to that of large architectures, making the light-weight architectures more useful for edge devices with lower compute power and lower energy needs. LGA propagates information spatially using a network of locally connected graphs, thereby facilitating the construction of a semantically coherent relation between any two spatially distant points that also takes into account the influence of the intermediate pixels. Moreover, the depth of the graph network can be used to adapt the extent of contextual spread to the target dataset, thereby able to explicitly control the added computational cost. To enhance the learning mechanism of LGA, we also introduce a novel contrastive loss term that helps our LGA module to couple well with the original architecture at the expense of minimal additional computational load. We show that incorporating LGA improves performance in three challenging applications, namely transparent object segmentation, image restoration for dehazing and optical flow estimation.

Vibration displacement measurement of bridge structural models using image super-resolution reconstruction and visual object detection network

Abstract Visual vibration measurement has emerged in the field of structural health monitoring in recent years, but it still has some shortcomings in terms of resolution, recognition rate and real-time performance. Considering the three aspects of recovering high-frequency image details, improving the compactness of the target bounding box, and reducing the computational time, we use the constructed image super-resolution reconstruction model and target detection model to measure the vibration displacement of the bridge structural model. First, we integrate the Transformer module into the Unet network with a simple structure. The Swin and Global Transformer Unet (SGTU) module constructed in this form can reduce the computational cost while reconstructing the large-resolution feature map target, and it can sharply edge information of the vibration target. We use the framework of the YOLOv5 algorithm as the backbone, and use the GhostBottleneck (GB) module to reduce the time for convolution operations to generate similar features. In addition, the proposed DWCBottleneck (DWCB) fusion module is also able to achieve high-level semantic fusion and network depth expansion with minimal computational cost. Finally, the center point offset of the bounding box predicted by the model can be used to obtain the displacement offset of the object in the image sequence. The position information of the target in the first frame image is used as the reference frame for calculating the offset, and the vibration displacement of the flexible structure in the image coordinate system is obtained by calculating the deviation of the displacement between the remaining frames and the first frame. We perform qualitative and quantitative comparisons in three aspects: video super-resolution reconstruction, visual detection robustness, and sensor vibration measurement displacement using a homemade vibration image dataset. The time–frequency domain displacement curves regressed by the visual vibration measurement algorithm are compared with the curves acquired after accelerometer acquisition, indicating the necessity of super-resolution reconstruction in visual vibration measurement.

Feature-fused residual network for time series classification

In various fields such as healthcare and transportation, accurately classifying time series data can provide important support for decision-making. To further improve the accuracy of time series classification, we propose a Feature-fused Residual Network based on Multi-scale Signed Recurrence Plot (MSRP-FFRN). This method transforms one-dimensional time series into two-dimensional images, representing the temporal correlation of time series in a two-dimensional space and revealing hidden details within the data. To enhance these details further, we extract multi-scale features by setting receptive fields of different sizes and using adaptive network depths, which improves image quality. To evaluate the performance of this method, we conducted experiments on 43 UCR datasets and compared it with nine state-of-the-art baseline methods. The experimental results show that MSRP-FFRN ranks first on critical difference diagram, achieving the highest accuracy on 18 datasets with an average accuracy of 89.9%, making it the best-performing method overall. Additionally, the effectiveness of this method is further validated through metrics such as Precision, Recall, and F1 score. Results from ablation experiments also highlight the efficacy of the improvements made by MSRP-FFRN.

Structure-Aware DropEdge Toward Deep Graph Convolutional Networks.

It has been discovered that graph convolutional networks (GCNs) encounter a remarkable drop in performance when multiple layers are piled up. The main factor that accounts for why deep GCNs fail lies in oversmoothing, which isolates the network output from the input with the increase of network depth, weakening expressivity and trainability. In this article, we start by investigating refined measures upon DropEdge-an existing simple yet effective technique to relieve oversmoothing. We term our method as DropEdge++ for its two structure-aware samplers in contrast to DropEdge: layer-dependent (LD) sampler and feature-dependent (FD) sampler. Regarding the LD sampler, we interestingly find that increasingly sampling edges from the bottom layer yields superior performance than the decreasing counterpart as well as DropEdge. We theoretically reveal this phenomenon with mean-edge-number (MEN), a metric closely related to oversmoothing. For the FD sampler, we associate the edge sampling probability with the feature similarity of node pairs and prove that it further correlates the convergence subspace of the output layer with the input features. Extensive experiments on several node classification benchmarks, including both full- and semi-supervised tasks, illustrate the efficacy of DropEdge++ and its compatibility with a variety of backbones by achieving generally better performance over DropEdge and the no-drop version.

An Image Steganalysis Algorithm Based on Multi-Resolution Feature Fusion

The misuse of image steganography poses significant risks to societal security. Whether images include concealed data is a critical problem of information security. Traditional convolutional layers often fail to adequately capture the global correlation of steganographic features as network depth increases, leading to redundant model parameters and missing key features, thereby weakening steganographic signal detection. To solve the problems of highly covert steganographic algorithms and the weakness of traditional methods, a steganography detection solution using multi-resolution feature fusion is presented. This approach uses a multi-resolution network to increase the interactivity from higher to lower resolution. The results of the experiments confirm that the proposed algorithm allows for a maximum accuracy of 90.56% for the embedding rate of 0.4bpp. The overall results prove that the proposed model achieves higher accuracy and better performance than some leading steganalysis models available when applied to different steganographic algorithms and embedding rate conditions.

Research on tool wear and breakage state recognition of heavy milling 508III steel based on ResNet-CBAM

Milling water chamber head material 508 III steel belongs to extreme manufacturing, and the tool failure is very serious in the process of machining. How to monitor and identify the tool wear damage state in time and effectively is a key problem to be solved urgently. Identifying chip morphology changes under machining conditions plays a very important role in characterizing tool wear and breakage. Tool wear has an important influence on the chip morphology during the process of milling 508III steel, and changes in chip morphology can also reflect the tool wear and breakage state. Therefore, this paper takes the chip morphology image as an important feature for tool wear and breakage recognition, and uses Gaussian fuzzy estimation morphology method to preprocess the chip morphology image. Build a ResNet network combined by convolutional block attention module (ResNet-CBAM) tool wear and breakage recognition model, and explore the selection of the model backbone network, determination of the ResNet backbone network depth, and fusion methods of different attention mechanisms. Through the ablation experiment and visual analysis of the attention mechanism module, it is verified that the ResNet-CBAM model proposed in this paper has an accuracy rate of 96.67% in tool wear and breakage state recognition, especially in the stage of severe tool wear and breakage. This study realizes the prediction and early warning of tool life, and provides effective guarantee for the efficient cutting of large parts of high-end equipment and the stable operation of manufacturing system.

Bidirectional Mamba with Dual-Branch Feature Extraction for Hyperspectral Image Classification.

The hyperspectral image (HSI) classification task is widely used in remote sensing image analysis. The HSI classification methods based on convolutional neural networks (CNNs) have greatly improved the classification performance. However, they cannot well utilize the sequential properties of spectral features and face the challenge of increasing computational cost with the increase in network depth. To address these shortcomings, this paper proposes a novel network with a CNN-Mamba architecture, called DBMamba, which uses a bidirectional Mamba to process spectral feature sequences at a linear computational cost. In the DBMamba, principal component analysis (PCA) is first used to extract the main features of the data. Then, a dual-branch CNN structure, with the fused features from spectral-spatial features by 3D-CNN and spatial features by 2D-CNN, is used to extract shallow spectral-spatial features. Finally, a bidirectional Mamba is used to effectively capture global contextual information in features and significantly enhance the extraction of spectral features. Experimental results on the Indian Pines, Salinas, and Pavia University datasets demonstrate that the classification performance surpasses that of many cutting-edge methods, improving by 1.04%, 0.15%, and 0.09%, respectively, over the competing SSFTT method. The research in this paper enhances the existing knowledge on HSI classification and provides valuable insights for future research in this field.

Enhancing Multimodal Emotional Information Extraction in Film and Television through Adaptive Feature Fusion with DenseNe, Transformer, and 3D CNN Models

ABSTRACT The extraction of multimodal emotional information enables a more nuanced representation of the emotional subtleties embedded in film and television works. However, conventional approaches that independently extract features from images and text fail to capture the rich semantic interplay between these modalities, impeding the feature learning process within each modality. To address this, this paper presents an innovative model for extracting emotional information from multimodal film and television content. The model utilizes DenseNe for image feature extraction, enhancing network depth via the MSC module. Text feature extraction is achieved through a Transformer encoder, while video feature extraction employs a 3D CNN model. Refinements are made to the number and placement of convolutional layers, planar convolution size, and 3D convolution depth. Moreover, a multi-head scaled dot-product attention mechanism is incorporated into the interaction module to compute the similarity between each image block within the sequence and every word in the text sequence. Experimental evaluations on the CMU-MOSEI and CMU-MOSI datasets showcase superior performance compared to the baseline model, achieving accuracy and F1 scores of 0.708 and 0.698, respectively. Noteworthy is the proposed adaptive feature fusion module, which enriches the expressiveness of pivotal emotional features while eliminating redundant data.

Ferrimagnet-Based Neuromorphic Device Mimicking the Ventral Visual Pathway for High-Accuracy Target Recognition.

The ventral visual pathway (VVP) of the human brain efficiently implements target recognition by employing a deep hierarchical structure to build complex visual concepts from simple features. Artificial neural networks (ANNs) based on spintronic devices are capable of target recognition, but their poor interpretability and limited network depth hinder ANNs from mimicking the VVP. Hardware implementation of the VVP requires a biorealistic spintronic device as well as the corresponding interpretable and deep network structure, which have not been reported so far. Here, we report a ferrimagnetic neuron with a continuously differentiable exponential linear unit (CeLu) activation function, which is closer to biological neurons and could mitigate the issue of limited network depth. Meanwhile, we also demonstrate that a ferrimagnet can construct artificial synapses with high linearity and symmetry to meet the requirements of weight update algorithms. Based on these neurons and synapses, we propose an all-spin convolutional neural network (CNN) with a high interpretability and deep neural network, to mimic the VVP. Compared to the state-of-the-art spintronic-based neuromorphic computing model, the CNN with bionic function, using experimentally derived device parameters, achieves high recognition accuracies of over 91% and 98% on the CIFAR-10 datasets and the MNIST datasets, respectively, showing improvements of 1.13% and 1.76%. Our work provides a promising method to improve the bionic performance of spintronic device-based neural networks.

Plastic Constitutive Training Method for Steel Based on a Recurrent Neural Network

The deep learning steel plastic constitutive model training method was studied based on the recurrent neural network (RNN) model to improve the allocative efficiency of the deep learning steel plastic constitutive model and promote its application in practical engineering. Two linear hardening constitutive datasets of steel were constructed using the Gaussian stochastic process. The RNN, long short-term memory (LSTM), and gated recurrent unit (GRU) were used as models for training. The effects of the data pre-processing method, neural network structure, and training method on the model training were analyzed. The prediction ability of the model for different scale series and the corresponding data demand were evaluated. The results show that LSTM and the GRU are more suitable for stress–strain prediction. The marginal effect of the stacked neural network depth and number gradually decreases, and the hysteresis curve can be accurately predicted by a two-layer RNN. The optimal structure of the two models is A50-100 and B150-150. The prediction accuracy of the models increased with the decrease in batch size and the increase in training batch, and the training time also increased significantly. The decay learning rate method could balance the prediction accuracy and training time, and the optimal initial learning rate, batch size, and training batch were 0.001, 60, and 100, respectively. The deep learning plastic constitutive model based on the optimal parameters can accurately predict the hysteresis curve of steel, and the prediction abilities of the GRU are 6.13, 6.7, and 3.3 times those of LSTM in short, medium, and long sequences, respectively.

SnapE-ResNet: A novel electronic nose classification algorithm for gas data collected by open sampling systems

Increasing the depth of the neural network and implementing the ensemble of neural networks are two main methods to improve the accuracy of gas recognition algorithms. However, the issues of optimization difficulties or excessive resource consumption may occur when training deep neural networks or integrating multiple neural networks. In this study, an algorithm of snapshot ensemble is combined with Residual network (ResNet) to form a SnapE-ResNet which could simplify the training of the network due to the residual structure of ResNet, while realizes the ensemble of multiple ResNet models without additional time consumption and helps the network escape from local minimum using cyclic cosine annealing algorithm. The effectiveness of the proposed SnapE-ResNet is verified through classification experiments on the public dataset. Furthermore, the long-term drift and short-term drift are reduced by applying a zero-offset baseline compensation algorithm and a gaussian noise that added to the baseline signal. The experimental result indicating a high gas classification accuracy of 99.8 % from SnapE-ResNet is obtained, outperforming the comparative models. In addition, the necessity for snapshot ensemble is validated by network ablation experiments. This study could offer a reference for gas classification tasks in the gas detection fields.

A generic shared attention mechanism for various backbone neural networks

The self-attention mechanism is crucial for enhancing various backbone neural networks’ performance. However, current methods add self-attention modules (SAMs) to each network layer without fully utilizing their potential, resulting in suboptimal performance and higher parameter consumption as network depth increases. In this paper, we reveal an inherent phenomenon: SAMs produce highly correlated attention maps across layers, with an average Pearson correlation coefficient of 0.85. Inspired by this inherent observation, we propose Dense-and-Implicit Attention (DIA), which shares SAMs across layers and uses a long short-term memory module to calibrate and connect these correlated attention maps, improving parameter efficiency. This approach aligns with the neural network’s dynamic system perspective. Extensive experiments show DIA consistently enhances various backbones like ResNet, Transformer, and UNet in tasks such as image classification, object detection, and image generation with diffusion models. Our analysis indicates that DIA’s effectiveness stems from its dense inter-layer information connections, absent in conventional mechanisms, stabilizing training and providing regularization effects. This paper’s insights advance our understanding of attention mechanisms, optimizing them and paving the way for future developments across diverse neural networks.

Hyperparameter Tuning Technique to Improve the Accuracy of Bridge Damage Identification Model

In recent years, active research has been conducted using deep learning to evaluate damage to aging bridges. However, this method is inappropriate for practical use because its performance deteriorates owing to numerous classifications, and it does not use photos of actual sites. To this end, this study used image data from an actual bridge management system as training data and employed a combined learning model for each member among various instance segmentation models, including YOLO, Mask R-CNN, and BlendMask. Meanwhile, techniques such as hyperparameter tuning are widely used to improve the accuracy of deep learning, and this study aimed to improve the accuracy of the existing model through this. The hyperparameters optimized in this study are DEPTH, learning rate (LR), and iterations (ITER) of the neural network. This technique can improve the accuracy by tuning only the hyperparameters while using the existing model for bridge damage identification as it is. As a result of the experiment, when DEPTH, LR, and ITER were set to the optimal values, mAP was improved by approximately 2.9% compared to the existing model.

Application of Visual Transformer in Low-resolution Thermal Infrared Image Recognition

Abstract Addressing the challenges of inadequate accuracy and limited robustness exhibited by current lightweight object detection networks specifically tailored for low-resolution thermal infrared face detection scenarios, this paper delves into developing an ultra-lightweight thermal infrared face detection algorithm that leverages visual attention mechanisms. To ascertain the optimal neural network complexity, a series of comparative experiments are meticulously conducted. With Yolo-FastestDet serving as the benchmark, this study endeavors to compress the backbone network, striking a delicate balance between network depth and detection speed. Additionally, to bolster the network’s capacity for profound feature extraction and precise discrimination of target edges and small objects, a Transformer-Encoder-based visual attention module is seamlessly integrated. Consequently, a lightweight face detection algorithm, enriched with attention mechanisms, is formulated. Furthermore, to mitigate the scarcity of low-resolution infrared face image data, a self-constructed visible-infrared face dataset is employed for training and evaluation purposes. The experimental outcomes reveal that the proposed algorithm attains an impressive mAP@0.5 score of 0.953 on the test dataset while satisfying the stringent real-time detection criterion of 30 frames per second (FPS) when deployed on an embedded Raspberry Pi CPU.

Cognitive diagnostic assessment: A Q-matrix constraint-based neural network method.

Cognitive diagnosis is a crucial element of intelligent education that aims to assess the proficiency of specific skills or traits in students at a refined level and provide insights into their strengths and weaknesses for personalized learning. Researchers have developed numerous cognitive diagnostic models. However, previous studies indicate that diagnostic accuracy can be significantly influenced by the appropriateness of the model and the sample size. Thus, designing a general model that can adapt to different assumptions and sample sizes remains a considerable challenge. Artificial neural networks have been proposed as a promising approach in some studies. In this paper, we propose a cognitive diagnosis model of a neural network constrained by a Q-matrix and named QNN. Specifically, we employ the Q-matrix to determine the connections between neurons and the width and depth of the neural network. Moreover, to reduce the human effort in the training algorithm, we designed a self-organizing map-based cognitive diagnosis training framework called SOM-NN, which enables the QNN to be trained unsupervised. Extensive experimental results on simulated and real datasets demonstrate that our approaches are effective in both accuracy and interpretability. Notably, under unsupervised conditions, our approach has significant advantages on small sample datasets with high levels of guessing and slipping, especially on the pattern-wise agreement rates. This work bridges the gap between psychometrics and machine learning and provides a realistic and implementable reference solution for classroom instructional assessment and the cold start of personalized and adaptive assessment systems.

Growing evolutional deep echo state network

Deep echo state network (Deep ESN) is a powerful method for time-series prediction, but developing the optimal structure of Deep ESN remains challenging. A novel generative framework, growing evolutional Deep ESN (GE Deep ESN) is proposed, where networks are generated through alternating growth and evolution. A model compression method is implemented during the evolution phase to optimize the inner structure of the reservoirs, and to automatically determine reservoir size. A simple termination criterion of growth is introduced to determine network depth. The feasibility and superiority of GE Deep ESN were validated experimentally with two benchmark prediction tasks. For real-time trajectory prediction in photoelectric tracking systems, an iterative learning model based on GE Deep ESN is proposed, the real time prediction result showed that network evolution enhances the model’s suitability for this task, and improves prediction precision.

An integrated approach for mechanical fault diagnosis using maximum mean square discrepancy representation and CNN-based mixed information fusion

Bearings are an essential component of modern industry and cross-domain diagnosis of bearings holds significant importance. However, in practical applications, issues such as insufficient training data and differences between equipment present challenges. Transfer learning has become one of the effective methods to address these problems. This article proposes a fault diagnosis method that combines maximum mean square discrepancy (MMSD) measurement with the convolutional neural network (CNN) with mixed information (MIXCNN). MIXCNN enhances spatial position discrimination through deep convolution and achieves cross-channel information interaction using traditional convolution. The introduction of residual connections reduces information loss, while increasing network depth focuses on highly distinguishable features. MMSD constructs a metric that comprehensively reflects the mean and variance information of data samples in the reproducing kernel Hilbert space, thereby enhancing domain confusion. Experimental results show that this method achieves high diagnostic accuracy in various transfer tasks, with a maximum accuracy of 99.29%, providing reliable support for bearing fault diagnosis.

Exploring the impact of network depth on 3D U-Net-based dose prediction for cervical cancer radiotherapy.

The 3D U-Net deep neural network structure is widely employed for dose prediction in radiotherapy. However, the attention to the network depth and its impact on the accuracy and robustness of dose prediction remains inadequate. 92 cervical cancer patients who underwent Volumetric Modulated Arc Therapy (VMAT) are geometrically augmented to investigate the effects of network depth on dose prediction by training and testing three different 3D U-Net structures with depths of 3, 4, and 5. For planning target volume (PTV), the differences between predicted and true values of D98, D99, and Homogeneity were statistically 1.00 ± 0.23, 0.32 ± 0.72, and -0.02 ± 0.02 for the model with a depth of 5, respectively. Compared to the other two models, these parameters were also better. For most of the organs at risk, the mean and maximum differences between the predicted values and the true values for the model with a depth of 5 were better than for the other two models. The results reveal that the network model with a depth of 5 exhibits superior performance, albeit at the expense of the longest training time and maximum computational memory in the three models. A small server with two NVIDIA GeForce RTX 3090 GPUs with 24 G of memory was employed for this training. For the 3D U-Net model with a depth of more than 5 cannot be supported due to insufficient training memory, the 3D U-Net neural network with a depth of 5 is the commonly used and optimal choice for small servers.