Shallow Features Research Articles

DRSI-Net: Dual-residual spatial interaction network for multi-person pose estimation

Multi-person pose estimation (MPPE), which aims to locate the key points for all persons in the frames, is an active research branch of computer vision. Variable human poses and complex scenes make MPPE dependent on local details and global structures; their absence may cause key point feature misalignment. In this case, high-order spatial interactions that can effectively link the local and global information of features are particularly important. However, most methods do not include spatial interactions. A few methods have low-order spatial interactions, but achieving a good balance between accuracy and complexity is challenging. To address the above problems, a dual-residual spatial interaction network (DRSI-Net) for MPPE with high accuracy and low complexity is proposed herein. Compared to other methods, DRSI-Net recursively performs residual spatial information interactions on the neighbouring features so that more useful spatial information can be retained and more similarities can be obtained between shallow and deep extracted features. The channel and spatial dual attention mechanism introduced in the multi-scale feature fusion also helps the network to adaptively focus on features relevant to the target key points and further refine the generated poses. Simultaneously, by optimising the interactive channel dimensions and dividing the gradient flow, the spatial interaction module is designed to be lightweight, thus reducing the complexity of the network. According to the experimental results on the COCO dataset, the proposed DRSI-Net outperforms other state-of-the-art methods in accuracy and complexity.

Diagnosis of brain disease based on the deep learning algorithms of neural imaging techniques

Neuroimaging technology is considered a non-invasive method research the structure and function of the brain which have been widely used in neuroscience, psychiatry, psychology, and other fields. The development of Deep Learning Neural Network (DLNN), based on the deep learning algorithms of neural imaging techniques in brain disease diagnosis plays a more and more important role. In this paper, a deep neural network imaging technology based on Stack Auto-Encoder (SAE) feature extraction is constructed, and then Support Vector Machine (SVM) was used to solve binary classification problems (Alzheimer’s disease [AD] and Mild Cognitive Impairment [MCI]). Four sets of experimental data were employed to perform the training and testing stages of DLNN. The number of neurons in each of the DLNNs was determined using the grid search technique. Overall, the results of DLNNs performance indicated that the SAE feature extraction was superior over (Accuracy Rate [AR] = 74.9% with structure of 93-171-49-22-93) shallow layer features extraction (AR = 70.8% with structure of 93-22-93) and primary features extraction (AR = 69.2%).

Automated instance segmentation of asphalt pavement patches based on deep learning

The location and pixel-level information of the patch are all critical data for the quantitative evaluation of pavement conditions. However, obtaining both parch location and pixel-level information simultaneously is a challenge in intelligent pavement patch surveys. This paper proposes a deep-learning-based patch instance segmentation network (PISNet) that employs you only look once (YOLO)v5 as the baseline and adds a semantic segmentation branch to provide an effective solution for this challenge. The proposed PISNet replaces the original backbone CSPDarknet53 and neck of YOLOv5 with a novel feature extractor named symmetrical pyramid network (SPN). The proposed SPN aims at repeating fusion and transfer of shallow semantic features and deep spatial localization features in the order of “FPN-PAN-FPN” such that the multi-scale semantic expression and localization ability of the feature map could be enhanced. Moreover, a modified feature selection module is also incorporated into the SPN as a skip connection to aggregate more spatial details of the feature map while suppressing redundant features. Experimental results show that compared with Mask region convolutional neural network (R-CNN), You only look at coefficients (YOLACT), YOLACT++, EfficientDet, Fully convolutional one stage object detector (FCOS), You only look once version 5m (YOLOv5m), U-Net, DeepLabv3+, and High resolution network-object contextual representations (HRNet-OCR), the proposed PISNet has the best detection performance. Meanwhile, the proposed PISNet achieves superior accuracy/frames per second trade-offs compared to Mask R-CNN, YOLACT, and YOLACT++. Particularly, the proposed PISNet has certain promising potential in supporting pavement patch detection in real-time scenarios and potentially degraded pavement patch detection. Moreover, the proposed PISNet can yield superior segmentation results compared with Mask R-CNN, YOLACT, YOLACT++, U-Net, HRNet-OCR, and DeepLabv3+ on public CRACK500 datasets. Code has been made available at: https://github.com/716HAZ/PISNet .

A novel facial age estimation through super-resolution based on deep learning

Age estimation encounters challenges due to the low resolution (LR) of images captured in real-world scenarios, emphasizing the critical role of image quality in achieving accurate age estimations. Despite efforts to enhance age estimation through LR image training, existing methodologies often overlook the incorporation of super-resolution (SR) as a crucial pre-processing step for LR images. This study introduces an innovative approach that integrates SR of facial images with age estimation methodologies. Specifically, a hierarchical reconstruction network (HRN) structure is employed to train LR images, fully integrating shallow concrete features and deep abstract features. Validation of the training outcomes includes evaluation metrics such as peak signal-to-noise ratio and structural similarity index measurement, showing superior performance compared to several state-of-the-art approaches. Subsequently, an enhanced soft stage regression network is designed for facial age measurement and trained using the SR-IMDB dataset generated by HRN to minimize mean absolute error. This approach achieves a final result of 7.33, indicating an average performance improvement of 10.94% compared to five other state-of-the-art methods. Experimental results highlight the effectiveness of integrating SR into the age estimation process.

Semantic segmentation for multisource remote sensing images incorporating feature slice reconstruction and attention upsampling

ABSTRACT Multisource remote sensing images have rich features and high interpretability and are widely employed in many applications. However, highly unbalanced category distributions and complex backgrounds have created some difficulties in the application of remote sensing image semantic segmentation tasks, such as low accuracy of small target segmentation and inaccurate edge extraction. To solve these problems, in this paper, a feature map segmentation reconstruction module and an attention upsampling module are proposed. In the encoder part, the input feature map is equally segmented, and the segmented feature map is enlarged to effectively improve the small target feature information expression ability in the model. In the decoder part, the key segmentation and location information of shallow features are obtained using the global view. The deep semantic information and shallow spatial location information are fully combined to achieve a more refined upsampling operation. In addition, the attention mechanism of the spatial and channel squeeze and excitation block (scSE) is applied to pay more attention to important features and to suppress irrelevant background and redundant information. To verify the effectiveness of the proposed method, the WHU-OPT-SAR dataset and six state-of-the-art algorithms are utilized in comparative experiments. The experimental results show that our model has demonstrated the best performance and low computational complexity. With only approximately half the floating-point operation count and the number of model parameters of the MCANet model, which is specially designed for the dataset, our model surpasses MCANet by 1.52% and 1.53% in terms of mean intersection over union (mIoU) and F1 score, respectively. In particular, for small object regions such as roads and other categories, compared to the baseline model, the IoU and F1 score of our model are improved by 5.27% and 3.99% and by 5.68% and 5.65%, respectively. These results demonstrate the superior performance of our model in terms of accuracy and efficiency.

Tomato leaf disease detection based on attention mechanism and multi-scale feature fusion.

When detecting tomato leaf diseases in natural environments, factors such as changes in lighting, occlusion, and the small size of leaf lesions pose challenges to detection accuracy. Therefore, this study proposes a tomato leaf disease detection method based on attention mechanisms and multi-scale feature fusion. Firstly, the Convolutional Block Attention Module (CBAM) is introduced into the backbone feature extraction network to enhance the ability to extract lesion features and suppress the effects of environmental interference. Secondly, shallow feature maps are introduced into the re-parameterized generalized feature pyramid network (RepGFPN), constructing a new multi-scale re-parameterized generalized feature fusion module (BiRepGFPN) to enhance feature fusion expression and improve the localization ability for small lesion features. Finally, the BiRepGFPN replaces the Path Aggregation Feature Pyramid Network (PAFPN) in the YOLOv6 model to achieve effective fusion of deep semantic and shallow spatial information. Experimental results indicate that, when evaluated on the publicly available PlantDoc dataset, the model's mean average precision (mAP) showed improvements of 7.7%, 11.8%, 3.4%, 5.7%, 4.3%, and 2.6% compared to YOLOX, YOLOv5, YOLOv6, YOLOv6-s, YOLOv7, and YOLOv8, respectively. When evaluated on the tomato leaf disease dataset, the model demonstrated a precision of 92.9%, a recall rate of 95.2%, an F1 score of 94.0%, and a mean average precision (mAP) of 93.8%, showing improvements of 2.3%, 4.0%, 3.1%, and 2.7% respectively compared to the baseline model. These results indicate that the proposed detection method possesses significant detection performance and generalization capabilities.

CAT-Unet: An enhanced U-Net architecture with coordinate attention and skip-neighborhood attention transformer for medical image segmentation

With the rise of deep learning, the U-Net network, based on a U-shaped architecture and skip connections, has found widespread application in various medical image segmentation tasks. However, the receptive field of the standard convolution operation is limited, because it is difficult to achieve global and long-distance semantic information interaction. Inspired by the advantages of ConvNext and Neighborhood Attention (NA), we propose CAT-Unet in this study to address the aforementioned challenges. We effectively reduce the number of parameters by utilizing large kernels and depthwise separable convolutions. Meanwhile, we introduce a Coordinate Attention (CA) module, which enables the model to learn more comprehensive and contextual information from surrounding regions. Furthermore, we introduce Skip-NAT (Neighborhood Attention Transformer) as the main algorithmic framework, replacing U-Net's original skip-connection layers, to lessen the impact of shallow features on network efficiency. Experimental results show that CAT-Unet achieves better segmentation results. On the ISIC2018 dataset, the best results for Dice (Dice Coefficient), IoU (Intersection over Union), and HD (Hausdorff Distance) are 90.26%, 83.58%, and 4.259, respectively. For the PH2 dataset, the best Dice, IoU, and HD results are 96.49%, 91.81%, and 3.971, respectively. Finally, on the DSB2018 dataset, the best Dice, IoU, and HD results are 94.58%, 88.78%, and 3.749, respectively.

Visual-guided scene-aware audio generation method based on hierarchical feature codec and rendering decision

Visually guided spatial sound generation (VGSSG) is a well-suited multimodal learning method for dealing with recorded videos. However, existing methods are difficult to be directly applied to spatial sound generation for movie clips. This is mainly due to (1) the existence of Cinematic Audiovisual Language (CAL) in movies, which makes it difficult to construct spatial sound mapping models directly through data-driven based methods. (2) The problem of the inadequate model performance, which is caused by the excessive heterogeneous gap between audiovisual modal information. To solve the aforementioned problems, we propose a VGSSG method based on CAL decision-making and hierarchical feature coding and decoding, which effectively accomplishes spatial sound generation based on the CAL of movies. Specifically, to solve the problem of CAL modeling, a multimodal information-guided movie audio rendering decision maker is established, which can decide the rendering strategy based on the CAL of the current clip. To narrow the heterogeneous gap that hinders the fusion between audiovisual modal data, we propose a codec structure based on hierarchical fusion of audiovisual features and full-scale skip-connections, which improves the efficiency of the comprehensive utilization of audiovisual modal data, and demonstrates the effectiveness of adopting shallow features in VGSSG task. We integrate both 2-channel and 6-channel spatial audio generation into a unified framework. In addition, we establish a movie audiovisual bimodal dataset with hand-crafted CAL annotations. Experimentally, we demonstrate that compared with the existing methods, our method has higher performance in terms of reducing generation distortion.

Construction site environment helmet detection method based on multi-scale context and attention fusion

Construction site environment helmet detection is of great significance for protecting workers’ lives and realizing the automation of safety management. Aiming at the current object detection methods for the complex construction site environment in the small-scale helmet object detection ability is insufficient. This paper proposes a construction site environment helmet detection method based on multi-scale context and attention fusion. The method is able to aggregate the multi-scale contextual semantics of deep image features through the proposed multi-scale context module and expand the receptive field in order to improve the network’s discriminative learning ability for small-scale helmet objects. Meanwhile, the proposed attention feature fusion module dynamically fuses features from shallow features and network decoding features to enhance the network’s ability to learn the expression of global feature dependencies and local spatial detail features of helmet objects, and further improve the network’s detection precision of helmet objects. The experimental results show that on the constructed safety helmet wearing dataset, the proposed method in this paper has good detection effect and balanced detection speed compared with the existing mainstream object detection methods.

Deep Fusion Module for Video Action Recognition

In video action recognition, effective spatiotemporal modeling is crucial. However, traditional two-stream methods face challenges in integrating spatial information from RGB images and temporary information from optical flow without long-range temporal modelling. To address these limitations, we propose the Deep Fusion Module (DFM), which focuses on the deep fusion of spatial and temporal information and consists of two components. First, we propose an Attention Fusion Module (AFM) to effectively fuse the shallow features obtained from a two-stream network, thereby facilitating the integration of spatial and temporal information. Next, we incorporate a SpatioTemporal Module (STM), comprising a ConvGRU and a 1×1 convolution, to model long-range temporal dependency and fuse spatial-temporal features. Experiments on the UCF101 dataset show that our method achieves 96.5% accuracy, outperforming baseline two-stream models by 0.3%.

Surface defect detection of hot rolled steel based on multi-scale feature fusion and attention mechanism residual block

To improve the precision of defect categorization and localization in images, this paper proposes an approach for detecting surface defects in hot-rolled steel strips. The approach uses an improved YOLOv5 network model to overcome the issues of inadequate feature extraction capacity and suboptimal feature integration when identifying surface defects on steel strips. The proposed method achieves higher detection accuracy and localization precision, making it more competitive and applicable in real production. Firstly, the multi-scale feature fusion (MSF) strategy is utilized to fuse shallow and deep features effectively and enrich detailed information relevant to target defects. Secondly, the CSPLayer Res2Attention block (CRA block) residual module is introduced to reduce the loss of defect information during hierarchical transmission, thereby enhancing the extraction of fine-grained features and improving the perception of details and global features. Finally, the experimental results indicate that the mAP on the NEU-DET and GC10-DET datasets approaches 78.5% and 67.3%, respectively, which is 4.9% and 2.1% higher than that of the baseline. Meanwhile, it has higher precision and more precise localization capabilities than other methods. Furthermore, it also achieves 59.2% mAP on the APDDD dataset, indicating its potential for growth in further domains.

ECG Classification using Deep Learning

Abstract: The importance of ECG categorization has grown significantly, as demonstrated by the many contemporary medical applications that address this issue. A plethora of machine learning (ML) tools are already available for the analysis and classification of ECG data. The primary drawbacks of these machine learning outcomes are the utilization of heuristic handcrafted or manufactured features with shallow feature learning architectures, or the acquisition of numerical data using image processing. The use of CNN, which recognizes and captures specific patterns in images, is one suggested solution for this problem utilizing deep learning architecture. Recurrent neural networks (RNN) and their variant LSTM are advantageous in the ECG classification task due to their implicit capacity to effectively capture temporal dependencies and noise while providing interpretability of hidden states. In contrast to human results, these Deep Learning Techniques yield results about the employment of Recurrent Neural Networks with LSTM layers and Convolutional Neural Networks without any featureengineered procedure and competing classification accuracy

Few-shot learning based on hierarchical feature fusion via relation networks

Few-shot learning, which aims to identify new classes with few samples, is an increasingly popular and crucial research topic in the machine learning. Recently, the development of deep learning has deepened the network structure of a few-shot model, thereby obtaining deeper features from the samples. This trend led to an increasing number of few-shot learning models pursuing more complex structures and deeper features. However, discarding shallow features and blindly pursuing the depth of sample feature levels is not reasonable. The features at different levels of the sample have different information and characteristics. In this paper, we propose a few-shot image classification model based on deep and shallow feature fusion and a coarse-grained relationship score network (HFFCR). First, we utilize networks with different depth structures as feature extractors and then fuse the two kinds of sample features. The fused sample features collect sample information at different levels. Second, we condense the fused features into a coarse-grained prototype point. Prototype points can better represent the information in this class and improve classification efficiency. Finally, we construct a relationship score network, concatenating the prototype points and query samples into a feature map and sending it into the network to calculate the relationship score. The classification criteria for learnable relationship scores reflect the information difference between the two samples. Experiments on three datasets show that HFFCR has advanced performance.

S-YOLOv5: A Lightweight Model for Detecting Objects Thrown from Tall Buildings in Communities

Objects thrown from tall buildings in communities are characterized by their small size, inconspicuous features, and high speed. Existing algorithms for detecting such objects face challenges, including excessive parameters, overly complex models that are difficult to implement, and insufficient detection accuracy. This study proposes a lightweight detection model for objects thrown from tall buildings in communities, named S-YOLOv5, to address these issues. The model is based on the YOLOv5 algorithm, and a lightweight convolutional neural network, Enhanced ShuffleNet (ESNet), is chosen as the backbone network to extract image features. On this basis, the initial stage of the backbone network is enhanced and the simplified attention module (SimAM) attention mechanism is added to utilize the rich position information and contour information in the shallow feature map to improve the detection of small targets. For feature fusion, the sparsely connected Path Aggregation Network (SCPANet) module is designed to use sparsely connected convolution (SCConv) instead of the regular convolution of the Path Aggregation Network (PANet) to fuse features efficiently. In addition, the model uses the normalized Wasserstein distance (NWD) loss function to reduce the sensitivity of positional bias. The accuracy of the model is further improved. Test results from the self-built objects thrown from tall buildings dataset show that S-YOLOv5 can detect objects thrown from tall buildings quickly and accurately, with an accuracy of 90.2% and a detection rate of 34.1 Fps/s. Compared with the original YOLOv5 model, the parameters are reduced by 87.3%, and the accuracy and rate are improved by 0.8% and 63%, respectively.

Industrial process fault diagnosis based on domain adaptive broad echo network

BackgroundIn response to the challenge that traditional fault diagnosis models are difficult to maintain satisfactory accuracy when data distribution changes due to changes in process conditions, a fault diagnosis model of industrial process based on domain adaptive broad echo network (DABEN) is proposed. MethodsThe DABEN model first constructs feature nodes through random feature mapping to extract shallow features of process data, and then inputs feature nodes into cascade reservoirs to extract dynamic features of different levels. On this basis, the objective function of DABEN is constructed, which starts from the four goals of minimizing the prediction error, maximum mean discrepancy distribution alignment, manifold regularization and minimizing cross-domain error to ensure that the features are as similar as possible between the source domain and the target domain. Significant FindingsFinally, two simulation cases show that DABEN can achieve good transfer fault diagnosis performance under different data distributions

Multi-scale object detection in UAV images based on adaptive feature fusion.

With the widespread use of UAVs, UAV aerial image target detection technology can be used for practical applications in the military, traffic planning, personnel search and rescue and other fields. In this paper, we propose a multi-scale UAV aerial image detection method based on adaptive feature fusion for solving the problem of detecting small target objects in UAV aerial images. This method automatically adjusts the convolution kernel receptive field and reduces the redundant background of the image by adding an adaptive feature extraction module (AFEM) to the backbone network. This enables it to obtain more accurately and effectively small target feature information. In addition, we design an adaptive feature weighted fusion network (SBiFPN) to effectively enhance the representation of shallow feature information of small targets. Finally, we add an additional small target detection scale to the original network to expand the receptive field of the network and strengthen the detection of small target objects. The training and testing are carried out on the VisDrone public dataset. The experimental results show that the proposed method can achieve 38.5% mAP, which is 2.0% higher than the baseline network YOLOv5s, and can still detect the UAV aerial image well in complex scenes.

Subsurface architecture of the Wajrakarur Kimberlite Field, Dharwar Craton, India: Evidence from ground magnetic surveys

The Ground magnetic study in the Wajrakarur Kimberlite Field and adjoining area has unveiled prominent geological variations, including felsic doming beneath the PGC-II formation to the west of the Cuddapah Basin and mafic doming beneath the Closepet Granite zone. These doming features within the upper mantle are identified as sources of mantle upwelling, which influence the emplacement of corresponding intrusive rocks. The presence of mafic chambers enhances the magnetic response, leading to localized high signatures in magnetic data and contributes to a better understanding of the subsurface composition and distribution of rock types. The present magnetic data throw lights to study the complexity of the sub-surface structure and geological activities, have shaped the study area and serve as valuable markers for understanding the geological evolution of the area. The intersections of magnetic lineament oriented in NW-SE, NE-SW, E-W yield crucial insights into geological structures and the emplacement of Kimberlite intrusions. Through comprehensive magnetic data analysis, the study offers a unique cross-sectional view of the subsurface, reaching depth up to 60 km. This perspective unveils magnetic sources and structures, revealing shallow features, interfaces of the Archaean basement, crustal roots, and upper mantle configuration. Profound magnetic gradients observed in the study area serve as indicators of fault lines influenced by tectonic activity, kimberlite intrusion, and other fault-related mineralization if any. These features contribute to a deeper understanding of the structural configurations and geological history of the area. The present study provides information about the lateral and vertical layouts of major structures as well as the deep-seated channels. It also introduces the mafic and felsic categories of rocks, explaining the complex relationships between underlying structures, geological characteristics, and magnetic anomalies.

YOLOLayout: Multi-Scale Cross Fusion Former for Document Layout Analysis

Document layout analysis (DLA) is a technique used to locate and classify layout elements in a document, such as Table, Figure, List, and Text. While deep-learning-based methods in computer vision have shown excellent performance in detecting Text and Figures, they are still unsatisfactory in accurately recognizing the blocks of List, Title, and Table categories with limited data. To address this issue, we propose a single-stage DLA model that incorporates a Multi-Scale Shallow Visual Feature Enhancement Module (MS-SVFEM) and a Multi-Scale Cross-Feature Fusion Module (MS-CFF). The MS-SVFEM extracts multi-scale spatial information through the channel attention module, spatial attention module, and multi-branch convolution. The MS-CFF fuses different level features through an attention mechanism. The experiments showed that the mAP accuracy of YOLOLayout compared to the baseline model is 2.2% and 1.5% higher on the PubLayNet Dataset and the ISCAS-CLAD dataset.

Flexible drug-target interaction prediction with interactive information extraction and trade-off

Drug-target interaction (DTI) prediction refers to the use of computational methods and models to predict the interaction between drugs and biological targets. DTI can help researchers understand the mechanism of action of drugs, discover new drug targets, and screen drug candidates. Recently, a large number of DTI models integrating deep drug-target interaction features have emerged to make up for the dilemma of incomplete information on shallow drug and target features. However, these models ignore the challenge of overlapping interaction information by simply integrating deep interaction information. This paper proposes a flexible DTI prediction with interactive information extraction and trade-off (FDTIIT) to address the above challenges. The main idea of FDTIIT is to use flexible mutual attention to extract interaction information about drugs and targets, and then limit the dependence between them to avoid redundant information. Specifically, FDTIIT mainly includes three parts: drug and target representation, drug-target interactive information extraction, and drug-target interactive information trade-off. Among them, the drug and target representation module mainly uses the graph convolutional network and convolutional neural network to learn the representation of drugs and targets. Then, the drug-target interactive information extraction module extracts the drug information hidden in the target and the target information hidden in the drug based on mutual attention. To avoid possible information overlap between drug representation and target representation after the fusion of interaction information, FDTIIT designs an interactive information trade-off module. This module limits the dependence between drug and target representation, providing more comprehensive information to support high-performance drug-target interaction prediction. Multiple experiments designed on three publicly available datasets validated FDTIIT’s effectiveness.

STFE-Net: A multi-stage approach to enhance statistical texture feature for defect detection on metal surfaces

Statistical texture features are essential for metal surface defect detection. However, low contrast and cluttered backgrounds exacerbate the intrinsic blurriness and variability of defect textures, hampering accurate identification and localization. To tackle these challenges, we propose a Statistical Texture Feature Enhancement Network (STFE-Net) based on YOLO model, a multi-stage approach based on pooling attention mechanism and cross-scale shallow feature reinforcement. Firstly, we enhance the image contrast and propose a Stacked Pooling Attention Residual (SPA-Res) module to capture better texture features; then, we embed the Shallow Feature Weighted (SFW) branches to achieve cross-layer shallow feature reinforcement and add feature layer (AFL) to extract rich semantic features; and finally, we propose an Efficient Complete Intersection over Union (ECIoU) loss function and employ four detection heads to achieve precise localization of defects. On the NEU-DET dataset, our model achieves the state-of-the-art performance. Especially, the mean Average Precision (mAP@IoU = 0.5) of our model is 82.3 %, which is 6.2 % higher than baseline network. With the input resolution of 640 × 640, STFE-Net achieves 54 FPS inference speed with a GeForce GTX 1080. In addition, the experimental results on the DAGM dataset and GC10-DET dataset demonstrate that the proposed model has excellent generalization ability.