High-resolution Network Research Articles

Automatic axial vertebral rotation estimation on radiographs for adolescent idiopathic scoliosis by deep learning

PurposeAdolescent idiopathic scoliosis (AIS) is a three-dimensional spine deformity governed by lateral curvature and axial vertebral rotation (AVR). Estimating AVR is important for treatment decisions and the prediction of AIS progression. However, manual AVR measurements have intra-observer and inter-observer errors, and therefore this paper proposes an automatic AVR measurement method to address the low precision issue of manual measurement. MethodWe develop an improved feature extraction module for vertebral landmark detection and pedicle segmentation. The improved coordinate convolution layer combined Polarized Self-Attention mechanism is applied in the feature extraction module to help the High-resolution Network to extract coordinate information. Based on vertebral landmark detection and pedicle segmentation, we propose an automatic AVR measurement algorithm to estimate the AVR. ResultsThe mean radial error (MRE) of vertebral landmark detection is 2.70 mm pixels, and the dice coefficient of pedicle segmentation is 72.45%. Compared with the original model, the MRE decreases by 2.13 mm, and the dice coefficient improves by 4.61%. For the AVR measurement results, the testing set contains 37 spine radiographs, including 481 vertebrae and 962 AVR measurements (481 vertebrae by two observers). The average measurement error is lower than 5°, which is within the standard of clinical measurement error. ConclusionThe results demonstrate that the proposed network performs well in vertebral landmark detection and pedicle segmentation, and the proposed AVR measurement is adopted for clinic diagnosis of AIS. SignificanceOur method achieves automatic AVR measurement, reducing the error introduced by manual measurement and improving the efficiency of orthopedists.

A High-Resolution Network with Strip Attention for Retinal Vessel Segmentation

Accurate segmentation of retinal vessels is an essential prerequisite for the subsequent analysis of fundus images. Recently, a number of methods based on deep learning have been proposed and shown to demonstrate promising segmentation performance, especially U-Net and its variants. However, tiny vessels and low-contrast vessels are hard to detect due to the issues of a loss of spatial details caused by consecutive down-sample operations and inadequate fusion of multi-level features caused by vanilla skip connections. To address these issues and enhance the segmentation precision of retinal vessels, we propose a novel high-resolution network with strip attention. Instead of the U-Net-shaped architecture, the proposed network follows an HRNet-shaped architecture as the basic network, learning high-resolution representations throughout the training process. In addition, a strip attention module including a horizontal attention mechanism and a vertical attention mechanism is designed to obtain long-range dependencies in the horizontal and vertical directions by calculating the similarity between each pixel and all pixels in the same row and the same column, respectively. For effective multi-layer feature fusion, we incorporate the strip attention module into the basic network to dynamically guide adjacent hierarchical features. Experimental results on the DRIVE and STARE datasets show that the proposed method can extract more tiny vessels and low-contrast vessels compared with existing mainstream methods, achieving accuracies of 96.16% and 97.08% and sensitivities of 82.68% and 89.36%, respectively. The proposed method has the potential to aid in the analysis of fundus images.

Edge Intelligence Empowered Vehicle Detection and Image Segmentation for Autonomous Vehicles

Edge intelligence (EI) migrates data and artificial intelligence (AI) to the “edge” of a network, enhancing the high-bandwidth and low-latency of wireless data transmission with the multiplier effect of 5G and AI, greatly improving the edges’ processing speed. Through integrating EI and computer vision technology, video surveillance systems in ITS can improve the processing capability of traffic information, which improves traffic efficiency and ensures traffic safety. Accordingly, first, we propose an edge intelligence-based improved-YOLOv4 vehicle detection algorithm, introducing an efficient channel attention (ECA) mechanism and a high-resolution network (HRNet) to enhance vehicle detection ability. Second, an edge intelligence-based improved DeepLabv3+ image segmentation algorithm is proposed, replacing the original backbone network with MobileNetv2 and using the softpool method, thus reducing the network size while improving the segmentation accuracy. Experimental results show that our proposed model has a higher average precision (AP) and can improve vehicle detection accuracy from 82.03% to 86.22%. The mean intersection over union (mIOU) of the image segmentation model improves from 73.32% to 75.63%.

An optimization high-resolution network for human pose recognition based on attention mechanism

An optimization high-resolution network for human pose recognition based on attention mechanism

RS-RCNN: an indoor window detection algorithm for autonomous spraying robot

The limitations of deep learning detection algorithms based on convolutional neural networks stem from the local characteristics inherent in convolutional operations. To overcome this issue, an RS-RCNN (ResNet_50 + Swin Transformer RCNN) object detection algorithm is proposed as an extension of Faster-RCNN. This algorithm incorporates the fusion of ResNet_50 and Swin Transformer networks as backbone networks for feature extraction. Moreover, it integrates the AAM_HRFPN (Attention Aggregation Module High resolution network) multi-feature fusion network and incorporates a linear attention mechanism. By employing the SIoU loss calculation method, the proposed RS-RCNN network enhances the representation of global context information and local semantic information, while improving fusion efficiency and detection accuracy. Compared to typical object detection networks, the proposed network achieves an AP value of 0.877, which is 7.4% points higher than the original network. The successful application of this method presents a novel solution for robot detection of non-spraying areas.

HS-YOLO: Small Object Detection for Power Operation Scenarios

Object detection methods are commonly employed in power safety monitoring systems to detect violations in surveillance scenes. However, traditional object detection methods are ineffective for small objects that are similar to the background information in the power monitoring scene, which consequently affects the performance of violation behavior detection. This paper proposed a small object detection algorithm named HS-YOLO, based on High-Resolution Network (HRNet) and sub-pixel convolution. First, to fully extract the microfeature information of the object, a small object feature extraction backbone network is proposed based on the HRNet structure. The feature maps of different scales are processed by multiple parallel branches and fused with each other in the network. Then, to fully retain the effective features of small objects, the sub-pixel convolution module is incorporated as the upsampling operator in the feature fusion network. The low-resolution feature map is upsampled to a higher resolution by reorganizing pixel values and performing padding operations in this module. On our self-constructed power operation dataset, the HS-YOLO algorithm achieved a mAP of 87.2%, which is a 3.5% improvement compared to YOLOv5. Particularly, the dataset’s AP for detecting small objects such as cuffs, necklines, and safety belts is improved by 10.7%, 5.8%, and 4.4%, respectively. These results demonstrate the effectiveness of our proposed method in detecting small objects in power operation scenarios.

Real-time high-resolution neural network with semantic guidance for crack segmentation

Deep learning plays an important role in crack segmentation, but most work utilize off-the-shelf or improved models that have not been specifically developed for this task. High-resolution convolution neural networks that are sensitive to objects’ location and detail help improve the performance of crack segmentation, yet conflict with real-time detection. This paper describes HrSegNet, a high-resolution network with semantic guidance specifically designed for crack segmentation, which guarantees real-time inference speed while preserving crack details. After evaluation on the composite dataset CrackSeg9k and the scenario-specific datasets Asphalt3k and Concrete3k, HrSegNet obtains state-of-the-art segmentation performance and efficiencies that far exceed those of the compared models. This approach demonstrates that there is a trade-off between high-resolution modeling and real-time detection, which fosters the use of edge devices to analyze cracks in real-world applications.

A PLANT RECOGNITION APPROACH USING HIGH RESOLUTION NETWORK

Plant species recognition plays an important role in agriculture, the pharmaceutical industry, and conservation. The traditional approaches may take days and have difficulties for non-experts. Several computer vision-based models have been proposed, which can partially assist and speed up the plant recognition process. Thanks to the development of data collection and computational systems, the models based on machine learning have considerably improved their performance in the last decades. In this paper, we present a model for plant recognition in Southeast Asia based on the high-resolution network. The evaluation is carried out on a public dataset consisting of 26 different species in Southeast Asia. It shows high accuracy in recognition.

Fundus-DeepNet: Multi-label deep learning classification system for enhanced detection of multiple ocular diseases through data fusion of fundus images

Detecting multiple ocular diseases in fundus images is crucial in ophthalmic diagnosis. This study introduces the Fundus-DeepNet system, an automated multi-label deep learning classification system designed to identify multiple ocular diseases by integrating feature representations from pairs of fundus images (e.g., left and right eyes). The study initiates with a comprehensive image pre-processing procedure, including circular border cropping, image resizing, contrast enhancement, noise removal, and data augmentation. Subsequently, discriminative deep feature representations are extracted using multiple deep learning blocks, namely the High-Resolution Network (HRNet) and Attention Block, which serve as feature descriptors. The SENet Block is then applied to further enhance the quality and robustness of feature representations from a pair of fundus images, ultimately consolidating them into a single feature representation. Finally, a sophisticated classification model, known as a Discriminative Restricted Boltzmann Machine (DRBM), is employed. By incorporating a Softmax layer, this DRBM is adept at generating a probability distribution that specifically identifies eight different ocular diseases. Extensive experiments were conducted on the challenging Ophthalmic Image Analysis-Ocular Disease Intelligent Recognition (OIA-ODIR) dataset, comprising diverse fundus images depicting eight different ocular diseases. The Fundus-DeepNet system demonstrated F1-scores, Kappa scores, AUC, and final scores of 88.56 %, 88.92 %, 99.76 %, and 92.41 % in the off-site test set, and 89.13 %, 88.98 %, 99.86 %, and 92.66 % in the on-site test set.In summary, the Fundus-DeepNet system exhibits outstanding proficiency in accurately detecting multiple ocular diseases, offering a promising solution for early diagnosis and treatment in ophthalmology.

Anomaly Perception Method of Substation Scene Based on High-Resolution Network and Difficult Sample Mining

The perception of anomalies in power scenarios plays a crucial role in the safe operation and fault prediction of power systems. However, traditional anomaly detection methods face challenges in identifying difficult samples due to the complexity and uneven distribution of power scenarios. This paper proposes a power scene anomaly perception method based on high-resolution networks and difficult sample mining. Firstly, a high-resolution network is introduced as the backbone for feature extraction, enhancing the ability to express fine details in power scenarios and capturing information on small target anomaly regions. Secondly, a strategy for mining difficult samples is employed to focus on learning and handling challenging and hard-to-recognize anomaly samples, thereby improving the overall anomaly detection performance. Lastly, the method incorporates GIOU loss and a flexible non-maximum suppression strategy to better adapt to the varying sizes and dense characteristics of power anomaly targets. This improvement enables higher adaptability in detecting anomalies in power scenarios. Experimental results demonstrate significant improvements in power scene anomaly perception and superior performance in handling challenging samples. This study holds practical value for fault diagnosis and safe operation in power systems.

High-Resolution Network with Transformer Embedding Parallel Detection for Small Object Detection in Optical Remote Sensing Images

Small object detection in remote sensing enables the identification and analysis of unapparent but important information, playing a crucial role in various ground monitoring tasks. Due to the small size, the available feature information contained in small objects is very limited, making them more easily buried by the complex background. As one of the research hotspots in remote sensing, although many breakthroughs have been made, there still exist two significant shortcomings for the existing approaches: first, the down-sampling operation commonly used for feature extraction can barely preserve weak features of objects in a tiny size; second, the convolutional neural network methods have limitations in modeling global context to address cluttered backgrounds. To tackle these issues, a high-resolution network with transformer embedding parallel detection (HRTP-Net) is proposed in this paper. A high-resolution feature fusion network (HR-FFN) is designed to solve the first problem by maintaining high spatial resolution features with enhanced semantic information. Furthermore, a Swin-transformer-based mixed attention module (STMA) is proposed to augment the object information in the transformer block by establishing a pixel-level correlation, thereby enabling global background–object modeling, which can address the second shortcoming. Finally, a parallel detection structure for remote sensing is constructed by integrating the attentional outputs of STMA with standard convolutional features. The proposed method effectively mitigates the impact of the intricate background on small objects. The comprehensive experiment results on three representative remote sensing datasets with small objects (MASATI, VEDAI and DOTA datasets) demonstrate that the proposed HRTP-Net achieves a promising and competitive performance.

Variation-learning high-resolution network for capsulorhexis recognition of cataract surgery

In cataract surgery, capsulorhexis is the first, one of the most critical steps. However, recognizing the margin of rhexis during capsulorhexis is challenging and has rarely been studied due to its intraoperative invisibility, variable shape, and lack of a proprietary labeled dataset. Capsulorhexis margin recognition is essential for robot-assisted surgery, surgical assessment, and intraoperative reference. To solve the problem, this paper proposes a creative dynamic margin recognition structure called THRC, which includes a novel segmentation network, THRNet, and a compensation block. First, the dilated HRNet is designed to expand the receptive field while maintaining high-resolution representation for better detail preservation. Second, cascaded transformer blocks are proposed. The channel transformer block (CTB) is developed to include fixed and dynamic weights while generating global context dependency for nonrigid tissue variation pattern learning. The layer transformer block (LTB) with redistributed transformer variables is then developed to fuse multi-resolution features effectively for better multi-scale semantic segmentation. Moreover, the compensation block compensates for inter-frame movement by key-point registration and recovers a more realistic margin by a smoothing process. Finally, to evaluate the proposed structure, we establish the capsulorhexis dataset CapsSeg. The THRNet achieves 78.24% mean intersection over union (mIoU) and 86.59% mean Dice (mDice), making it equivalent in performance to state-of-the-art networks, but with a lower computational cost. The feasibility of the compensation block is also visualized.

Deep learning to obtain high-throughput morphological phenotypes and its genetic correlation with swimming performance in juvenile large yellow croaker

Breeding for swimming performance in fish may cause changes in morphological traits in offspring, which are not only related to industrial efficiency, but also affect animal welfare. However, the genetic correlation between swimming performance and morphological traits in juvenile large yellow croaker remains unclear, and traditional morphological traits collection is time-consuming and laborious. Deep learning offers new opportunities for high-throughput phenomics, especially for complex morphological traits. In this study, the automatic High-Resolution Network (HRNet) based on a deep learning approach was used for the first time to effectively detect the morphological traits of juvenile large yellow croaker. We assessed the swimming performance (Ucrit) of 1300 juvenile fish, and conducted a genome-wide association study (GWAS) and genetic parameters study on the morphological traits of 383 juvenile fish using a 55 K SNP array. The morphological traits data of 383 fish comes from the predicted results of the HRNet model. The results showed that: (1) All of the 10 morphological traits showed positively correlated with Ucrit, and mostly positive genetic correlated with Ucrit; (2) The heritability of all 10 morphological traits was low to moderate; (3) We identified 4 SNPs significantly associated with morphological traits, and identified candidate genes including sox8, setd6, and ctbp2, and enriched significant pathways including Notch signaling pathway. In conclusion, this study provides an efficient and multi-context adaptive HRNet based on deep learning for high-throughput morphological traits detection of juvenile large yellow croaker, and analyses the genetic basis between morphological traits and swimming performance for this species, which provides a new understanding of the genetic basis of morphological traits of large yellow croaker, and provides a basis for breeding programs.

CFRLA-Net: A Context-Aware Feature Representation Learning Anchor-Free Network for Pedestrian Detection

High resolution and strong semantic representation are both vital for feature extraction networks of pedestrian detection. The existing high-resolution network (HRNet) has presented a promising performance for pedestrian detection. However, we observed that it still has some significant shortcomings for heavily occluded and small-scale pedestrians. In this paper, we propose to address the shortcomings by extracting semantic and spatial context from HRNet. Specifically, we propose a Context-aware Feature Representation Learning Module (CFRL-Module), which combines a Multi-scale Feature Context Extraction Parallel Block for Convolution and Self-attention (CEPCA-Block) with two parallel paths and an Equivalent FFN (EFFN) Block. The core CEPCA-Block adopts a parallel design to integrate convolution and multi-head self-attention (MHSA) with low parameter computational cost, which can obtain the deep semantic context by convolution path and precise context by MHSA path. Furthermore, to overcome the inefficiency of global MHSA in high-resolution pedestrian detection, we propose a novel local window MHSA, which can significantly reduce memory consumption but barely affect the detection performance. Cascading the proposed CFRL-Module with the anchor-free detection head constitutes our Context-aware Feature Representation Learning Anchor-Free Network (CFRLA-Net). The proposed CFRLA-Net can catch a high-level understanding of the heavily occluded and small-scale pedestrian instances based on HRNet, which can effectively solve the limitation of the insufficient feature extraction ability of HRNet for the hard samples. Experimental results show that CFRLA-Net achieves state-of-the-art performance on CityPersons, Caltech, and CrowdHuman benchmarks.

Optimization of ICANet lightweight human pose estimation based on HRNet

The existing high-resolution human pose estimation models have low applicability in practical applications due to their large parameter quantity and high computational complexity. To address these issues, this paper proposes a human pose estimation network with a lightweight inverted residual coordinate attention network (ICANet) based on the high-resolution network (HRNet). With the introduction of CoordAttention mechanism and the inverted residual module, this paper proposes two lightweight network modules, namely ICAneck and ICAblock, which not only reduce model parameter quantity and computational complexity, but also achieve feature enhancement of long-range dependence and precise position information in spatial directions of the feature map. Experimental results show that compared to HRNet, the ICANet model proposed in this paper reduces its parameter quantity by 53.7% and computational complexity by 32.4% on the COCO validation set, and lowers its parameter amount by 53.7% and computational complexity by 32.6% on the MPII validation set. Practical applications prove that ICANet still achieves high-precision detection of human key points with fewer parameters and lower computational complexity, and has higher applicability and practicality compared with common human pose estimation networks such as the Stacked Hourglass Network (Hourglass), Cascaded Pyramid Network (CPN), and SimpleBaseline, and therefore has better applicability and practicality.

VHR-BirdPose: Vision Transformer-Based HRNet for Bird Pose Estimation with Attention Mechanism

Pose estimation plays a crucial role in recognizing and analyzing the postures, actions, and movements of humans and animals using computer vision and machine learning techniques. However, bird pose estimation encounters specific challenges, including bird diversity, posture variation, and the fine granularity of posture. To overcome these challenges, we propose VHR-BirdPose, a method that combines Vision Transformer (ViT) and Deep High-Resolution Network (HRNet) with an attention mechanism. VHR-BirdPose effectively extracts features using Vision Transformer’s self-attention mechanism, which captures global dependencies in the images and allows for better capturing of pose details and changes. The attention mechanism is employed to enhance the focus on bird keypoints, improving the accuracy of pose estimation. By combining HRNet with Vision Transformer, our model can extract multi-scale features while maintaining high-resolution details and incorporating richer semantic information through the attention mechanism. This integration of HRNet and Vision Transformer leverages the advantages of both models, resulting in accurate and robust bird pose estimation. We conducted extensive experiments on the Animal Kingdom dataset to evaluate the performance of VHR-BirdPose. The results demonstrate that our proposed method achieves state-of-the-art performance in bird pose estimation. VHR-BirdPose based on bird images is of great significance for the advancement of bird behaviors, ecological understanding, and the protection of bird populations.

E-HRNet: Enhanced Semantic Segmentation Using Squeeze and Excitation

In the field of computer vision, convolutional neural network (CNN)-based models have demonstrated high accuracy and good generalization performance. However, in semantic segmentation, CNN-based models have a problem—the spatial and global context information is lost owing to a decrease in resolution during feature extraction. High-resolution networks (HRNets) can resolve this problem by keeping high-resolution processing layers parallel. However, information loss still occurs. Therefore, in this study, we propose an HRNet combined with an attention module to address the issue of information loss. The attention module is strategically placed immediately after each convolution to alleviate information loss by emphasizing the information retained at each stage. To achieve this, we employed a squeeze-and-excitation (SE) block as the attention module, which can seamlessly integrate into any model and enhance the performance without imposing significant parameter increases. It emphasizes the spatial and global context information by compressing and recalibrating features through global average pooling (GAP). A performance comparison between the existing HRNet model and the proposed model using various datasets show that the mean class-wise intersection over union (mIoU) and mean pixel accuracy (MeanACC) improved with the proposed model, however, there was a small increase in the number of parameters. With cityscapes dataset, MeanACC decreased by 0.1% with the proposed model compared to the baseline model, but mIoU increased by 0.5%. With the LIP dataset, the MeanACC and mIoU increased by 0.3% and 0.4%, respectively. The mIoU also decreased by 0.1% with the PASCAL Context dataset, whereas the MeanACC increased by 0.7%. Overall, the proposed model showed improved performance compared to the existing model.

Underwater Target Detection Based on Parallel High-Resolution Networks.

A parallel high-resolution underwater target detection network is proposed to address the problems of complex underwater scenes and limited target feature extraction capability. First, a high-resolution network (HRNet), a lighter high-resolution human posture estimation network, is used to improve the target feature representation and effectively reduce the semantic information lost in the image during sampling. Then, the attention module (A-CBAM) is improved to capture complex feature distributions by modeling the two-dimensional space in the activation function stage through the introduction of the flexible rectified linear units (FReLU) activation function to achieve pixel-level spatial information modeling capability. Feature enhancement in the spatial and channel dimensions is performed to improve understanding of fuzzy targets and small target objects and to better capture irregular and detailed object layouts. Finally, a receptive field augmentation module (RFAM) is constructed to obtain sufficient semantic information and rich detail information to further enhance the robustness and discrimination of features and improve the detection capability of the model for multi-scale underwater targets. Experimental results show that the method achieves 81.17%, 77.02%, and 82.9% mean average precision (mAP) on three publicly available datasets, specifically underwater robot professional contest (URPC2020, URPC2018) and pattern analysis, statistical modeling, and computational learning visual object classes (PASCAL VOC2007), respectively, demonstrating the effectiveness of the proposed network.

A standards-based approach to enhancing interoperability of low-cost industrial IoT flood sensors for transportation system resilience

Transportation infrastructure assets are particularly vulnerable to natural hazards, which are becoming more frequent and severe due to climate change and extreme weather conditions. Floods and flash floods are among the deadliest natural hazards, accounting for 50% of vehicle-related fatalities. This underscores the need for timely transportation flood detection systems adoption. However, current flood detection technologies are inadequate in terms of coverage, speed, geographical specificity, and interoperability, making it difficult for emergency managers to respond effectively to flood events. To address this issue, we propose a high-resolution network of low-cost Industrial Internet of Things (IIoT) sensing devices deployed at flood-prone transportation assets. These sensors collect location-specific data, which is then published in standardized formats, interfaces, and protocols, enabling other systems to generate flood forecasts, nowcasts, and warnings. A framework for Incident Management Systems (IMS) was also discussed to highlight the need for system interoperability during disaster management operations. Our solution employs standards-based interoperability, using the OIIE™ OpenO&M ecosystem architecture, to enable seamless interaction between interdependent systems and manages the risk to critical transportation infrastructure. The technology was tested at a microscale level to evaluate its performance. The model architecture supports scalable systems of systems interoperability for standardized use cases and common asset classes used in transportation, energy, facilities, and other critical infrastructure.

Detection of Hindwing Landmarks Using Transfer Learning and High-Resolution Networks.

Hindwing venation is one of the most important morphological features for the functional and evolutionary analysis of beetles, as it is one of the key features used for the analysis of beetle flight performance and the design of beetle-like flapping wing micro aerial vehicles. However, manual landmark annotation for hindwing morphological analysis is a time-consuming process hindering the development of wing morphology research. In this paper, we present a novel approach for the detection of landmarks on the hindwings of leaf beetles (Coleoptera, Chrysomelidae) using a limited number of samples. The proposed method entails the transfer of a pre-existing model, trained on a large natural image dataset, to the specific domain of leaf beetle hindwings. This is achieved by using a deep high-resolution network as the backbone. The low-stage network parameters are frozen, while the high-stage parameters are re-trained to construct a leaf beetle hindwing landmark detection model. A leaf beetle hindwing landmark dataset was constructed, and the network was trained on varying numbers of randomly selected hindwing samples. The results demonstrate that the average detection normalized mean error for specific landmarks of leaf beetle hindwings (100 samples) remains below 0.02 and only reached 0.045 when using a mere three samples for training. Comparative analyses reveal that the proposed approach out-performs a prevalently used method (i.e., a deep residual network). This study showcases the practicability of employing natural images-specifically, those in ImageNet-for the purpose of pre-training leaf beetle hindwing landmark detection models in particular, providing a promising approach for insect wing venation digitization.