Pyramid Space Research Articles

Lightweight Object Detection Based on Autonomous Driving

Abstract Addressing the issues of low accuracy and large size for tiny objects like pedestrians, non-motor vehicles, and vehicles in traditional object detection models for autonomous driving. An advanced variant of the YOLOv7-tiny model for automatic driving object detection is proposed in this study. The VoVEGSCSP module, designed for cross-layer integration within local networks, is introduced. The SPPCSPC module was replaced with the Space Pyramid Pool Fast (SPPF). This reduces parameters and calculations while preserving the diversity of the original feature scale. CARAFE is a lightweight universal upsampling operator that replaces nearest neighbor interpolation in the upsampling module, thereby reducing feature information loss during upsampling. Results demonstrate that on the KITTI dataset, the improved YOLOv7-tiny algorithm reduces parameters by 35.7% and computation by 34.1%, with only a 0.7% decrease in mAP@0.5 accuracy.

Advancing high-resolution remote sensing: a compact and powerful approach to semantic segmentation

ABSTRACT Deep learning (DL)-based approaches are notable for their ability to establish feature associations without relying on physical constraints, unlike traditional strategies that are complex and dependent on expert experience. However, three main challenges hinder the versatility of semantic segmentation models. First, the targets in these images are dense and exist at varying spatial scales, which imposes higher demands on the model for accurate segmentation across scales. Second, the segmentation of small targets in the images is often overlooked, leading to a compromise between fine segmentation and model efficiency. Lastly, the data-intensive nature of remote sensing images and the resource-intensive operations of large-scale networks impose significant communication and computation burdens on edge devices, which may not have sufficient resources to handle them effectively. To address these challenges, this paper proposes a lightweight semantic segmentation method for remote sensing images to achieve high-precision segmentation for multi-scale targets while maintaining low computational complexity. The main components include: (1) embedding the inverted residual block structure to minimize the number of model parameters and computational costs; (2) introducing the parallel irregular space pyramid pooling module to efficiently aggregate multi-scale contextual information for fine-grained recognition of small targets; and (3) embedding transfer learning into the encoder-decoder structure to speed up the convergence rate and improve multi-scale feature fusion capability, thereby reducing semantic information loss. The proposed lightweight method has been extensively tested on real-world high-resolution remote sensing datasets. It achieved PA, MPA, MIoU, and FWIoU scores of 87.90%, 75.76%, 66.29%, and 78.81% on the Vaihingen dataset; 87.03%, 85.31%, 74.85%, and 77.54% on the Potsdam dataset; and 95.37%, 83.33%, 75.70%, and 91.31% on the Aeroscapes dataset. Compared to other popular semantic segmentation models, the proposed method achieved the highest values in all four evaluation indicators, demonstrating its effectiveness and superiority.

Hierarchical optimization by spatial-temporal indictor in multi-scale decision pyramid for constrained large-scale multi-objective problems

Most existing constrained multi-objective evolutionary algorithms (CMOEAs) are not so efficient when handling constrained large-scale multi-objective problems (CLSMOPs). To overcome CLSMOPs, a hierarchical optimization, based on the spatial–temporal indictor in the decision pyramid (HOSTI-DP), is proposed to determine the dynamic trade-off between feasible and infeasible solutions in the effectively shrunken searching area. The decision pyramid consists of multiple decision spaces with a set of descending numbers of variables; it is initially constructed to speed up the solution convergence by the layer-by-layer dimensionality reduction and increase the population diversity based on the accumulation of these layers. The spatial–temporal indictor considers both the optimization stage (time) and the hierarchy structure of the decision pyramid (space) and it is designed to restrain the prematurely feasible solutions and find the promising infeasible solutions. Finally, the hierarchical optimization strategy, which applies the spatial–temporal indicator to the evolutionary algorithms in every layer of the decision pyramid and dynamically arranging the proportion among them, is designed for comprehensive balance of objectives, constraints, and computational complexity. Nine representative and state-of-the-art CMOEAs have been compared with the HOSTI-DP to demonstrate its effectiveness through comparative experiments on CLSMOPs with equality and inequality constraints and 1000 decision variables. Experimental results show that HOSTI-DP can significantly improve the performance of these CMOEAs.

AENet: Image Retrieval of Kazakh Handwritten Documents Based on Attention Mechanism and Feature Aggregation

Kazakh is one of the multilingual languages of China and is widely spoken in some areas of Xinjiang, China. However, due to the fact that Kazakh is a language in which several characters are glued together to form a continuous word with a unique shape and complex structural combinations of relationships. This paper explores a solution for offline image retrieval of handwritten Kazakh words, which is a challenging task because, due to the lack of relevant datasets and the special writing morphology of the Kazakh language, traditional text image retrieval algorithms often struggle to achieve satisfactory results when dealing with writing styles that are varied and adherent to the language. Therefore, a dataset of offline Kazakh handwritten document images was created in this paper. The dataset contains 300 pages of document images with 20 500 words. Then, a new model called the ‘AENet’ is proposed. The model utilizes an attention mechanism to focus more finely on focal regions such as centers, inflection points, and contours of handwritten word images and to capture important local features from different scales. Fusion space pyramid pooling, feature aggregation, encoding operations, and feature downscaling and reconstruction are used to extract and reconstruct more representative feature representations from local to global to capture the overall information in the word images. Through experimental evaluation on Kazak‐80, Zilla‐64, and HWDB1.1‐375 datasets, it is verified that the method significantly improves the mAP for image retrieval of handwritten words, which is especially applicable to adhesive languages like Kazakh. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

Research on multi-task perception network of traffic scene based on feature fusion1

Aiming at the problems of low precision and low real-time performance when deploying to embedded platforms in existing multi-task networks, this paper proposes a traffic scene multi-task perception network model (ETS_YOLOP) based on feature fusion. Firstly, an Efficient Attention Control Aggregation Network Module (EACAN) is constructed to improve the real-time perception of the model, and the Space Pyramid Pool Fast Convolutional Module (SPPFCSPC) is used at the end of the backbone network to increase the receptive field. Finally, a Multiscale Convolution Transformer Fusion Module (CTFM) is designed in the task branch to better capture global information and rich context information. The experimental results show that compared with the YOLOP model, the ETS_YOLOP model has a significant improvement in perception accuracy, 156% in real-time performance, 0.4% increase in mAP on the object detection task, 0.5% increase in mIoU on the drivable area segmentation task, and an 11.4% increase in accuracy on the lane detection task. In order to verify the real-time perception of the model on the embedded platform, the ETS_YOLOP model is deployed on the Huawei MDC300F computing platform. Under the condition of the image input size of 640×640, the average frame rate can reach 55FPS, which can realize real-time perception on the embedded platform.

A quality grade classification method for fresh tea leaves based on an improved YOLOv8x-SPPCSPC-CBAM model

In light of the prevalent issues concerning the mechanical grading of fresh tea leaves, characterized by high damage rates and poor accuracy, as well as the limited grading precision through the integration of machine vision and machine learning (ML) algorithms, this study presents an innovative approach for classifying the quality grade of fresh tea leaves. This approach leverages an integration of image recognition and deep learning (DL) algorithm to accurately classify tea leaves’ grades by identifying distinct bud and leaf combinations. The method begins by acquiring separate images of orderly scattered and randomly stacked fresh tea leaves. These images undergo data augmentation techniques, such as rotation, flipping, and contrast adjustment, to form the scattered and stacked tea leaves datasets. Subsequently, the YOLOv8x model was enhanced by Space pyramid pooling improvements (SPPCSPC) and the concentration-based attention module (CBAM). The established YOLOv8x-SPPCSPC-CBAM model is evaluated by comparing it with popular DL models, including Faster R-CNN, YOLOv5x, and YOLOv8x. The experimental findings reveal that the YOLOv8x-SPPCSPC-CBAM model delivers the most impressive results. For the scattered tea leaves, the mean average precision, precision, recall, and number of images processed per second rates of 98.2%, 95.8%, 96.7%, and 2.77, respectively, while for stacked tea leaves, they are 99.1%, 99.1%, 97.7% and 2.35, respectively. This study provides a robust framework for accurately classifying the quality grade of fresh tea leaves.

Enhanced YOLO Network for Improving the Efficiency of Traffic Sign Detection

One important task for autonomous driving is the precise detection and recognition of road traffic signs. This research focuses on a comprehensive set of 72 distinct traffic signs that are prevalent on urban roads in China, with the goal of developing an enhanced You Only Look Once (YOLO) network model tailored for this specific task. The modifications include the omission of the terminal convolution module and Conv3 (C3) module within the backbone network. Additionally, the 32-fold downsampling is replaced with a 16-fold downsampling, and a feature fusion module with dimensions of 152 × 152 is introduced in the feature layer. To capture a more encompassing context, a novel hybrid space pyramid pooling module, referred to as Hybrid Spatial Pyramid Pooling Fast (H-SPPF), is introduced. Furthermore, a channel attention mechanism is integrated into the framework, combined with three other improved methodologies. Upon evaluation, the enhanced algorithm demonstrates impressive results, achieving a precision rate of 91.72%, a recall rate of 91.77%, and a mean average precision (mAP) of 93.88% at an intersection over union (IoU) threshold of 0.5. Additionally, the method also achieves an mAP of 75.81% for a variety of IoU criteria between 0.5 and 0.95. These achievements are validated on an augmented dataset established for this study.

High‐precision real‐time urine crystallization recognition based on dilated bilinear space pyramid ConvNext

AbstractUrinary sediment image detection, as one of the three major routine clinical tests in medical practice, is an important method for physical examination and diagnosis of urinary system diseases. Crystalluria detection is a subtask of urinary sediment image detection, focusing on detecting and identifying crystalline components in urine. To address the issues of low accuracy and inefficiency caused by small crystal granularity in crystalluria detection, we propose the Dilated Bilinear Space Pyramid ConvNext Network (DBSPC‐Net), which achieves high‐precision real‐time crystalluria detection. DBSPC‐Net ingeniously combines dilated convolution pooling with bilinear space pyramid, introducing Dilated Bilinear Space Pyramid Pooling (DBSPP) to enlarge the receptive field and capture information at multiple scales. Additionally, we utilize the Normalized Gaussian Wasserstein Distance Loss (NWDLoss) instead of Intersection over Union (IoU) to enhance the recognition of small targets. Finally, the ConvNext module is employed to fuse local and global features, enhancing urine crystal recognition accuracy and speed. The crystalluria dataset is sourced from 400 actual patients in a hospital. It comprises five main types of urine crystals, namely calcium oxalate dihydrate, calcium oxalate monohydrate, uric acid, ammonium magnesium phosphate, and cystine. Experimental results demonstrate that the proposed improved model achieves an average precision of 87.34% and a detection time of 7.9 ms per urine crystal image. DBSPC‐Net can accurately and rapidly identify crystalluria objects in scenarios involving microscope mica compensation, meeting the requirements of algorithmic detection accuracy and real‐time performance in crystalluria detection.

Semantic Segmentation of Images Based on Multi-Feature Fusion and Convolutional Neural Networks

Image semantic segmentation technology is one of the core research contents in the field of computer vision. With the improvement of computer performance and the continuous development of deep learning technology, researchers have more and more enthusiasm to study the actual effect and performance of image semantic segmentation. The results of deep semantic segmentation allow computers to have a more detailed and accurate understanding of images, and have a wide range of application needs in the fields of autonomous driving, intelligent security, medical imaging, remote sensing images, etc. However, the existing image semantic segmentation algorithms have the disadvantages of easy discontinuous results and insufficient prediction accuracy. In this paper, we take deep learning-based image semantic segmentation technology as the research object to explore the improvement of the image semantic segmentation algorithm and its application in road scenarios. First, this paper proposes MCU-Net method based on residual fusion and multi-scale contextual information. MCU-Net uses residual fusion module to deepen the network structure and improve the ability of U-Net to acquire deeper features. Then a top-down and bottom-up path is constructed for feature information between different levels, and the spatial and semantic information contained in shallow and deep features in the network is fully utilized by fusing features from different levels. In addition, an enhanced void space pyramid pooling module is added for feature information between the same levels, which enables the output features to have a larger range of semantic information. Second, this paper proposes the DAMCU-Net method based on attention mechanism and edge detection based on MCU-Net. DAMCU-Net extracts global contextual information by the attention mechanism optimization module, while fusing features using dense jump connections to facilitate the network to recover more spatial detail information during upsampling, and uses the FReLU activation function to improve the segmentation capability of the network for complex targets. For the edge information lost in the feature extraction process, the edge detection branch is added to supplement the feature information of the main path by feature fusion to achieve the optimization of the edge information.

SPA-UNet: A liver tumor segmentation network based on fused multi-scale features.

Liver tumor segmentation is a critical part in the diagnosis and treatment of liver cancer. While U-shaped convolutional neural networks (UNets) have made significant strides in medical image segmentation, challenges remain in accurately segmenting tumor boundaries and detecting small tumors, resulting in low segmentation accuracy. To improve the segmentation accuracy of liver tumors, this work proposes space pyramid attention (SPA)-UNet, a novel image segmentation network with an encoder-decoder architecture. SPA-UNet consists of four modules: (1) Spatial pyramid convolution block (SPCB), extracting multi-scale features by fusing three sets of dilated convolutions with different rates. (2) Spatial pyramid pooling block (SPPB), performing downsampling to reduce image size. (3) Upsample module, integrating dense positional and semantic information. (4) Residual attention block (RA-Block), enabling precise tumor localization. The encoder incorporates 5 SPCBs and 4 SPPBs to capture contextual information. The decoder consists of the Upsample module and RA-Block, and finally a segmentation head outputs segmented images of liver and liver tumor. Experiments using the liver tumor segmentation dataset demonstrate that SPA-UNet surpasses the traditional UNet model, achieving a 1.0 and 2.0% improvement in intersection over union indicators for liver and tumors, respectively, along with increased recall rates by 1.2 and 1.8%. These advancements provide a dependable foundation for liver cancer diagnosis and treatment.

Lightweight Model for Pavement Defect Detection Based on Improved YOLOv7.

Existing pavement defect detection models face challenges in balancing detection accuracy and speed while being constrained by large parameter sizes, hindering deployment on edge terminal devices with limited computing resources. To address these issues, this paper proposes a lightweight pavement defect detection model based on an improved YOLOv7 architecture. The model introduces four key enhancements: first, the incorporation of the SPPCSPC_Group grouped space pyramid pooling module to reduce the parameter load and computational complexity; second, the utilization of the K-means clustering algorithm for generating anchors, accelerating model convergence; third, the integration of the Ghost Conv module, enhancing feature extraction while minimizing the parameters and calculations; fourth, introduction of the CBAM convolution module to enrich the semantic information in the last layer of the backbone network. The experimental results demonstrate that the improved model achieved an average accuracy of 91%, and the accuracy in detecting broken plates and repaired models increased by 9% and 8%, respectively, compared to the original model. Moreover, the improved model exhibited reductions of 14.4% and 29.3% in the calculations and parameters, respectively, and a 29.1% decrease in the model size, resulting in an impressive 80 FPS (frames per second). The enhanced YOLOv7 successfully balances parameter reduction and computation while maintaining high accuracy, making it a more suitable choice for pavement defect detection compared with other algorithms.

A Workpiece-Dense Scene Object Detection Method Based on Improved YOLOv5

Aiming at the problem of detection difficulties caused by the characteristics of high similarity and disorderly arrangement of workpieces in dense scenes of industrial production lines, this paper proposes a workpiece detection method based on improved YOLOv5, which embeds a coordinate attention mechanism in the feature extraction network to enhance the network’s focus on important features and enhance the model’s ability to pinpoint targets. The pooling structure of the space pyramid has been replaced, which reduces the amount of calculation and further improves the running speed. A weighted bidirectional feature pyramid is introduced in the feature fusion network to realize efficient bidirectional cross-scale connection and weighted feature fusion, and improve the detection ability of small targets and dense targets. The SIoU loss function is used to improve the training speed and further improve the detection performance of the model. The average accuracy of the improved model on the self-built artifact dataset is improved by 5% compared with the original model and the number of model parameters is 14.6MB, which is only 0.5MB higher than the original model. It is proved that the improved model has the characteristics of high detection accuracy, strong robustness and light weight.

Traditional Village Building Extraction Based on Improved Mask R-CNN: A Case Study of Beijing, China

As an essential material carrier of cultural heritage, the accurate identification and effective monitoring of buildings in traditional Chinese villages are of great significance to the sustainable development of villages. However, along with rapid urbanization in recent years, many towns have experienced problems such as private construction, hollowing out, and land abuse, destroying the traditional appearance of villages. This study combines deep learning technology and UAV remote sensing to propose a high-precision extraction method for conventional village architecture. Firstly, this study constructs the first sample database of traditional village architecture based on UAV remote sensing orthophotos of eight representative villages in Beijing, combined with fine classification; secondly, in the face of the diversity and complexity of the built environment in traditional villages, we use the Mask R-CNN instance segmentation model as the basis and Path Aggregate Feature Pyramid Network (PAFPN) and Atlas Space Pyramid Pool (ASPP) as the main strategies to enhance the backbone model for multi-scale feature extraction and fusion, using data increment and migration learning as auxiliary means to overcome the shortage of labeled data. The results showed that some categories could achieve more than 91% accuracy, with average precision, recall, F1-score, and Intersection over Union (IoU) values reaching 71.3% (+7.8%), 81.9% (+4.6%), 75.7% (+6.0%), and 69.4% (+8.5%), respectively. The application practice in Hexi village shows that the method has good generalization ability and robustness, and has good application prospects for future traditional village conservation.

An Inter-Ridge Navigation Path Extraction Method Based on Res2net50 Segmentation Model

The extraction of navigation lines plays a crucial role in the autonomous navigation of agricultural robots. This work offers a method of ridge navigation route extraction, based on deep learning, to address the issues of poor real-time performance and light interference in navigation path recognition in a field environment. This technique is based on the Res2net50 model and incorporates the Squeeze-and-Excitation Networks (SE) attention mechanism to focus on the key aspects of the image. The empty space pyramid pooling module is presented to further extract high-level semantic data and enhance the network’s capacity for fine-grained representation. A skip connection is used to combine the high-level semantic characteristics and low-level textural features that are extracted. The results of the ridge prediction are then obtained, followed by the realization of the final image segmentation, through sampling. Lastly, the navigation line is fitted once the navigation feature points have been retrieved using the resulting ridge segmentation mask. The outcomes of the experiment reveal that: the Mean Intersection over Union (MIOU) and F-measure values of the inter-ridge navigation path extraction approach suggested in this paper are increased by 0.157 and 0.061, respectively, compared with the Res2net50 network. Under various illumination situations, the average pixel error is 8.27 pixels and the average angle error is 1.395°. This technique is appropriate for ridge operations and can successfully increase network prediction model accuracy.

Super-resolution reconstruction of binocular image based on multi-level fusion attention network

Super-resolution reconstruction of binocular image based on multi-level fusion attention network

DeepMDSCBA: An Improved Semantic Segmentation Model Based on DeepLabV3+ for Apple Images.

The semantic segmentation of apples from images plays an important role in the automation of the apple industry. However, existing semantic segmentation methods such as FCN and UNet have the disadvantages of a low speed and accuracy for the segmentation of apple images with complex backgrounds or rotten parts. In view of these problems, a network segmentation model based on deep learning, DeepMDSCBA, is proposed in this paper. The model is based on the DeepLabV3+ structure, and a lightweight MobileNet module is used in the encoder for the extraction of features, which can reduce the amount of parameter calculations and the memory requirements. Instead of ordinary convolution, depthwise separable convolution is used in DeepMDSCBA to reduce the number of parameters to improve the calculation speed. In the feature extraction module and the cavity space pyramid pooling module of DeepMDSCBA, a Convolutional Block Attention module is added to filter background information in order to reduce the loss of the edge detail information of apples in images, improve the accuracy of feature extraction, and effectively reduce the loss of feature details and deep information. This paper also explored the effects of rot degree, rot position, apple variety, and background complexity on the semantic segmentation performance of apple images, and then it verified the robustness of the method. The experimental results showed that the PA of this model could reach 95.3% and the MIoU could reach 87.1%, which were improved by 3.4% and 3.1% compared with DeepLabV3+, respectively, and superior to those of other semantic segmentation networks such as UNet and PSPNet. In addition, the DeepMDSCBA model proposed in this paper was shown to have a better performance than the other considered methods under different factors such as the degree or position of rotten parts, apple varieties, and complex backgrounds.

JDE multi-object tracking algorithm integrating multi-level semantic enhancement

In order to solve the problem of target ID switching caused by target occlusion and insufficient ID information and location information extraction in JDE(joint detection and embedding) algorithm, an improved multi-target tracking algorithm based on JDE is proposed in this paper. Firstly, the SPA feature space pyramid attention module is used to expand the receptive field and obtain more abundant semantic information to improve the detection accuracy of the model for different scale targets. Secondly, the FCN network makes the header and ID Embedding task collaborative learning to alleviate the excessive competition and enhance the original semantic information, effectively reducing the number of ID switching. Finally, PCCs-Ma motion measurement can strengthen the connection between Kalman filtering prediction and observation, and improve the reliability of similarity discrimination of motion characteristics. In order to verify the effectiveness of the algorithm, the JDE algorithm and the proposed algorithm are compared in the same experimental environment. The experimental results show that the average accuracy of model detection is improved by 3.94 %. On the MOT16 dataset, the MOTA and IDF1 indexes are increased by 6.9 %, and the number of ID switching of the improved algorithm is significantly reduced, and good tracking results are achieved.

Building extraction algorithm from remote sensing images based on improved DeepLabv3+ network

With the development of deep learning, quickly extracting high-precision building information from remote sensing images has become the research focus of intelligent application and processing of remote sensing data. Aiming at the problems of slow extraction speed and incomplete edge segmentation in building extraction in remote sensing images, a building extraction algorithm of remote sensing images based on an improved deeplabv3+ network is proposed. The more lightweight network MobileNetv3 is used to replace the original deeplabv3+ semantic segmentation model feature extraction backbone network Xception, and the standard convolution in the hole space pyramid pooling module is replaced with deep separable convolution, which reduces the amount of calculation and improves the training speed.DAMM (Dual Attention Mechanism Module) is connected in parallel with ASPP (Atous Spatial Pyramid Pooling) to improve the segmentation accuracy of edge targets. The model is verified on WHU and Massachusetts data sets. The results show that the number of training parameters and training time of the model are reduced, and the accuracy of the building extraction is effectively improved, which can meet the requirements of rapid extraction of high-precision buildings.

MODELLING AND DYNAMIC ANALYSIS OF THREE-DIRECTION GRID PRESTRESSED RETICULATED MEGA-STRUCTURE


 
 
 In this paper, the generation way of three-direction grid prestressed reticulated mega- structure is studied, and the three-direction grid prestressed reticulated mega-structure composite joint specific formation ideas are proposed. The important control parameters of three-direction grid prestressed reticulated mega-structure are determined, and the geometric modelling method of hexagon across the middle joint and octagon across the side joint is proposed. Coordinate formula of hexagonal and octagonal joints are derived to ensure reasonable grid arrangement and uniform stress distribution of the structure. Combined with the geometric parameters of the mega grid, a new method for creating the geometric model of three-direction grid prestressed mega reticulated structure is realized. The quadrangular pyramid space grid structure with the same length, span, rise and grid size and the three-direction grid prestressed grid structure are calculated, and the advantages of three-direction grid prestressed grid structure in dynamic performance are compared and analyzed.
 
 

Multilevel feature fusion dilated convolutional network for semantic segmentation

Recently, convolutional neural network (CNN) has led to significant improvement in the field of computer vision, especially the improvement of the accuracy and speed of semantic segmentation tasks, which greatly improved robot scene perception. In this article, we propose a multilevel feature fusion dilated convolution network (Refine-DeepLab). By improving the space pyramid pooling structure, we propose a multiscale hybrid dilated convolution module, which captures the rich context information and effectively alleviates the contradiction between the receptive field size and the dilated convolution operation. At the same time, the high-level semantic information and low-level semantic information obtained through multi-level and multi-scale feature extraction can effectively improve the capture of global information and improve the performance of large-scale target segmentation. The encoder–decoder gradually recovers spatial information while capturing high-level semantic information, resulting in sharper object boundaries. Extensive experiments verify the effectiveness of our proposed Refine-DeepLab model, evaluate our approaches thoroughly on the PASCAL VOC 2012 data set without MS COCO data set pretraining, and achieve a state-of-art result of 81.73% mean interaction-over-union in the validate set.