Features Of Different Scales Research Articles

GA-Net: Ghost convolution adaptive fusion skin lesion segmentation network

Automatic segmentation of skin lesions is a pivotal task in computer-aided diagnosis, playing a crucial role in the early detection and treatment of skin cancer. Despite the existence of numerous deep learning-based segmentation methods, the extraction of lesion features remains inadequate as a result of the segmentation process. Consequently, skin lesion image segmentation continues to face challenges regarding missing detailed information and inaccurate segmentation of the lesion region. In this paper, we propose a ghost convolution adaptive fusion network for skin lesion segmentation. First, the neural network incorporates a ghost module instead of the ordinary convolution layer, generating a rich skin lesion feature map for comprehensive target feature extraction. Subsequently, the network employs an adaptive fusion module and bilateral attention module to connect the encoding and decoding layers, facilitating the integration of shallow and deep network information. Moreover, multi-level output patterns are used for pixel prediction. Layer feature fusion effectively combines output features of different scales, thus improving image segmentation accuracy. The proposed network was extensively evaluated on three publicly available datasets: ISIC2016, ISIC2017, and ISIC2018. The experimental results demonstrated accuracies of 96.42%, 94.07%, and 95.03%, and kappa coefficients of 90.41%, 81.08%, and 86.96%, respectively. The overall performance of our network surpassed that of existing networks. Simulation experiments further revealed that the ghost convolution adaptive fusion network exhibited superior segmentation results for skin lesion images, offering new possibilities for the diagnosis of skin diseases.

Improved YOLOv4-tiny Target Detection Method Based on Adaptive Self-Order Piecewise Enhancement and Multiscale Feature Optimization

To improve the accuracy of material identification under low contrast conditions, this paper proposes an improved YOLOv4-tiny target detection method based on an adaptive self-order piecewise enhancement and multiscale feature optimization. The model first constructs an adaptive self-rank piecewise enhancement algorithm to enhance low-contrast images and then considers the fast detection ability of the YOLOv4-tiny network. To make the detection network have a higher accuracy, this paper adds an SE channel attention mechanism and an SPP module to this lightweight backbone network to increase the receptive field of the model and enrich the expression ability of the feature map. The network can pay more attention to salient information, suppress edge information, and effectively improve the training accuracy of the model. At the same time, to better fuse the features of different scales, the FPN multiscale feature fusion structure is redesigned to strengthen the fusion of semantic information at all levels of the network, enhance the ability of network feature extraction, and improve the overall detection accuracy of the model. The experimental results show that compared with the mainstream network framework, the improved YOLOv4-tiny network in this paper effectively improves the running speed and target detection accuracy of the model, and its mAP index reaches 98.85%, achieving better detection results.

FBC-ANet: A Semantic Segmentation Model for UAV Forest Fire Images Combining Boundary Enhancement and Context Awareness

Forest fires are one of the most serious natural disasters that threaten forest resources. The early and accurate identification of forest fires is crucial for reducing losses. Compared with satellites and sensors, unmanned aerial vehicles (UAVs) are widely used in forest fire monitoring tasks due to their flexibility and wide coverage. The key to fire monitoring is to accurately segment the area where the fire is located in the image. However, for early forest fire monitoring, fires captured remotely by UAVs have the characteristics of a small area, irregular contour, and susceptibility to forest cover, making the accurate segmentation of fire areas from images a challenge. This article proposes an FBC-ANet network architecture that integrates boundary enhancement modules and context-aware modules into a lightweight encoder–decoder network. FBC-Anet can extract deep semantic features from images and enhance shallow edge features, thereby achieving an effective segmentation of forest fire areas in the image. The FBC-ANet model uses an Xception network as the backbone of an encoder to extract features of different scales from images. By transforming the extracted deep semantic features through the CIA module, the model’s feature learning ability for fire pixels is enhanced, making feature extraction more robust. FBC-ANet integrates the decoder into the BEM module to enhance the extraction of shallow edge features in images. The experimental results indicate that the FBC-ANet model has a better segmentation performance for small target forest fires compared to the baseline model. The segmentation accuracy on the dataset FLAME is 92.19%, the F1 score is 90.76%, and the IoU reaches 83.08%. This indicates that the FBC-ANet model can indeed extract more valuable features related to fire in the image, thereby better segmenting the fire area from the image.

Gesture image recognition method based on DC-Res2Net and a feature fusion attention module

To extract decisive features from gesture images and solve the problem of information redundancy in the existing gesture recognition methods, we propose a new multi-scale feature extraction module named densely connected Res2Net (DC-Res2Net) and design a feature fusion attention module (FFA). Firstly, based on the new dimension residual network (Res2Net), the DC-Res2Net uses channel grouping to extract fine-grained multi-scale features, and dense connection has been adopted to extract stronger features of different scales. Then, we apply a selective kernel network (SK-Net) to enhance the representation of effective features. Afterwards, the FFA has been designed to remove redundant information in features by fusing low-level location features with high-level semantic features. Finally, experiments have been conducted to validate our method on the OUHANDS, ASL, and NUS-II datasets. The results demonstrate the superiority of DC-Res2Net and FFA, which can extract more decisive features and remove redundant information while ensuring high recognition accuracy and low computational complexity.

Few-Shot 3D Point Cloud Semantic Segmentation via Stratified Class-Specific Attention Based Transformer Network

3D point cloud semantic segmentation aims to group all points into different semantic categories, which benefits important applications such as point cloud scene reconstruction and understanding. Existing supervised point cloud semantic segmentation methods usually require large-scale annotated point clouds for training and cannot handle new categories. While a few-shot learning method was proposed recently to address these two problems, it suffers from high computational complexity caused by graph construction and inability to learn fine-grained relationships among points due to the use of pooling operations. In this paper, we further address these problems by developing a new multi-layer transformer network for few-shot point cloud semantic segmentation. In the proposed network, the query point cloud features are aggregated based on the class-specific support features in different scales. Without using pooling operations, our method makes full use of all pixel-level features from the support samples. By better leveraging the support features for few-shot learning, the proposed method achieves the new state-of-the-art performance, with 15% less inference time, over existing few-shot 3D point cloud segmentation models on the S3DIS dataset and the ScanNet dataset. Our code is available at https://github.com/czzhang179/SCAT.

CenterNet-Auto: A Multi-object Visual Detection Algorithm for Autonomous Driving Scenes Based on Improved CenterNet

With the rise in popularity of autonomous driving, the speed and accuracy of surrounding objects' detection by in-vehicle sensing technology is becoming increasingly important for autonomous vehicles. Building on CenterNet, this paper proposes CenterNet-Auto, a new anchor-free detection network for driving scenes that can satisfy the detection speed requirements while ensuring detection accuracy. The network's backbone uses the RepVGG model transformed through structural re-parameterization technology. Features of different scales are fused, and feature pyramids and deformable convolution are added after the backbone to accurately detect objects of different sizes. To solve the occlusion problem in the driving scene, this paper proposes the Average Border Model, which supports locating the object using the boundary feature information. The test results demonstrate that the proposed algorithm outperforms CenterNet regarding speed and accuracy on the BDD dataset. The accuracy reaches 55.6%, and the speed reaches 30 FPS, meeting the speed and accuracy requirements in a driving scene.

Reduced-reference image deblurring quality assessment based on multi-scale feature enhancement and aggregation

Image deblurring is a basic task in the field of computer vision, and has attracted much attention because of its application prospects in traffic monitoring and medical imaging, etc. Due to the inherent weakness of the model, it is difficult to obtain well-pleasing deblurred images for all the visual contents so far. Therefore, how to objectively evaluate the quality of these deblurred results is very important for the rapid development of image deblurring. In recent years, numerous convolutional neural networks based quality assessment methods have been proposed to automatically predict the quality of synthetic and authentic distorted images, producing results that are mildly consistent with subjective perception. However, they are limited in Image Deblurring Quality Assessment (IDQA). For IDQA, it is more meaningful to predict the quality difference of blurry-deblurred image (BDI) pair than to make prediction on single deblurred image. Inspired by this, we propose a novel reduced-reference image deblurring quality assessment method based on multi-scale feature enhancement and aggregation. Firstly, the multi-scale features of BDI pair are generated from a versatile vision Transformer. Secondly, the discrepancy information is exploited to implicitly enhance the initial deep features. Finally, the enhanced features of different scales are aggregated and then mapped to the quality difference of BDI pair. Experimental results on four challenging datasets demonstrate that the proposed method is superior to the state-of-the-art quality assessment methods.

CUCN: continuously updated connection network for low-light image enhancement

Low-light (LL) images make subsequent computer vision tasks difficult due to its low contrast. To solve this problem, the LL image enhancement problem is regarded as narrowing the gap between LL images and normal-light images in the process of iterative learning. A continuously updated connection network (CUCN) that is inspired by the recent development of convolutional neural networks is proposed. The proposed CUCN is composed of a continuously updated 4-units module (CU4UM), a feature fusion module (FFM), and a color enhancement module (CEM). The CU4UM adopts a connection method to combine four U-shaped structures with continuously updated parameters. To retain the global and local details while brightening an image, the FFM effectively combines the features of different layers in different U-shaped structures and assigns appropriate weights to each channel. The CEM takes the residuals of multiple features of different scales and performs iterative weighting to obtain more natural color information. The proposed method is evaluated on different public datasets, and the results show that the proposed CUCN method is superior to other state-of-the-art methods in terms of both subjective and objective metrics.

Multi-scale graph reasoning network for remote sensing image change detection

ABSTRACT The technology of change detection from a pair of registered satellite remote sensing images is of great importance for the study of landform changes and thus has attracted much attention. Recently, convolutional neural networks have been favoured in many applications due to their excellent feature extraction performance. However, for high-resolution satellite remote sensing images, previous methods are difficult to obtain finer details of change detection. There are many complex scenery and objects in satellite remote sensing images, which leads to the decline of the performance of the network model. In order to better solve the above problems, we propose a graph convolution-based multi-scale network with a channel attention mechanism. Specifically, we first project the initial features at different levels as vertex features, and there is a mutual relationship between the vertices. After that, we learn the correlations between these vertices through a graph reasoning. We then reproject the newly obtained vertex features to the original pixel space to obtain fine feature maps of the same size and fuse fine features of different scales through the attention fusion module. Finally, we realize the change detection of the input bi-temporal image through the feature difference module of the decoder. We separately conduct extensive experiments on three different high-resolution datasets to demonstrate that our method is effective.

LSA-Net: Location and shape attention network for automatic surface defect segmentation

Neural network algorithms for segmenting defects are widely used in industrial production. However, how to fuse the location information of defects in a single model and avoid the features extracted by different submodules gradually tend to be similar during the training is still a problem. To solve these problems, an end-to-end network is proposed that focuses on defect location and shape features, which can guarantee the difference between features extracted by different submodules. In the encoding stage, the location attention module enhances the perception of defect locations. The shape detection module with feature difference loss is designed to strengthen the detection of defect shapes. In the decoding stage, the features of different scales are fused to obtain the final defect region. The experimental results confirm the effectiveness of the proposed location and shape detection modules in the intersection over union on four datasets.

Optical Soft Tactile Sensor Algorithm Based on Multiscale ResNet

When grabbing an object, a robot needs to be installed with a sensor on its finger to sense the position of the object and the intensity of the contact force. Therefore, the accuracy of tactile sensor’s recognition is very important to grasp the object successfully. In our work, we design a data acquisition scheme and establish the corresponding deep learning dataset taking the optical-fiber-based tactile sensor as the research object. This kind of sensor is composed of different layers. Those layers are deformed when the force is applied on the surface. Since the extent of deformation is proportional to the value of the force, the color and brightness of the collected pictures will change according to Poisson effect. A novel multi-scale ResNet is proposed and compared with some mainstream networks such as AlexNet, ResNet, VGG, DenseNet, and GoogLeNet under the same dataset. The dataset we collected contains normal force, shear force, and torsion. It can provide better calibrations between the image change and force value. The proposed network is investigated experimentally. The results demonstrate that it can achieve enhanced performance by extracting image features of different scales. This multi-scale ResNet can be widely applied to the deep learning training of mechanics sensors based on the image and is also well-suited for the calibration tasks such as classification and regression prediction of mechanics sensors based on multiple data features.

IDD-Net: Industrial defect detection method based on Deep-Learning

Detecting defects in industrial products is one of the most widespread applications of industrial automation. Various product defects, large similarities, and drastic changes in scale in industrial scenarios pose challenges to existing industrial inspection networks. This paper proposes a deep learning-based industrial defect detection method (IDD-Net) to address the above challenges. Specifically, IDD-Net has three distinct features. First, for the defects of diversity and similarity (rolled-in_scale, crazing in steel defects), IDD-Net designed a novel local–global backbone feature network (LGB-Net). Second, IDD-Net proposes a novel Three-Layer Feature Aggregation network (TFLA-Net) to solve the problem of drastic scale changes. TFLA-Net adopts a novel three-layer descending method to aggregate semantic and fine-grained features effectively. At the same time, the dense connection of adjacent nodes of TFLA-Net ensures the efficient fusion of features of different scales in the network. In particular, this paper proposes a novel IoU loss (Defect-IoU loss) for the problem of object loss deviation at different scales. The novelty of Defect-IoU Loss is that the loss value is scaled by the difference in the area of different scale objects, which is more conducive to the balance of multi-scale object loss. The experimental results show that the calculation amount of IDD-Net is only 24.9 Gflops, and the mAP@.5 of 79.66%, 99.5%, and 95.9% in the steel defect, aluminium defect, and PCB defect datasets were respectively obtained, surpassing all comparison models. In addition, the test in the actual industrial scene also demonstrates the feasibility of the application of IDD-Net.

LSCB: a lightweight feature extraction block for SAR automatic target recognition and detection

ABSTRACT Automatic target recognition and detection in synthetic aperture radar (SAR) images is playing a significant role in military and civilian fields. Traditional methods are weak in migration capabilities and cumbersome in manual design. Existing deep learning-based methods can achieve high recognition and detection accuracy, but most of them have a very large number of parameters and slow recognition and computational efficiency, which makes it difficult for them to migrate to edge computing equipments, and computational efficiency is crucial in rapid maritime rescue and emergency military deployment. Therefore, we propose a lightweight feature extraction block, lightweight split concat block (LSCB), for SAR automatic target recognition and detection. Specifically, LSCB consists of several operators, such as split, depthwise convolution (DC), multiple kernel sizes block (MKSB), and channel-shuffle (CS). MKSB is a novel channel attention mechanism designed by us, which can make feature maps have target features of different scales and adjust their receptive field sizes adaptively. In addition, to assess the performance of LSCB, we use LSCB to build LSCNet-R and LSCNet-D for SAR target recognition and detection, respectively, and conduct extensive experiments on MSTAR, SSDD, and SAR-Ship-Dataset. The experimental results show that our LSCNet-R and LSCNet-D built with LSCB achieve higher recognition and detection accuracy and computational efficiency with few parameters compared with existing excellent methods. This further illustrates that the proposed LSCB can extract fine and stable features with few parameters.

Fully feature fusion based neural network for COVID-19 lesion segmentation in CT images

Coronavirus Disease 2019 (COVID-19) spreads around the world, seriously affecting people’s health. Computed tomography (CT) images contain rich semantic information as an auxiliary diagnosis method. However, the automatic segmentation of COVID-19 lesions in CT images faces several challenges, including inconsistency in size and shape of the lesion, the high variability of the lesion, and the low contrast of pixel values between the lesion and normal tissue surrounding the lesion. Therefore, this paper proposes a Fully Feature Fusion Based Neural Network for COVID-19 Lesion Segmentation in CT Images (F3-Net). F3-Net uses an encoder–decoder architecture. In F3-Net, the Multiple Scale Module (MSM) can sense features of different scales, and Dense Path Module (DPM) is used to eliminate the semantic gap between features. The Attention Fusion Module (AFM) is the attention module, which can better fuse the multiple features. Furthermore, we proposed an improved loss function LossCovid−BCE that pays more attention to the lesions based on the prior knowledge of the distribution of COVID-19 lesions in the lungs. Finally, we verified the superior performance of F3-Net on a COVID-19 segmentation dataset, experiments demonstrate that the proposed model can segment COVID-19 lesions more accurately in CT images than benchmarks of state of the art.

MSRA-Net: Tumor segmentation network based on Multi-scale Residual Attention

Automatic Medical segmentation of medical images is an important part in the field of computer medical diagnosis, among which tumor segmentation is an important branch of medical image segmentation. Accurate automatic segmentation method is very important in medical diagnosis and treatment. Positron emission computed tomography (PET) and X-ray computed tomography (CT) images are widely used in medical image segmentation to help doctors accurately locate information such as tumor location and shape, providing metabolic and anatomical information, respectively. At present, PET/CT images have not been effectively combined in the research of medical image segmentation, and the complementary semantic information between the superficial and deep layers of neural network has not been ensured. To solve the above problems, this paper proposed a Multi-scale Residual Attention network (MSRA-Net) for tumor segmentation of PET/CT. We first use an attention-fusion based approach to automatically learn the tumor-related areas of PET images and weaken the irrelevant area. Then, the segmentation results of PET branch are processed to optimize the segmentation results of CT branch by using attention mechanism. The proposed neural network (MSRA-Net) can effectively fuse PET image and CT image, which can improve the precision of tumor segmentation by using complementary information of the multi-modal image, and reduce the uncertainty of single modal image segmentation. Proposed model uses multi-scale attention mechanism and residual module, which fuse multi-scale features to form complementary features of different scales. We compare with state-of-the-art medical image segmentation methods. The experiment showed that the Dice coefficient of the proposed network in soft tissue sarcoma and lymphoma datasets increased by 8.5% and 6.1% respectively compared with UNet, showing a significant improvement.

MOP-SLAM: A real time SLAM system based on multi-head optical flow estimation network

SLAM (simultaneous localization and mapping) is becoming a significant technology in driverless vehicles nowadays. Sometimes the optical flow method can be used to track feature points to reduce the amount of calculation. But the traditional optical flow method is based on the assumption that the luminosity will not change and has high requirements for the environment. Therefore, an optical flow calculation method based on neural network called Optical-Net is proposed in this paper to match and track feature and corner points. We use the multi-head attention module to extract and fuse the features of different scales. The training results show that the Optical-Net designed in this study effectively improves the robustness and accuracy of optical flow estimation when the object has large displacement and small displacement. Meanwhile, to address the problem of reduced matching accuracy during video input due to dynamic blurring, a video dynamic deblurring algorithm based on the one-dimensional Wineman filter is adopted to preprocess the input video. Finally, we replaced the tracking thread in traditional SLAM. The performance of the improved algorithm is verified by experiments on our own driverless car.

MCBAM-GAN: The Gan Spatiotemporal Fusion Model Based on Multiscale and CBAM for Remote Sensing Images

Due to the limitations of current technology and budget, as well as the influence of various factors, obtaining remote sensing images with high-temporal and high-spatial (HTHS) resolution simultaneously is a major challenge. In this paper, we propose the GAN spatiotemporal fusion model Based on multiscale and convolutional block attention module (CBAM) for remote sensing images (MCBAM-GAN) to produce high-quality HTHS fusion images. The model is divided into three stages: multi-level feature extraction, multi-feature fusion, and multi-scale reconstruction. First of all, we use the U-NET structure in the generator to deal with the significant differences in image resolution while avoiding the reduction in resolution due to the limitation of GPU memory. Second, a flexible CBAM module is added to adaptively re-scale the spatial and channel features without increasing the computational cost, to enhance the salient areas and extract more detailed features. Considering that features of different scales play an essential role in the fusion, the idea of multiscale is added to extract features of different scales in different scenes and finally use them in the multi loss reconstruction stage. Finally, to check the validity of MCBAM-GAN model, we test it on LGC and CIA datasets and compare it with the classical algorithm for spatiotemporal fusion. The results show that the model performs well in this paper.

A Fine-Grained Recognition Neural Network with High-Order Feature Maps via Graph-Based Embedding for Natural Bird Diversity Conservation.

The conservation of avian diversity plays a critical role in maintaining ecological balance and ecosystem function, as well as having a profound impact on human survival and livelihood. With species' continuous and rapid decline, information and intelligent technology have provided innovative knowledge about how functional biological diversity interacts with environmental changes. Especially in complex natural scenes, identifying bird species with a real-time and accurate pattern is vital to protect the ecological environment and maintain biodiversity changes. Aiming at the fine-grained problem in bird image recognition, this paper proposes a fine-grained detection neural network based on optimizing the YOLOV5 structure via a graph pyramid attention convolution operation. Firstly, the Cross Stage Partial (CSP) structure is introduced to a brand-new backbone classification network (GPA-Net) for significantly reducing the whole model's parameters. Then, the graph pyramid structure is applied to learn the bird image features of different scales, which enhances the fine-grained learning ability and embeds high-order features to reduce parameters. Thirdly, YOLOV5 with the soft non-maximum suppression (NMS) strategy is adopted to design the detector composition, improving the detection capability for small targets. Detailed experiments demonstrated that the proposed model achieves better or equivalent accuracy results, over-performing current advanced models in bird species identification, and is more stable and suitable for practical applications in biodiversity conservation.

A Depth-Wise Separable U-Net Architecture with Multiscale Filters to Detect Sinkholes

Numerous variants of the basic deep segmentation model—U-Net—have emerged in recent years, achieving reliable performance across different benchmarks. In this paper, we propose an improved version of U-Net with higher performance and reduced complexity. This improvement was achieved by introducing a sparsely connected depth-wise separable block with multiscale filters, enabling the network to capture features of different scales. The use of depth-wise separable convolution significantly reduces the number of trainable parameters, making the training faster, while reducing the risk of overfitting. We used our developed sinkhole dataset and the available benchmark nuclei dataset to assess the proposed model’s performance. Pixel-wise annotation is laborious and requires a great deal of human expertise; therefore, we propose a fully deep convolutional autoencoder network that utilizes the proposed block to automatically annotate the sinkhole dataset. Our segmentation model outperformed the state-of-the-art methods, including U-Net, Attention U-Net, Depth-Separable U-Net, and Inception U-Net, achieving an average improvement of 1.2% and 1.4%, respectively, on the sinkhole and the nuclei datasets, with 94% and 92% accuracy, as well as a reduced training time. It also achieved 83% and 80% intersection-over-union (IoU) on the two datasets, respectively, which is an 11.8% and 9.3% average improvement over the above-mentioned models.

Multi-scale Edge-guided Learning for 3D Reconstruction

Single-view three-dimensional (3D) object reconstruction has always been a long-term challenging task. Objects with complex topologies are hard to accurately reconstruct, which makes existing methods suffer from blurring of shape boundaries between multiple components in the object. Moreover, most of them cannot balance learning between global geometric structure information and local detail information. In this article, we propose a multi-scale edge-guided learning network (MEGLN) to utilize the global edge information guiding the network to better capture and recover local details. The goal is to exploit the multi-scale learning strategy to learn global edge information and local details, thus achieving robust 3D object reconstruction. We first design a multi-scale Gaussian difference block (MGDB) to extract global edge geometry features for input images of different scales and adopt the attention mechanism to aggregate the extracted global edge geometry features of different scales. Second, we design a multi-scale feature interaction block (MFIB) to learn local details, which utilizes the multi-scale feature interaction to capture the features of multiple objects or components at multiple scales. The MFIB can learn and capture better as much local detail information as possible under the guidance of global edge information. Finally, we dynamically fuse the predicted probabilities of the MGDB and MFIB to obtain the final predicted result, which makes our MEGLN able to recover 3D shapes with global complex topological structures and rich local details via the multi-scale learning strategy. Extensive qualitative and quantitative experimental results on the ShapeNet dataset demonstrate that our approach achieves competitive performance compared with state-of-the-art methods. Code is available at https://github.com/Ray-tju/MEGLN .