Deep Layer Aggregation Research Articles

Multi-scale Information Aggregation for Spoofing Detection

Synthesis artifacts that span scales from small to large are important cues for spoofing detection. However, few spoofing detection models leverage artifacts across different scales together. In this paper, we propose a spoofing detection system built on SincNet and Deep Layer Aggregation (DLA), which leverages speech representations at different levels to distinguish synthetic speech. DLA is totally convolutional with an iterative tree-like structure. The unique topology of DLA makes possible compounding of speech features from convolution layers at different depths, and therefore the local and the global speech representations can be incorporated simultaneously. Moreover, SincNet is employed as the frontend feature extractor to circumvent manual feature extraction and selection. SincNet can learn fine-grained features directly from the input speech waveform, thus making the proposed spoofing detection system end-to-end. The proposed system outperforms the baselines when tested on ASVspoof LA and DF datasets. Notably, our single model surpasses all competing systems in ASVspoof DF competition with an equal error rate (EER) of 13.99%, which demonstrates the importance of multi-scale information aggregation for synthetic speech detection.

Enhancing trabecular CT scans based on deep learning with multi-strategy fusion

Trabecular bone analysis plays a crucial role in understanding bone health and disease, with applications like osteoporosis diagnosis. This paper presents a comprehensive study on 3D trabecular computed tomography (CT) image restoration, addressing significant challenges in this domain. The research introduces a backbone model, Cascade-SwinUNETR, for single-view 3D CT image restoration. This model leverages deep layer aggregation with supervision and capabilities of Swin-Transformer to excel in feature extraction. Additionally, this study also brings DVSR3D, a dual-view restoration model, achieving good performance through deep feature fusion with attention mechanisms and Autoencoders. Furthermore, an Unsupervised Domain Adaptation (UDA) method is introduced, allowing models to adapt to input data distributions without additional labels, holding significant potential for real-world medical applications, and eliminating the need for invasive data collection procedures. The study also includes the curation of a new dual-view dataset for CT image restoration, addressing the scarcity of real human bone data in Micro-CT. Finally, the dual-view approach is validated through downstream medical bone microstructure measurements. Our contributions open several paths for trabecular bone analysis, promising improved clinical outcomes in bone health assessment and diagnosis.

Three-Dimensional Point Cloud Object Detection Based on Feature Fusion and Enhancement

With the continuous emergence and development of 3D sensors in recent years, it has become increasingly convenient to collect point cloud data for 3D object detection tasks, such as the field of autonomous driving. But when using these existing methods, there are two problems that cannot be ignored: (1) The bird’s eye view (BEV) is a widely used method in 3D objective detection; however, the BEV usually compresses dimensions by combined height, dimension, and channels, which makes the process of feature extraction in feature fusion more difficult. (2) Light detection and ranging (LiDAR) has a much larger effective scanning depth, which causes the sector to become sparse in deep space and the uneven distribution of point cloud data. This results in few features in the distribution of neighboring points around the key points of interest. The following is the solution proposed in this paper: (1) This paper proposes multi-scale feature fusion composed of feature maps at different levels made of Deep Layer Aggregation (DLA) and a feature fusion module for the BEV. (2) A point completion network is used to improve the prediction results by completing the feature points inside the candidate boxes in the second stage, thereby strengthening their position features. Supervised contrastive learning is applied to enhance the segmentation results, improving the discrimination capability between the foreground and background. Experiments show these new additions can achieve improvements of 2.7%, 2.4%, and 2.5%, respectively, on KITTI easy, moderate, and hard tasks. Further ablation experiments show that each addition has promising improvement over the baseline.

MSPA-DLA++: A Multi-Scale Phase Attention Deep Layer Aggregation for Lesion Detection in Multi-Phase CT Images.

Object detection using convolutional neural networks (CNNs) has achieved high performance and achieved state-of-the-art results with natural images. Compared to natural images, medical images present several challenges for lesion detection. First, the sizes of lesions vary tremendously, from several millimeters to several centimeters. Scale variations significantly affect lesion detection accuracy, especially for the detection of small lesions. Moreover, the effective extraction of temporal and spatial features from multi-phase CT images is also an important issue. In this paper, we propose a group-based deep layer aggregation method with multiphase attention for liver lesion detection in multi-phase CT images. The method, which is called MSPA-DLA++, is a backbone feature extraction network for anchor-free liver lesion detection in multi-phase CT images that addresses scale variations and extracts hidden features from such images. The effectiveness of the proposed method is demonstrated on public datasets (LiTS2017) and our private multiphase dataset. The results of the experiments show that MSPA-DLA++ can improve upon the performance of state-of-the-art networks by approximately 3.7%.

Local eye-net: An attention based deep learning architecture for localization of eyes

Development of human machine interface has become a necessity for modern day machines to catalyze more autonomy and more efficiency. Gaze driven human intervention is an effective and convenient option for creating an interface to alleviate human errors. Facial landmark detection is very crucial for designing a robust gaze detection system. Regression based methods capacitate good spatial localization of the landmarks corresponding to different parts of the faces. But there are still scope of improvements which have been addressed by incorporating attention. In this paper, we have proposed a deep coarse-to-fine architecture called LocalEyenet for localization of only the eye regions that can be trained end-to-end. The model architecture, build on stacked hourglass backbone, learns the self-attention in feature maps which aids in preserving global as well as local spatial dependencies in face image. We have incorporated deep layer aggregation in each hourglass to minimize the loss of attention over the depth of architecture. Our model shows good generalization ability in cross-dataset evaluation and in real-time localization of eyes.

A Benchmark for Maritime Object Detection with Centernet on an Improved Dataset, ABOships-PLUS

Object detection from waterborne imagery is an essential aspect in maritime traffic management, navigation safety and coastal security. Building efficient autonomous systems, which can take decisions in critical situations, requires an integration of complex object detectors in real time. Object detectors trained on generic datasets often give unsatisfactory results in complex scenarios like the maritime environment, since only a fraction of their images contain maritime vessels. Publicly available domain-specific datasets are scarce, and they are limited in the number of images and annotations. Compared to object detection in applications such as autonomous vehicles, maritime vessel detection is considerably reduced in computer vision research. This creates a deficit in exhaustive benchmarking studies for maritime detection datasets. To bridge this gap, we relabel the ABOships dataset and benchmark a state-of-the-art center-based detector, Centernet, on the newly relabeled dataset, ABOships-PLUS. We explore its performance under different feature extractors, and investigate the effect of object size and inter-class variation on detection accuracy. The reported benchmarking illustrates that the ABOships-PLUS dataset is adequate to use in supervised domain adaptation. The experimental results show that Centernet with DLA (Deep Layer Aggregation) as a feature extractor achieved the highest accuracy in detecting maritime objects overall (with mean average precision 74.4%).

Deep Layer Aggregation Architectures for Photorealistic Universal Style Transfer.

This paper introduces a deep learning approach to photorealistic universal style transfer that extends the PhotoNet network architecture by adding extra feature-aggregation modules. Given a pair of images representing the content and the reference of style, we augment the state-of-the-art solution mentioned above with deeper aggregation, to better fuse content and style information across the decoding layers. As opposed to the more flexible implementation of PhotoNet (i.e., PhotoNAS), which targets the minimization of inference time, our method aims to achieve better image reconstruction and a more pleasant stylization. We propose several deep layer aggregation architectures to be used as wrappers over PhotoNet, to enhance the stylization and quality of the output image.

An efficient activity recognition for homecare robots from multi-modal communication dataset

Human environments are designed and managed by humans for humans. Thus, adding robots to interact with humans and perform specific tasks appropriately is an essential topic in robotics research. In recent decades, object recognition, human skeletal, and face recognition frameworks have been implemented to support the tasks of robots. However, recognition of activities and interactions between humans and surrounding objects is an ongoing and more challenging problem. Therefore, this study proposed a graph neural network (GNN) approach to directly recognize human activity at home using vision and speech teaching data. Focus was given to the problem of classifying three activities, namely, eating, working, and reading, where these activities were conducted in the same environment. From the experiments, observations, and analyses, this proved to be quite a challenging problem to solve using only traditional convolutional neural networks (CNN) and video datasets. In the proposed method, an activity classification was learned from a 3D detected object corresponding to the human position. Next, human utterances were used to label the activity from the collected human and object 3D positions. The experiment, involving data collection and learning, was demonstrated by using human-robot communication. It was shown that the proposed method had the shortest training time of 100.346 seconds with 6000 positions from the dataset and was able to recognize the three activities more accurately than the deep layer aggregation (DLA) and X3D networks with video datasets.

Location of Fruits by Counting: A Point-to-Point Approach

The emergence of deep learning-based methods for harvesting and yield estimates, including object detection or image segmentation-based methods, has notably improved performance but has also resulted in large annotation workloads. Considering the difficulty of such annotation, a method for locating fruit is developed in this study using only center-point labeling information. To address point labeling, the weighted Hausdorff distance is chosen as the loss function of the corresponding network, while deep layer aggregation (DLA) is used to contend with the variability in the visible area of the fruit. The performance of our method in terms of both detection and position is not inferior to the method based on Mask-RCNN. Experiments on a public apple dataset are provided to further demonstrate the performance of the proposed method. Specifically, no more than two targets had positioning deviations exceeding five pixels within the field of view.

Multi-Site and Multi-Scale Unbalanced Ship Detection Based on CenterNet

Object detection plays an essential role in the computer vision domain, especially the machine learning-based approach, which has developed rapidly in the past decades. However, the development of convolutional neural networks in the marine field is relatively slow, such as in ship classification and tracking. In this paper, ship detection is considered as a central point classification and regression task but discards the non-maximum suppression operation. We first improved the deep layer aggregation network to enhance the feature extraction capability of tiny targets, then reduced the number of parameters through the lightweight convolution module, and finally employed a unique activation function to enhance the nonlinearity of the model. By doing this, the improved network not only suits unbalanced sample ratios in classifying, but is more robust in scenarios where both the number and resolution of samples are unstable. Experimental results demonstrate that the proposed approach obtains outstanding performance and especially suits tiny object detection compared with current advanced methods. Furthermore, in contrast to the original CenterNet framework, the mAP of the proposed approach increased by 5.6%.

Enhanced Through-the-Wall Radar Imaging Based on Deep Layer Aggregation

Enhanced Through-the-Wall Radar Imaging Based on Deep Layer Aggregation

Fast Multi-Shadow Tracking for Video-SAR Using Triplet Attention Mechanism

This article extends the shadow tracking for video-synthetic aperture radar (SAR) from a single-target framework to a multitarget framework, which is crucial for SAR ground moving targets’ identification. Inspired by FairMOT, the multitarget tracking framework for SAR shadow tracking is improved by using the triplet attention (TriAtt) mechanism and the lightweight multiscale network. By employing the ability to fuse spatial and feature dimensions of TriAtt and combining the lightweight network optimized by multiscale encoder–decoder and dilated convolution, a fast multiscale feature extraction module (FMsFEM) embedded with TriAtt is proposed for better tracking efficiency and performance. Experiments on the Sandiego video-SAR dataset validate that the TriAtt mechanism can improve the tracking performance of deep layer aggregation (DLA)-34, DLA-18, and FMsFEM significantly. FMsFEM with embedded TriAtt outperforms the state-of-the-art network (FairMOT with backbones of DLA-34 and DLA-18) with much faster frame rates. The average frame rates of FMsFEM and FMsFEM-TriAtt reach 60.32 and 56.13 fps for datasets with an image size of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1088\times 608$ </tex-math></inline-formula> , which are about three times higher than the frame rates of others.

Small-Sized Vehicle Detection in Remote Sensing Image Based on Keypoint Detection

The vehicle detection in remote sensing images is a challenging task due to the small size of the objects and interference of a complex background. Traditional methods require a large number of anchor boxes, and the intersection rate between these anchor boxes and an object’s real position boxes needs to be high enough. Moreover, the size and aspect ratio of each anchor box need to be designed manually. For small objects, more anchor boxes need to be set. To solve these problems, we regard the small object as a keypoint in the relevant background and propose an anchor-free vehicle detection network (AVD-kpNet) to robustly detect small-sized vehicles in remote sensing images. The AVD-kpNet framework fuses features across layers with a deep layer aggregation architecture, preserving the fine features of small objects. First, considering the correlation between the object and the surrounding background, a 2D Gaussian distribution strategy is adopted to describe the ground truth, instead of a hard label approach. Moreover, we redesign the corresponding focus loss function. Experimental results demonstrate that our method has a higher accuracy for the small-sized vehicle detection task in remote sensing images compared with several advanced methods.

DGANet: Dynamic Gradient Adjustment Anchor-Free Object Detection in Optical Remote Sensing Images

Remote sensing image object detection has been studied by many researchers in recent years using deep neural networks. However, optical remote sensing images contain many scenes with small and dense objects, resulting in a high rate of misrecognition. Firstly, in this work we selected a deep layer aggregation network with updated deformable convolution layers as the backbone to extract object features. The detection and classification of objects was based on the center-point network without non-maximum suppression. Secondly, the dynamic gradient adjustment embedded into the classification loss function was put forward to harmonize the quantity imbalance between easy and hard examples, as well as between positive and negative examples. Furthermore, the complete intersection over union (CIoU) loss function was selected as the objective function of bounding box regression, which achieves better convergence speed and accuracy. Finally, in order to validate the effectiveness and precision of the dynamic gradient adjustment network (DGANet), we conducted a series of experiments in remote sensing public datasets UCAS-AOD and LEVIR. The comparison experiments demonstrate that the DGANet achieves a more accurate detection result in optical remote sensing images.

Bolt damage identification based on orientation-aware center point estimation network

With the development of deep learning, object detection algorithms based on horizontal box are widely used in the field of damage identification. However, damages can be in any direction and position, and they are not necessarily horizontal or vertical. This article proposes a bolt damage identification network, namely, orientation-aware center point estimation network, which models a damage as a center point of its rotated bounding box. The proposed orientation-aware center point estimation network uses deep layer aggregation network to search center points and regress to all other damage properties, such as size and angle. A loss function is designed to improve the optimization efficiency of network. Orientation-aware center point estimation network is applied to bolt damage detection, and comparison with the well-known Faster Region-Convolutional Neural Network (a benchmark using horizontal bounding box) demonstrates the accuracy of the proposed method. Finally, videos were utilized to verify the capability of the proposed orientation-aware center point estimation network in real-time detection of bolt damages.

Accurate and fast mitotic detection using an anchor-free method based on full-scale connection with recurrent deep layer aggregation in 4D microscopy images

BackgroundTo effectively detect and investigate various cell-related diseases, it is essential to understand cell behaviour. The ability to detection mitotic cells is a fundamental step in diagnosing cell-related diseases. Convolutional neural networks (CNNs) have been successfully applied to object detection tasks, however, when applied to mitotic cell detection, most existing methods generate high false-positive rates due to the complex characteristics that differentiate normal cells from mitotic cells. Cell size and orientation variations in each stage make detecting mitotic cells difficult in 2D approaches. Therefore, effective extraction of the spatial and temporal features from mitotic data is an important and challenging task. The computational time required for detection is another major concern for mitotic detection in 4D microscopic images.ResultsIn this paper, we propose a backbone feature extraction network named full scale connected recurrent deep layer aggregation (RDLA++) for anchor-free mitotic detection. We utilize a 2.5D method that includes 3D spatial information extracted from several 2D images from neighbouring slices that form a multi-stream input.ConclusionsOur proposed technique addresses the scale variation problem and can efficiently extract spatial and temporal features from 4D microscopic images, resulting in improved detection accuracy and reduced computation time compared with those of other state-of-the-art methods.

Deep Layer Aggregation with Cross Attention for Lane Detection

As an important part of driverless vehicle research and high-precision map making, lane detection technology needs to be improved urgently. However, it is difficult for current networks to obtain long-distance semantic information, which leads to the confusion of categories in narrow lanes. This paper adds cross attention based on deep layer aggregation to obtain long-distance semantic information, and constructs time series filter to filter in time domain, the method is simple and robust. Our encoder is based on resnet-50. According to the experimental results, slightly changing resnet-50 can achieve better results. Our method is evaluated on Baidu ApolloSpace land segmentation dataset, increases 3.4% relative to DeeplabV3+, and cross attention with time series filter contribute more than 1% mIoU accuracy.