Small Size Objects Research Articles

Jersey number detection using synthetic data in a low-data regime.

Player identification is an essential and complex task in sports video analysis. Different strategies have been devised over the years and identification based on jersey numbers is one of the most common approaches given its versatility and relative simplicity. However, automatic detection of jersey numbers is challenging due to changing camera angles, low video resolution, small object size in wide-range shots, and transient changes in the player's posture and movement. In this paper, we present a novel approach for jersey number identification in a small, highly imbalanced dataset from the Seattle Seahawks practice videos. We generate novel synthetic datasets of different complexities to mitigate the data imbalance and scarcity in the samples. To show the effectiveness of our synthetic data generation, we use a multi-step strategy that enforces attention to a particular region of interest (player's torso), to identify jersey numbers. The solution first identifies and crops players in a frame using a person detection model, then utilizes a human pose estimation model to localize jersey numbers in the detected players, obviating the need for annotating bounding boxes for number detection. We experimented with two sets of Convolutional Neural Networks (CNNs) with different learning objectives: multi-class for two-digit number identification and multi-label for digit-wise detection to compare performance. Our experiments indicate that our novel synthetic data generation method improves the accuracy of various CNN models by 9% overall, and 18% on low frequency numbers.

Chiral topographic instability in shrinking spheres

Many biological structures exhibit intriguing morphological patterns adapted to environmental cues, which contribute to their important biological functions and also inspire material designs. Here, we report a chiral wrinkling topography in shrinking core–shell spheres, as observed in excessively dehydrated passion fruit and experimentally demonstrated in silicon core–shells under air extraction. Upon shrinkage deformation, the surface initially buckles into a buckyball pattern (periodic hexagons and pentagons) and then transforms into a chiral mode. The neighbouring chiral cellular patterns can further interact with each other, resulting in secondary symmetry breaking and the formation of two types of topological network. We develop a core–shell model and derive a universal scaling law to understand the underlying morphoelastic mechanism and to effectively describe and predict such chiral symmetry breaking far beyond the critical instability threshold. Moreover, we show experimentally that the chiral characteristic adapted to local perturbation can be harnessed to effectively and stably grasp small-sized objects of various shapes and made of different stiff and soft materials. Our results not only reveal chiral instability topographies, providing fundamental insights into the surface morphogenesis of the deformed core–shell spheres that are ubiquitous in the real world, but also demonstrate potential applications of adaptive grasping based on delicate chiral localization.

Improving small objects detection using transformer

General artificial intelligence counteracts the inductive bias of an algorithm and tunes the algorithm for out-of-distribution generalization. A conspicuous impact of the inductive bias is an unceasing trend in improving deep learning performance. Although a quintessential attention-based object detection technique, DETR, shows better accuracy than its predecessors, its accuracy deteriorates for detecting small-sized (in-perspective) objects. This study examines the inductive bias of DETR and proposes a normalized inductive bias for object detection using data fusion, SOF-DETR. A technique of lazy-fusion of features is introduced in SOF-DETR, which sustains deep contextual information of objects present in an image. The features from multiple subsequent deep layers are fused for object queries that learn long and short-distance spatial association in an image using the attention mechanism. Experimental results on the MS COCO and Udacity Self Driving Car datasets assert the effectiveness of the added normalized inductive bias and feature fusion techniques, showing increased COCO mAP scores on small-sized objects.

An Improved YOLOV5 Based on Triplet Attention and Prediction Head Optimization for Marine Organism Detection on Underwater Mobile Platforms

Machine vision-based automatic detection of marine organisms is a fundamental task for the effective analysis of production and habitat changes in marine ranches. However, challenges of underwater imaging, such as blurring, image degradation, scale variation of marine organisms, and background complexity, have limited the performance of image recognition. To overcome these issues, underwater object detection is implemented by an improved YOLOV5 with an attention mechanism and multiple-scale detection strategies for detecting four types of common marine organisms in the natural scene. An image enhancement module is employed to improve the image quality and extend the observation range. Subsequently, a triplet attention mechanism is introduced to the YOLOV5 model to improve the feature extraction ability. Moreover, the structure of the prediction head of YOLOV5 is optimized to capture small-sized objects. Ablation studies are conducted to analyze and validate the effective performance of each module. Moreover, performance evaluation results demonstrate that our proposed marine organism detection model is superior to the state-of-the-art models in both accuracy and speed. Furthermore, the proposed model is deployed on an embedded device and its processing time is less than 1 s. These results show that the proposed model has the potential for real-time observation by mobile platforms or undersea equipment.

Multi-Scale Features Fusion Convolutional Neural Networks for Rice Leaf Disease Identification

It is time-consuming and labor-intensive to detect rice diseases manually. The purpose of this research is to develop a convolutional neural networks (CNNs)-based system to automatically detect the diseased rice leaf infected with rice leaf blast, helminthosporium leaf blight, and bacterial leaf blight. The sizes of rice leaf spots vary with the severity of disease infection. A single model based CNN cannot effectively classify images, especially for images with objects of small size as well as multiple object scales, and complicated image background. In this research, a multiscale serial convolution neural network (MSSCNN) and a multiscale parallel convolution neural network (MSPCNN) are proposed to identify diseased rice leaf spots based on multi-modal fusion to extract different perception features and combine them to improve the performances obtained by using only one modality. Experimental results delineate that MSSCNN and MSPCNN can get better performance in identifying the diseased rice leaves. In MSPCNN, the features of tiny spots on diseased rice leaves can be completely preserved. MSPCNN can hence offer better performances than MSSCNN. Additionally, MSPCNN architecture is suitable for parallel computing environment.

SF-YOLOv5: A Lightweight Small Object Detection Algorithm Based on Improved Feature Fusion Mode.

In the research of computer vision, a very challenging problem is the detection of small objects. The existing detection algorithms often focus on detecting full-scale objects, without making proprietary optimization for detecting small-size objects. For small objects dense scenes, not only the accuracy is low, but also there is a certain waste of computing resources. An improved detection algorithm was proposed for small objects based on YOLOv5. By reasonably clipping the feature map output of the large object detection layer, the computing resources required by the model were significantly reduced and the model becomes more lightweight. An improved feature fusion method (PB-FPN) for small object detection based on PANet and BiFPN was proposed, which effectively increased the detection ability for small object of the algorithm. By introducing the spatial pyramid pooling (SPP) in the backbone network into the feature fusion network and connecting with the model prediction head, the performance of the algorithm was effectively enhanced. The experiments demonstrated that the improved algorithm has very good results in detection accuracy and real-time ability. Compared with the classical YOLOv5, the mAP@0.5 and mAP@0.5:0.95 of SF-YOLOv5 were increased by 1.6% and 0.8%, respectively, the number of parameters of the network were reduced by 68.2%, computational resources (FLOPs) were reduced by 12.7%, and the inferring time of the mode was reduced by 6.9%.

A Deep Learning Model for Small-size Defective Components Detection in Power Transmission Tower

Unmanned Aerial Vehicle (UAV) inspection has gradually replaced manual inspection of transmission tower, which produces many images. While it is laborious and time-consuming to manually analyze these images, there are also challenges in automatically detecting small-size defective components such as bolts in transmission tower images, due to problems including complex background, small size, and many similar objects of bolts. In this paper, by virtue of multi-scale features and context information, we propose a deep neural network named Camp-Net (Context Information and Multi-Scale Pyramid Network) to identify bolts defect in transmission tower images. First, multi-scale feature fusion combines deep features and shallow features in convolutional networks to detect small-size bolts. Second, context information fusion puts the information around bolts into the detection network to remove the disturbance of complex background and similar objects. An image dataset containing defective bolts and normal bolts is constructed for model training and testing. Experimental results show that bolts with loose pins and bolts without pins among fittings in transmission tower can be accurately identified with the proposed model. The Average Precision (AP) of defective bolts detection of this model can be 11.4% higher than that of the commonly used high performance model, Faster R-CNN.

Optimisation of Deep Learning Small-Object Detectors with Novel Explainable Verification.

In this paper, we present a novel methodology based on machine learning for identifying the most appropriate from a set of available state-of-the-art object detectors for a given application. Our particular interest is to develop a road map for identifying verifiably optimal selections, especially for challenging applications such as detecting small objects in a mixed-size object dataset. State-of-the-art object detection systems often find the localisation of small-size objects challenging since most are usually trained on large-size objects. These contain abundant information as they occupy a large number of pixels relative to the total image size. This fact is normally exploited by the model during training and inference processes. To dissect and understand this process, our approach systematically examines detectors’ performances using two very distinct deep convolutional networks. The first is the single-stage YOLO V3 and the second is the double-stage Faster R-CNN. Specifically, our proposed method explores and visually illustrates the impact of feature extraction layers, number of anchor boxes, data augmentation, etc., utilising ideas from the field of explainable Artificial Intelligence (XAI). Our results, for example, show that multi-head YOLO V3 detectors trained using augmented data produce better performance even with a fewer number of anchor boxes. Moreover, robustness regarding the detector’s ability to explain how a specific decision was reached is investigated using different explanation techniques. Finally, two new visualisation techniques are proposed, WS-Grad and Concat-Grad, for identifying explanation cues of different detectors. These are applied to specific object detection tasks to illustrate their reliability and transparency with respect to the decision process. It is shown that the proposed techniques can result in high resolution and comprehensive heatmaps of the image areas, significantly affecting detector decisions as compared to the state-of-the-art techniques tested.

Automated image quality assessment of mammography phantoms: a systematic review.

Computerized image analysis is a viable technique for evaluating image quality as a complement to human observers. To systematically review the image analysis software used in the assessment of 2D image quality using mammography phantoms. A systematic search of multiple databases was performed from inception to July 2020 for articles that incorporated computerized analysis of 2D images of physical mammography phantoms to determine image quality. A total of 26 studies were included, 12 were carried out using direct digital imaging and 14 using screen film mammography. The ACR phantom (model-156) was the most frequently evaluated phantom, possibly due to the lack of accepted standard software. In comparison to the inter-observer variations, the computerized image analysis was more consistent in scoring test objects. The template matching method was found to be one of the most reliable algorithms, especially for high-contrast test objects, while several algorithms found low-contrast test objects to be harder to distinguish due to the smaller contrast variations between test objects and their backgrounds. This was particularly true for small object sizes. Image analysis software was in agreement with human observers but demonstrated higher consistency and reproducibility of quality evaluation. Additionally, using computerized analysis, several quantitative metrics such as contrast-to-noise ratio (CNR) and the signal-to-noise ratio (SNR) could be used to complement the conventional scoring method. Implementing a computerized approach for monitoring image quality over time would be crucial to detect any deteriorating mammography system before clinical images are impacted.

SiamGauss: Siamese region proposal network with Gaussian head for visual object tracking

We propose SiamGauss, a Siamese region proposal network with a Gaussian head for single-target visual object tracking for aerial benchmarks. Visual tracking in aerial videos faces unique challenges due to the large field of view resulting in small size objects, similar looking objects (confusers) in close proximity, occlusions, and fast motion due to simultaneous object and camera motion. In Siamese tracking, a cross-correlation ration is performed in the embedding space to obtain a similarity map of the target within a search frame, which is then used to localize the target. The proposed Gaussian head helps suppress the activation produced in the similarity map on confusers present in the search frame during training while boosting the confidence on the target. This activation suppression improves the confuser awareness of our tracker. In addition, improving the activation on the target helps maintain tracking consistency in fast motion. Our proposed Gaussian head is only applied during training and introduces no additional computational overhead during inference while tracking. Thus, SiamGauss achieves fast runtime performance. We evaluate our method on multiple aerial benchmarks showing that SiamGauss performs favorably with state-of-the-art trackers while rating at a frame rate of 96 frames per second.

RAMC: A Rotation Adaptive Tracker with Motion Constraint for Satellite Video Single-Object Tracking

Single-object tracking (SOT) in satellite videos (SVs) is a promising and challenging task in the remote sensing community. In terms of the object itself and the tracking algorithm, the rotation of small-sized objects and tracking drift are common problems due to the nadir view coupled with a complex background. This article proposes a novel rotation adaptive tracker with motion constraint (RAMC) to explore how the hybridization of angle and motion information can be utilized to boost SV object tracking from two branches: rotation and translation. We decouple the rotation and translation motion patterns. The rotation phenomenon is decomposed into the translation solution to achieve adaptive rotation estimation in the rotation branch. In the translation branch, the appearance and motion information are synergized to enhance the object representations and address the tracking drift issue. Moreover, an internal shrinkage (IS) strategy is proposed to optimize the evaluation process of trackers. Extensive experiments on space-born SV datasets captured from the Jilin-1 satellite constellation and International Space Station (ISS) are conducted. The results demonstrate the superiority of the proposed method over other algorithms. With an area under the curve (AUC) of 0.785 and 0.946 in the success and precision plots, respectively, the proposed RAMC achieves optimal performance while running at real-time speed.

Enhanced Single Shot Small Object Detector for Aerial Imagery Using Super-Resolution, Feature Fusion and Deconvolution.

One common issue of object detection in aerial imagery is the small size of objects in proportion to the overall image size. This is mainly caused by high camera altitude and wide-angle lenses that are commonly used in drones aimed to maximize the coverage. State-of-the-art general purpose object detector tend to under-perform and struggle with small object detection due to loss of spatial features and weak feature representation of the small objects and sheer imbalance between objects and the background. This paper aims to address small object detection in aerial imagery by offering a Convolutional Neural Network (CNN) model that utilizes the Single Shot multi-box Detector (SSD) as the baseline network and extends its small object detection performance with feature enhancement modules including super-resolution, deconvolution and feature fusion. These modules are collectively aimed at improving the feature representation of small objects at the prediction layer. The performance of the proposed model is evaluated using three datasets including two aerial images datasets that mainly consist of small objects. The proposed model is compared with the state-of-the-art small object detectors. Experiment results demonstrate improvements in the mean Absolute Precision (mAP) and Recall values in comparison to the state-of-the-art small object detectors that investigated in this study.

HrreNet: semantic segmentation network for moderate and high-resolution satellite images

ABSTRACT Semantic segmentation is a commonly used intelligent interpretation method. However, semantic segmentation via deep learning usually faces several problems: resolution loss due to downsampling, difficulty in obtaining context information, and imprecise object boundaries. These problems lead to a poor performance when using satellite images instead of conventional images and limit the value of intelligent interpretation of satellite images in practical application. In this study, we construct a Qilian Ecological Resource Extraction Dataset (QERED) with multiple scene satellite images of Qilian after preprocessing via correction, fusion, mosaic, and annotation. We then propose a segmentation network, namely, High-resolution Resource Extracting Network (HrreNet), by using high-resolution feature representation, multi-scale context fusion, boundary refinement with relearning, and structural similarity loss. Experiments on QERED show that HrreNet greatly improves performance on small-size objects and slightly improves performance on larger-size objects. HrreNet achieves the best result of 73.36% mIoU on a moderate-resolution satellite dataset CCFD and 80.87% mIoU on a high-resolution satellite dataset Vaihingen.

Small Object Detection Network Based on Feature Information Enhancement.

Due to the small size and weak characteristics of small objects, the performance of existing object detection algorithms for small objects is not ideal. In this paper, we propose a small object detection network based on feature information enhancement to improve the detection effect of small objects. In our method, two key modules, information enhancement module and dense atrous convolution module, are proposed to enhance the expression and discrimination ability of feature information. The detection accuracy of this method on PASCAL VOC, MS COCO, and UCAS-AOD data sets is 81.3%, 34.8%, and 87.0%, respectively. In addition, the detection results of this paper in detecting small objects are slightly (0.2% and 0.1%) higher than the current advanced algorithms (YOLOv4 and DETR, respectively). Moreover, when these two modules are integrated into other algorithms, such as RFBNet, it can also bring considerable improvement.

Comparative analysis of validating parameters in the deep learning models for remotely sensed images

The recognition of object in remotely sensed images is a complex task. The immense research is running in the field of remote sensing due to the availability of high resolution satellite images. The detection of object is a challenging task due to the complex background and small object size in remotely sensed images. The object detection in remote sensing images has a vital role in the field of navigation, salvage, and military. The performance of traditional algorithms is very less due to the usage of handcrafted features. With the initiation of Deep Learning algorithms, various Convolutional Neural Networks (CNN) based model have been utilized to detect the objects with high-resolution remotely sensed images. In this research paper various CNN based models has been compared and analyzed. Object detection approaches are broadly categorized in two ways-one based on the region matching and second based on the one-stage target detection. The researchers have compared the result of R-CNN, SPP Net , fast R-CNN, faster R-CNN, R-FCN, Mask R-CNN SSD (Single Shot Multibox Detector), DSSD (Deconvolution Single Shot Multibox Detector), FSSD , YOLO v1,YOLO v2, YOLO v3, Gaussian YOLO v3, RetinaNet which conclude that the minimal average precision for the region based category is best shown by Mask R-CNN with 39.8 mAP in the COCO parameter test and for the one stage detector YOLO v3 shows the best case for the COCO parameter test with 69.1 mAP. In the second phase of the review the researchers found that in comparison to the region based and one stage detector the YOLO v3 model from one stage detector shows the best detection precision percentage with the highest 87% in identifying the object called ship.

Yolov4-tiny with wing convolution layer for detecting fish body part

Detection of a fish's eye, tail and body is the initial process in the vision system for determining the freshness and species of fish, as well as calculating the number of fish automatically in the fishing industry. Classification performance of a system is affected by its ability to detect the intact body of a fish or its body parts. The You Only Look Once version 4 tiny (Yolov4-tiny) is a lightweight object detector that can detect body parts of a fish with fairly good detection accuracy. However, massive siltation of convolution layer in the Yolov4-tiny backbone leads to low feature diversity. This research proposes a modification of the Yolov4-tiny architecture to improve detection accuracy by enhancing and balancing feature diversity and attaching an extra-branch detector to detect small-sized objects. In addition, we propose the use of bottleneck and expansion convolution to reduce computational resources usage. Our contributions are enhancing feature diversity using a wing convolution layer (WCL), balancing feature diversity using tiny spatial pyramid pooling (Tiny-SPP), reducing computational resources of feature pyramid network (FPN) connections using bottleneck and expansion convolution (BEC), and detecting small objects using an extra-branch as a third-scale detector. Our experimental results show that the proposed model outperforms the original model and other modified Yolov4-tiny models with Precision, Recall, AP, and mAP of 97.48%, 93.3%, 94.07%, and 92.38% respectively. The proposed model is smaller in size and more efficient in the use of computing resources.

Analysing Key Steps of the Photogrammetric Pipeline for Museum Artefacts 3D Digitisation

In recent years, massive digitisation of cultural heritage (CH) assets has become a focus of European programmes and initiatives. Among CH settings, attention is reserved to the immense and precious museum collections, whose digital 3D reproduction can support broader non-invasive analyses and stimulate the realisation of more attractive and interactive exhibitions. The reconstruction pipeline typically includes numerous processing steps when passive techniques are selected to deal with object digitisation. This article presents some insights on critical operations, which, based on our experience, can rule the quality of the final models and the reconstruction times for delivering 3D heritage results, while boosting the sustainability of digital cultural contents. The depth of field (DoF) problem is explored in the acquisition phase when surveying medium and small-sized objects. Techniques for deblurring images and masking object backgrounds are examined relative to the pre-processing stage. Some point cloud denoising and mesh simplification procedures are analysed in data post-processing. Hints on physically-based rendering (PBR) materials are also presented as closing operations of the reconstruction pipeline. This paper explores these processes mainly through experiments, providing a practical guide, tricks, and suggestions when tackling museum digitisation projects.

SFGNet detecting objects via spatial fine-grained feature and enhanced RPN with spatial context

Object detection, which is one of the most fundamental visual recognition tasks, has been a hotspot in computer vision. CNN (Convolutional Neural Networks) have been widely employed for building detector. Due to the success of RPN (Region Proposal Network), the two-stage detectors get both classification accuracy and precise regression bounding boxes. However, they still struggle in small-size object detection. In this paper, we present a deep network, namely Spatial Fine-Grained Network (SFGN). The SFGN that exploits Spatial Fine-Grained Features (SFGF) concatenates the higher resolution features, which is fine-grained with the low resolution features and high-level semantic by stacking spatial features for fine-grained features. An enhanced region proposal generator is proposed to get the objectless for small object to obtain a small set of proposal. The contextual information surrounding the region of interest is embedded using local spatial information for increasing the useful information and discriminating the background. For improving the detection performance, we use a simple yet surprisingly effective online hard example mining (OHEM) algorithm for training region proposal generator. It embeds an efficiently implemented soft non-maximum suppression (soft-NMS) for replacing with tradition NMS to obtain consistent improvements without increasing the computational complexity in inference. On PASCAL VOC 2007 and PASCAL VOC 2012 datasets, our SFGN improves baseline model from 81.2% mAP to 80.6% mAP. On MS COCO dataset, SFGN also performs better than baseline model. As intuition suggests, our detection results provide strong evidence that our SFGN improves detection accuracy, especially in small object test.

Dilated convolution based RCNN using feature fusion for Low-Altitude aerial objects

The low-altitude aerial objects are hard to detect by existing deep learning-based object detectors because of the scale variance, small size, and occlusion-related problems. Deep learning-based detectors do not consider contextual information about the scale information of small-sized objects in low-altitude aerial images. This paper proposes a new system using the concept of receptive fields and fusion of feature maps to improve the efficiency of deep object detectors in low-altitude aerial images. A Dilated ResNet Module (DRM) is proposed, motivated from the trident networks, which works on dilated convolutions to study the contextual data for specifically small-sized objects. Applicability of this component builds the model strong towards scale variations in low-altitude aerial objects. Then, Feature Fusion Module (FFM) is created to offer semantic intelligence for better detection of low-altitude aerial objects. We have chosen vastly deployed faster RCNN as the base detector for the proposal of our technique. The dilated convolution-based RCNN using feature fusion (DCRFF) system is implemented on a benchmark low-altitude UAV based-object detection dataset, VisDrone, which contains multiple object categories of pedestrians, vehicles in crowded scenes. The experiments exhibit the enactment of the given detector on chosen low-altitude aerial object dataset. The proposed system of DCRFF achieves 35.04% mAP on the challenging VisDrone dataset, indicating an average improvement of 2% when compared.

Underwater object detection algorithm based on channel attention and feature fusion

Due to the color deviation, low contrast and fuzzy object in underwater optical images, there are some problems in underwater object detection, such as missed detection and false detection. In order to solve the above-mentioned problems, an underwater object detection algorithm is proposed based on the channel attention and feature fusion for underwater optical images. The excitation residual module is designed based on the channel attention, and the forward propagation feature information is adaptively allocated weights to highlight the salience of different channel feature maps, which improves the network ability to extract high-frequency information from the underwater images. The multi-scale feature fusion module is designed to add a large scale feature map for object detection, which improves the detection performance of the network for small size objects by using its corresponding small size receptive field, and further improves the detection accuracy of the network for different size objects in the underwater environment. To improve the generalization performance of the network to the underwater environment, the data augmentation method based on the stitching and fusion is designed to simulate the overlap, occlusion and blurring of underwater objects, which improves the adaptability of the network to the underwater environment. Through experiments on the public dataset URPC, the algorithm in this paper improves the mean average precision by 5.42%, 3.20% and 0.9% compared with YOLOv3, YOLOv4 and YOLOv5, respectively, effectively improving the missed and false detection of objects of different sizes in complex underwater environments.