Small Size Objects Research Articles

MIYOLO: Modification of Improved YOLO-v3

In the world of detection and classification, You Only Look Once (YOLO)-v3 has been maintaining its position as the best classifier and detector and is due to its fastness, accuracy, less complex structure, continuous modifications, improvements, etc. Several approaches have been proposed in this paper to modify improved YOLO-v3. Our main contributions are (1) increasing the number of anchor boxes; (2) replacement of K-means clustering with density-based spatial clustering of applications with noise (DBSCAN); (3) introducing final bounding box alignment (FBBA) technique; and (4) replacement of multi-scaling with a depth calculation algorithm. At first, we increased the number of anchor boxes from 9 to 11 which resulted in a higher performance of 6% for both visual object classes and common objects in context datasets, where increment in computational cost was negligible. Similarly, it has been analyzed that a true number of anchor boxes can increase efficiency. So, instead of generating a static number of anchor boxes with k-means clustering, DBSCAN has been proposed for generating a dynamic and accurate number of anchor boxes. For minimizing the localization error, a new FBBA method has been introduced which would exactly overlap the object. In the same way, depth calculation algorithm can solve the small and large size object detection and classification problem without doing multi-scaling.

Impact of loss functions on semantic segmentation in far‐field monitoring

AbstractAlthough previous research laid the foundation for vision‐based monitoring systems using convolutional neural networks (CNNs), too little attention has been paid to the challenges associated with data imbalance and varying object sizes in far‐field monitoring. To fill the knowledge gap, this paper investigates various loss functions to design a customized loss function to address the challenges. Scaffold installation operations recorded by camcorders were selected as the subject of analysis in a far‐field surveillance setting. It was confirmed that the data imbalance between the workers, hardhats, harnesses, straps, and hooks caused poor performances especially for small size objects. This problem was mitigated by employing a region‐based loss and Focal loss terms in the loss function of segmentation models. The findings illustrate the importance of the loss function design in improving performance of CNN models for far‐field construction site monitoring.

HANDLING CRITICAL ASPECTS IN MASSIVE PHOTOGRAMMETRIC DIGITIZATION OF MUSEUM ASSETS

Abstract. In recent years, a growing interest in the 3D digitisation of museum assets has been pushed by the evident advantages of digital copies in supporting and advancing the knowledge, preservation and promotion of historical artefacts. Realising photo-realistic and precise digital twins of medium and small-sized movable objects implies several operations, still hiring open research problems and hampering the complete automation and derivation of satisfactory results while limiting processing time. The work examines some recurrent issues and potential solutions, summing up several experiences of photogrammetric-based massive digitisation projects. In particular, the article presents some insights into three crucial aspects of the photogrammetric pipeline. The first experiments tackle the Depth of Field (DoF) problem, especially when digitising small artefacts with macro-lenses. On the processing side, two decisive and time-consuming tasks are instead investigated: background masking and point cloud editing, exploring and proposing automatic solutions for speeding up the reconstruction process.

Impact of the Radar Image Resolution of Military Objects on the Accuracy of their Classification by a Deep Convolutional Neural Network

Introduction. Deep convolutional neural networks are considered as one of the most promising tools for classifying small-sized objects on radar images. However, no systemic study has been reported so far on the dependence between the classification accuracy achieved by convolutional neural networks and such an important image characteristic as resolution.Aim. Determination of a dependence between of the accuracy of classifying military objects by a deep convolutional neural network and the resolution of their radar images.Materials and methods. An eight-layer convolutional neural network was designed, trained and tested using the Keras library and Tensorflow 2.0 framework. For training and testing, the open part of the standard MSTAR dataset comprising ten classes of military objects radar images was used. The initial weight values of the MobileNetV1 and Xception networks used for a comparative assessment of the achieved classification accuracy were obtained from the training results on the Imagenet.Results. The accuracy of classifying military objects decreases rapidly along with a deterioration in resolution, amounting to 97.91, 90.22, 79.13, 52.2 and 23.68 % at a resolution of 0.3, 0.6, 0.9, 1.5 and 3 m, respectively. It is shown that the use of pretrained MobileNetV1 and Xception networks does not lead to an improvement in the classification accuracy compared to a simple VGG-type network.Conclusion. Effective recognition of military objects at a resolution worse than one meter is practically impossible. The classification accuracy of deep neural networks depends significantly on the difference in the image resolution of the training and test sets. A significant increase in the resistance of the classification accuracy to changes in the resolution can be achieved by training on a set of images with different resolutions.

High-pitch sounds small for domestic dogs: abstract crossmodal correspondences between auditory pitch and visual size.

Humans possess intuitive associations linking certain non-redundant features of stimuli—e.g. high-pitched sounds with small object size (or similarly, low-pitched sounds with large object size). This phenomenon, known as crossmodal correspondence, has been identified in humans across multiple different senses. There is some evidence that non-human animals also form crossmodal correspondences, but the known examples are mostly limited to the associations between the pitch of vocalizations and the size of callers. To investigate whether domestic dogs, like humans, show abstract pitch-size association, we first trained dogs to approach and touch an object after hearing a sound emanating from it. Subsequently, we repeated the task but presented dogs with two objects differing in size, only one of which was playing a sound. The sound was either high or low pitched, thereby creating trials that were either congruent (high pitch from small object; low pitch from large objects) or incongruent (the reverse). We found that dogs reacted faster on congruent versus incongruent trials. Moreover, their accuracy was at chance on incongruent trials, but significantly above chance for congruent trials. Our results suggest that non-human animals show abstract pitch sound correspondences, indicating these correspondences may not be uniquely human but rather a sensory processing feature shared by other species.

Lightweight and computationally faster Hypermetropic Convolutional Neural Network for small size object detection

Object detection has been an active area of research over the past two decades. The complexity of detecting an object increases with the increase in object speed and decrease in object size. Similar scenarios are observed in sports video analysis, vision systems of robots, driverless cars and much more. This led to the need for an efficient neural network that can detect small size objects. Further, most of the real-time applications use single board computers such as Jetson Nano, TX2, Xavier, Raspberry Pi and the like. The state-of-the-art of Deep Learning models such as YOLOv4, v3, YOLOR, YOLOX and SSD show poor run-time performance on these devices. Their lighter versions YOLOv3-tiny, YOLOv4-tiny and YOLOX-nano run nearly at 24 frames per second (fps) on Jetson Nano; however, their detection accuracy on small-sized objects is unsatisfactory. This paper focuses on developing a computationally lighter Convolutional Neural network(CNN) to detect small-sized objects efficiently. A novel hypermetropic CNN was developed to meet the above requirements. The improvement in detection is made by extracting more features from the shallow layers and transferring low-level features to the deeper layers. The network is hypermetropic because it performs well on distant objects and lags on nearby objects. The proposed model's performance is compared with the state-of-the-art models on various public datasets such as the VEDAI dataset, Visdrone dataset, and a few classes from the MS COCO and OID dataset. The proposed model shows impressive improvements in detecting small-size objects, and a 32% increase in the fps is observed on Jetson Nano.

The Utilisation of Native Woodland in Norse Greenland

ABSTRACT Wood was a key raw material for past societies and no less so in the Arctic region where woodland is sparse. When Norse settlers came to Greenland in the late tenth century AD, the pristine environment had not been affected by other humans for centuries. It has been assumed that the Norse had a negative effect on the environment, with drastic woodland clearance resulting in a more or less treeless landscape, contributing to the eventual demise of the Norse Greenlandic society. Recent palynological studies indicate that the environmental impact was in fact more complex. Wood taxonomic analyses on artefacts and samples from five Norse Greenlandic sites (1000–1400 AD) show that 36% of the combined assemblages (total of 8552 pieces) derive from native woodland. Mostly, it was used to make small-sized objects, but it was also an important source of nutrients, fuel, fodder and insulation. The proportion of native woodland was significantly higher on medium-sized farms than at the one high-status farm studied, indicating that socioeconomic factors impacted wood acquisition. Although local woodlands could not sustain all the timber needs of the Norse Greenlanders, it made up a substantial component of their wood procurement strategies.

3D object detection combining semantic and geometric features from point clouds

Background: 3D object detection based on point clouds in road scenes has attracted much attention recently. The voxel-based methods voxelize the scene to regular grids, which can be processed with the advanced feature learning frameworks based on convolutional layers for semantic feature learning. The point-based methods can extract the geometric feature of the point due to the coordinate reservations. The combination of the two is effective for 3D object detection. However, the current methods use a voxel-based detection head with anchors for classification and localization. Although the preset anchors cover the entire scene, it is not suitable for detection tasks with larger scenes and multiple categories of objects, due to the limitation of the voxel size. Additionally, the misalignment between the predicted confidence and proposals in the Regions of the Interest (ROI) selection bring obstacles to 3D object detection. Methods: We investigate the combination of voxel-based methods and point-based methods for 3D object detection. Additionally, a voxel-to-point module that captures semantic and geometric features is proposed in the paper. The voxel-to-point module is conducive to the detection of small-size objects and avoids the presets of anchors in the inference stage. Moreover, a confidence adjustment module with the center-boundary-aware confidence attention is proposed to solve the misalignment between the predicted confidence and proposals in the regions of the interest selection. Results: The proposed method has achieved state-of-the-art results for 3D object detection in the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) object detection dataset. Actually, as of September 19, 2021, our method ranked 1st in the 3D and Bird Eyes View (BEV) detection of cyclists tagged with difficulty level ‘easy’, and ranked 2nd in the 3D detection of cyclists tagged with ‘moderate’. Conclusions: We propose an end-to-end two-stage 3D object detector with voxel-to-point module and confidence adjustment module.

Towards densely clustered tiny pest detection in the wild environment

Our life is populated with many small-size objects, such as human in aerial images and tiny pests in agriculture. Current generic and small object detection methods are only focus on tackling their sizes rather than distribution. Considering this limitation, we state a Densely Clustered Tiny (DCT) object detection problem using a novel metric Object Density Level (ODL) to measure the object distribution in an image. The DCT problem allows varied densely distributed objects in the real-world captured images. In dealing with the DCT problem, we select two kinds of aphids that usually gather into cliques in the real-world agricultural environment, and build an aphid dataset APHID-4K in our task. Accompanying the DCT task, we propose a novel DCT detection network (DCTDet) to address this challenge. Specifically, a Cluster Region Proposal Network (ClusRPN) is trained to select appropriate densely distributed object cluster regions from images. These candidates are classified into different groups according to their density. A Density Merging and Partition module (DMP) merges and partitions them respectively and finally outputs cluster regions with uniform size and density to a subsequent Local Detector Group (LDG). In addition, we also use Composited Cluster data Generation (CCG) to present a large-scale dataset for ClusRPN optimization for robust training procedure and theoretically analyze their effects in detail. Experiments on APHID-4K and another clustered small object detection dataset VisDrone show that our DCTDet achieves state-of-the-art performance.

Deep Neural Network Based Vehicle Detection and Classification of Aerial Images

The detection of the objects in the ariel image has a significant impact on the field of parking space management, traffic management activities and surveillance systems. Traditional vehicle detection algorithms have some limitations as these algorithms are not working with the complex background and with the small size of object in bigger scenes. It is observed that researchers are facing numerous problems in vehicle detection and classification, i.e., complicated background, the vehicle’s modest size, other objects with similar visual appearances are not correctly addressed. A robust algorithm for vehicle detection and classification has been proposed to overcome the limitation of existing techniques in this research work. We propose an algorithm based on Convolutional Neural Network (CNN) to detect the vehicle and classify it into light and heavy vehicles. The performance of this approach was evaluated using a variety of benchmark datasets, including VEDAI, VIVID, UC Merced Land Use, and the Self database. To validate the results, various performance parameters such as accuracy, precision, recall, error, and F1-Score were calculated. The results suggest that the proposed technique has a higher detection rate, which is approximately 92.06% on the VEDAI dataset, 95.73% on the VIVID dataset, 90.17% on the UC Merced Land dataset, and 96.16% on the Self dataset.

Алгоритм распознавания малоразмерных воздушных объектов на основе анализа радиолокационных спектральных портретов

During the current time the permanent growth in the use of various unmanned aerial vehicles generates a whole set of tasks to be solved for the detection of flying aerial objects. One of such tasks can be described as a recognition of objects moving in the airspace, which includes unmanned vehicles and ornithological objects usually observed by a radar. The purpose of this paper is to develop an algorithm capable of recognition of the small-sized aerial objects monitored by radar with a probability no less than 85%. Trajectory and signal components specific to the considered classes of small-sized aerial objects should act as features for recognition. The effectiveness of the application of trajectory features is not sufficient enough for recognition due to the similarity of the movement and maneuver of the considered objects. Signal features representations are spectral images received by radar. To obtain spectral images various transformations of the signals are conducted. The algorithm incapsulates the transformation of the received signal, obtaining a spectral image and automatic object recognition using artificial intelligence. The results of this algorithm make it possible to develop specialized software for the aerial object recognition for future radar systems for surveillance of airports and important infrastructure.

Small object detection in remote sensing images based on super-resolution

Accurate objects detection in remote sensing images is very important, because security, transportation, and rescue applications in military and civilian fields require fully analyzing and using these images. To address the problem that many small-sized objects in remote sensing images are difficult to detect, this paper proposes an improved S2ANET-SR model based on S2A-NET network. In this paper, the original and reduced image are fed to the detection network at the same time, and then a super-resolution enhancement module for the reduced image is designed to enhance the feature extraction of small objects, after that, the perceptual loss and texture matching loss is proposed as supervision. Extensional experiments are conducted to evaluate the performance on the general remote sensing dataset DOTA, and the results show that our proposed method can achieve 74.47% mAP, which is 0.79% better than the accuracy of S2A-NET.

A Survey of Deep Learning-Based Object Detection Methods and Datasets for Overhead Imagery

Significant advancements and progress made in recent computer vision research enable more effective processing of various objects in high-resolution overhead imagery obtained by various sources from drones, airplanes, and satellites. In particular, overhead images combined with computer vision allow many real-world uses for economic, commercial, and humanitarian purposes, including assessing economic impact from access crop yields, financial supply chain prediction for company’s revenue management, and rapid disaster surveillance system (wildfire alarms, rising sea levels, weather forecast). Likewise, object detection in overhead images provides insight for use in many real-world applications yet is still challenging because of substantial image volumes, inconsistent image resolution, small-sized objects, highly complex backgrounds, and nonuniform object classes. Although extensive studies in deep learning-based object detection have achieved remarkable performance and success, they are still ineffective yielding a low detection performance, due to the underlying difficulties in overhead images. Thus, high-performing object detection in overhead images is an active research field to overcome such difficulties. This survey paper provides a comprehensive overview and comparative reviews on the most up-to-date deep learning-based object detection in overhead images. Especially, our work can shed light on capturing the most recent advancements of object detection methods in overhead images and the introduction of overhead datasets that have not been comprehensively surveyed before.

CT-Net: An Efficient Network for Low-Altitude Object Detection Based on Convolution and Transformer

With the growing popularity of civilian unmanned aerial vehicles (UAVs), unauthorized flights are on the rise accordingly. Therefore, it is critical to detect low-altitude UAVs for protecting personal privacy and public safety. Though substantial progress has been made in UAV detection, the existing detection methods still have problems in balancing the detection accuracy, the model size, and the detection speed. To address these limitations, this article proposes a novel deep learning method named convolution–transformer network (CT-Net). First, the attention-enhanced transformer block (AETB), which builds a feature-enhanced multihead self-attention (FEMSA), is introduced into the backbone of the network to improve the feature extraction ability of the model. Then, a lightweight bottleneck module (LBM) is utilized to control the computation load and reduce the parameters. Finally, we present a directional feature fusion structure (DFFS) to improve the accuracy of detection when processing multiscale objects, especially small-size objects. The proposed scheme achieves 0.966 mAP with the input size of 640 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times640$ </tex-math></inline-formula> pixels on our low-altitude small-object dataset, and the performance is superior to YOLOv5. Furthermore, the experimental result on MS COCO shows that the CT-Net can outperform the current state-of-the-art detectors by a large margin. Thus, it is feasible to apply CT-Net to low-altitude small-object detection according to the experimental results.

Human Pose Estimation and Object Interaction for Sports Behaviour

In the new era of technology, daily human activities are becoming more challenging in terms of monitoring complex scenes and backgrounds. To understand the scenes and activities from human life logs, human-object interaction (HOI) is important in terms of visual relationship detection and human pose estimation. Activities understanding and interaction recognition between human and object along with the pose estimation and interaction modeling have been explained. Some existing algorithms and feature extraction procedures are complicated including accurate detection of rare human postures, occluded regions, and unsatisfactory detection of objects, especially small-sized objects. The existing HOI detection techniques are instance-centric (object-based) where interaction is predicted between all the pairs. Such estimation depends on appearance features and spatial information. Therefore, we propose a novel approach to demonstrate that the appearance features alone are not sufficient to predict the HOI. Furthermore, we detect the human body parts by using the Gaussian Matric Model (GMM) followed by object detection using YOLO. We predict the interaction points which directly classify the interaction and pair them with densely predicted HOI vectors by using the interaction algorithm. The interactions are linked with the human and object to predict the actions. The experiments have been performed on two benchmark HOI datasets demonstrating the proposed approach.

Deep Siamese Network With Motion Fitting for Object Tracking in Satellite Videos

With the advancement in remote sensing satellite technology, object tracking in satellite videos has become an emerging research field. However, due to small object size, little appearance features, and poor distinguishability between targets and the background, traditional trackers with handcraft visual features achieve poor results in satellite videos. Deep neural networks have shown powerful potential for object tracking in ordinary videos but remain developing in satellite videos. In this letter, a Siamese network and a motion regression network are adopted to form a two-stream deep neural network (SRN) for satellite object tracking, which simultaneously utilizes appearance and motion features. Besides, a trajectory fitting motion (TFM) model based on history trajectories is also employed to further alleviate model drift. Comprehensive experiments demonstrate that the proposed method performs favorably compared with the state-of-the-art tracking methods.

Synthesis of algorithm of distinction of type low-flying high-speed small size object on its acoustic signal

Synthesis of algorithm of distinction of type low-flying high-speed small size object on its acoustic signal

Rotation Equivariant Feature Image Pyramid Network for Object Detection in Optical Remote Sensing Imagery

Detection of objects is extremely important in various aerial vision-based applications. Over the last few years, the methods based on convolution neural networks (CNNs) have made substantial progress. However, because of the large variety of object scales, densities, and arbitrary orientations, the current detectors struggle with the extraction of semantically strong features for small-scale objects by a predefined convolution kernel. To address this problem, we propose the rotation equivariant feature image pyramid network (REFIPN), an image pyramid network based on rotation equivariance convolution. The proposed model adopts single-shot detector in parallel with a lightweight image pyramid module (LIPM) to extract representative features and generate regions of interest in an optimization approach. The proposed network extracts feature in a wide range of scales and orientations by using novel convolution filters. These features are used to generate vector fields and determine the weight and angle of the highest-scoring orientation for all spatial locations on an image. By this approach, the performance for small-sized object detection is enhanced without sacrificing the performance for large-sized object detection. The performance of the proposed model is validated on two commonly used aerial benchmarks and the results show our proposed model can achieve state-of-the-art performance with satisfactory efficiency.

A Small-Sized Object Detection Oriented Multi-Scale Feature Fusion Approach With Application to Defect Detection

Object detection is a well-known task in the field of computer vision, especially the small target detection problem that has aroused great academic attention. In order to improve the detection performance of small objects, in this article, a novel enhanced multiscale feature fusion method is proposed, namely, the atrous spatial pyramid pooling-balanced-feature pyramid network (ABFPN). In particular, the atrous convolution operators with different dilation rates are employed to make full use of context information, where the skip connection is applied to achieve sufficient feature fusions. In addition, there is a balanced module to integrate and enhance features at different levels. The performance of the proposed ABFPN is evaluated on three public benchmark datasets, and experimental results demonstrate that it is a reliable and efficient feature fusion method. Furthermore, in order to validate the applicational potential in small objects, the developed ABFPN is utilized to detect surface tiny defects of the printed circuit board (PCB), which acts as the neck part of an improved PCB defect detection (IPDD) framework. While designing the IPDD, several powerful strategies are also employed to further improve the overall performance, which is evaluated via extensive ablation studies. Experiments on a public PCB defect detection database have demonstrated the superiority of the designed IPDD framework against the other seven state-of-the-art methods, which further validates the practicality of the proposed ABFPN.

Neurocomputational mechanisms underlying cross-modal associations and their influence on perceptual decisions

When exposed to complementary features of information across sensory modalities, our brains formulate cross-modal associations between features of stimuli presented separately to multiple modalities. For example, auditory pitch-visual size associations map high-pitch tones with small-size visual objects, and low-pitch tones with large-size visual objects. Preferential, or congruent, cross-modal associations have been shown to affect behavioural performance, i.e. choice accuracy and reaction time (RT) across multisensory decision-making paradigms. However, the neural mechanisms underpinning such influences in perceptual decision formation remain unclear. Here, we sought to identify when perceptual improvements from associative congruency emerge in the brain during decision formation. In particular, we asked whether such improvements represent ‘early’ sensory processing benefits, or ‘late’ post-sensory changes in decision dynamics. Using a modified version of the Implicit Association Test (IAT), coupled with electroencephalography (EEG), we measured the neural activity underlying the effect of auditory stimulus-driven pitch-size associations on perceptual decision formation. Behavioural results showed that participants responded significantly faster during trials when auditory pitch was congruent, rather than incongruent, with its associative visual size counterpart. We used multivariate Linear Discriminant Analysis (LDA) to characterise the spatiotemporal dynamics of EEG activity underpinning IAT performance. We found an ‘Early’ component (∼100–110 ms post-stimulus onset) coinciding with the time of maximal discrimination of the auditory stimuli, and a ‘Late’ component (∼330–340 ms post-stimulus onset) underlying IAT performance. To characterise the functional role of these components in decision formation, we incorporated a neurally-informed Hierarchical Drift Diffusion Model (HDDM), revealing that the Late component decreases response caution, requiring less sensory evidence to be accumulated, whereas the Early component increased the duration of sensory-encoding processes for incongruent trials. Overall, our results provide a mechanistic insight into the contribution of ‘early’ sensory processing, as well as ‘late’ post-sensory neural representations of associative congruency to perceptual decision formation.