Objects In Images Research Articles

Privacy‐protected object detection through trustworthy image fusion

AbstractThe neural network‐based technologies have emerged as a potent method for image fusion, object detection, and other computer vision tasks as the rapid development of deep learning. Multi‐band infrared images, in particular, capture a more extensive range of radiation details and information compared to conventional single‐band infrared images. Consequently, the fusion of multi‐band infrared images can provide more features for object detection. However, it is crucial to consider that infrared images may contain sensitive information, potentially leading to privacy concerns. Ensuring datasets privacy protection plays a crucial role in the fusion and tracking process. To address both the need for improved detection performance and the necessity for privacy protection in the infrared environment, we proposed a procedure for object detection based on multi‐band infrared image datasets and utilized the transfer learning technique to migrate knowledge learned from external infrared data to internal infrared data, thereby training the infrared image fusion model and detection model. The procedure consists of several steps: (1) data preprocessing of multi‐band infrared images, (2) multi‐band infrared image fusion, and (3) object detection. Standard evaluation metrics for image fusion and object detection ensure the authenticity of the experiments. The comprehensive validation experiments demonstrate the effectiveness of the proposed procedure in object detection tasks. Furthermore, the transfer learning can train our datasets and update the model without exposing the original data. This aspect of transfer learning is particularly beneficial for maintaining the privacy of multi‐band infrared images during the fusion and detection processes.

Intricate multiple scattering features of artificial facilities in X-Band SAR images

Abstract. Due to the intricate distortion and reflection geometry of the SAR signal, it is typically difficult to determine the multiple scattering of large artificial objects in SAR images. This work presents a scattering point path tracking model that utilizes the real three-dimensional dimensions of targets, based on the geometric optics method. Three different artificial structures, including light poles, cable stayed bridges, and power transmission lines, are carefully analysed in time-series SAR images with their simulated multiple scattering results. The results demonstrate that the routes determined by the model are consistent with the multiple scattering features on SAR images. Moreover, the time-series data demonstrate that ripples in the water's surface have a significant impact on the multi-scattering features of power lines and bridges. The double scattering features of the light pole provides a novel approach to the process of permanent scatterers (PS) in urban areas. The instances presented in this study demonstrate the effectiveness of the scattering point path tracking model in identifying the various artificial facility targets on different reflective surfaces. It will be a useful tool for deciphering the multiple scattering of large artificial structures when their 3D model is known.

Finding the Next Successor: Visual Message in Caricature Image

Caricature is a form of visual communication wherein messages are conveyed through exaggerated and often humorous distortions of characters, serving as a critique of societal issues. Caricature often serves to critique various aspects of society, including social, cultural, economic, legal, and political conditions. Similarly, the political landscape surrounding the 2024 general election has intensified, fostering active and diverse discussions through both direct and indirect media channels. Mass media visualizes the very busy pros and cons of the Indonesian future successors. The proponents and the supported people are arguing and the one that can represent them will be the leader of government in 2024-2029. Funny, amusing, and intriguing caricature images in National Mass Media are interesting to study to see how the interpretation on the form of a caricature object is using the meaningful visual signs. This research was limited to caricature images by a cartoonist, Kang Yuyun, that was published in TEMPO National mass media in January-June 2023. The object of illustrative image was limited using a purposive sample that is considered representing the objective of research. The data mining was carried out through observation, interview, document, and literature. Observation was conducted directly in Tempo’s Office, Jakarta; interview was conducted online with the cartoonist; document included the caricature images that have been published on Sunday; and literature came from scientific works such as journal, proceeding, and books. All data were collected to be validated using triangulation and then analyzed using Roland Barthes’ semiotic theory about denotation, connotation, and myth. The findings reveal that Kang Yuyun’s caricature images in TEMPO mass media during the 2024 General Election depict three groups supporting Ganjar Pranowo, Prabowo Subianto, and Anis Bawesdan as potential successors, emphasizing unity in nation-building and promotion. Keywords: caricature, mass media, the successor, semiotics

Research on Bidirectional Multi-Span Feature Pyramid and Key Feature Capture Object Detection Network

UAV remote sensing (RS) image object detection is a very valuable and challenging technology. This article discusses the importance of key features and proposes an object detection network (URSNet) based on a bidirectional multi-span feature pyramid and key feature capture mechanism. Firstly, a bidirectional multi-span feature pyramid (BMSFPN) is constructed. In the process of bidirectional sampling, bicubic interpolation and cross layer fusion are used to filter out image noise and enhance the details of object features. Secondly, the designed feature polarization module (FPM) uses the internal polarization attention mechanism to build a powerful feature representation for classification and regression tasks, making it easier for the network to capture the key object features with more semantic discrimination. In addition, the anchor rotation alignment module (ARAM) further refines the preset anchor frame based on the key regression features extracted by FPM to obtain high-quality rotation anchors with a high matching degree and rich positioning visual information. Finally, the dynamic anchor optimization module (DAOM) is used to improve the ability of feature alignment and positive and negative sample discrimination of the model so that the model can dynamically select the candidate anchor to capture the key regression features so as to further eliminate the deviation between the classification and regression. URSNet has conducted comprehensive ablation and SOTA comparative experiments on challenging RS datasets such as DOTA-V2.0, DIOR and RSOD. The optimal experimental results (87.19% mAP, 108.2 FPS) show that URSNet has efficient and reliable detection performance.

Detection of objects in satellite and aerial imagery using channel and spatially attentive YOLO-CSL for surveillance

A novel lightweight Rotational Object Detection algorithm is proposed to overcome the shortcomings of conventional computer-vision-aided object detection methods used in Remote Sensing and Surveillance that overlook the variability in size and orientation of objects in satellite and aerial images. This advanced algorithm integrates a branch dedicated to angle prediction and employs the circular smooth label (CSL) method for angle classification. This approach is suitable for scenarios that require detection in rotational boxes. Our work is further distinguished by the introduction of a novel Channel and Spatial Attention (CSA) module, which is seamlessly integrated into the YOLOv5-CSL framework via the C3CS module. This module accentuates pertinent features through both the channel and spatial attention mechanisms. In addition, bicubic interpolation and the GELU activation function were incorporated into the YOLOv5-CSLA model. Our model achieved 57.86 mAP on the challenging DOTA v2 dataset surpassing the second-best method by 0.20 points and simultaneously consuming 11 million fewer parameters and 103 fewer GFLOPs (our model consumes 25 M Params and 54 GFLOPs), justifying its suitability for deployment on a large majority of platforms, as the compute required is a challenge in real-time deployment scenarios.

Improved Hybrid K-Nearest Neighbours Techniques in Segmentation of Low-Grade Tumor and Cerebrospinal Fluid

In image segmentation, identifying information or object detection in medical im-ages is crucial, particularly information that is harder to spot in magnetic resonance imaging (MRI) of low-grade tumors or cerebrospinal fluid (CSF). To address the aforementioned problems associated with missing data in MRI images and the low quality of MRI images that required longer processing times, this research is to seg-ment brain tumors or detect CSF in four-dimensional MRI images. A new hybrid k-nearest neighbors (k-NN) framework is also proposed, which consists of three tech-niques: correlation matrices of discrete Fourier transform (CM-DFT), Laplace Eigen maps of locally preserving projection (LELPP), and a hybrid GrabCut hidden Markov model of k-mean clustering (GCHMkC). The combination of the Hidden Markov Model (HMM) and the k-mean clustering technique is known as the Hidden Markov Model of the k-mean clustering method (HMMkC). To begin with, the Graph Cut and Support Vector Machine (GCSVM) and the GCHMkC approach are combined. The method increased the quality of the images suggested by the methodology, achieving an accuracy of 99.83%, a sensitivity of 99.99%, a specificity of 99.8%, and a computa-tional execution time of 14.9 seconds. Second, a technique called CM-DFT is sug-gested to improve MRI images while resolving the issue of missing imputation data. The accuracy of the MRI image datasets was improved to 99.84%, the time-lag in the hybrid k-NN algorithm was reduced to 99%, the missing data ratio was reduced to 0.9%, 10%, and 12%, and the correctness of the imputed data was improved to 1.533 seconds with computational execution. Thirdly, the nonlinear data is reduced and unnecessary features are eliminated using the Laplace Eigen maps of locally pre-serving projection (LELPP) approach. The hybrid k-NN algorithm used by the tech-nique yields results with 99% accuracy and an execution duration of 2.42 seconds.

TA-YOLO: a lightweight small object detection model based on multi-dimensional trans-attention module for remote sensing images

Object detection plays a vital role in remote sensing applications. Although object detection has achieved proud results in natural images, these methods are difficult to be directly applied to remote sensing images. Remote sensing images often have complex backgrounds and small objects, which results in a highly unbalanced distribution of foreground and complex background information. In order to solve the above problems, this paper proposes a multi-head channel and spatial trans-attention (MCSTA) module, which performs remote pixel interaction from the channel and spatial dimensions respectively to complete the attention feature capture function. It is a plug-and-play module that can be easily embedded in any other natural image object detection convolutional neural network, making it quickly applicable to remote sensing images. First, in order to reduce computational complexity and improve feature richness, we use a special linear convolution to obtain three projection features instead of the simple matrix multiplication transformation in Transformer. Second, we obtain trans-attention maps in different dimensions in a manner similar to the self-attention mechanism to capture the interrelationships of features in channels and spaces. In this process, we use a multi-head mechanism to perform parallel operations to improve speed. Furthermore, in order to avoid large-scale matrix operations, we specially designed an attention blocking mode to reduce computer memory usage and increase operation speed. Finally, we embedded the trans-attention module into YOLOv8, added a new detection head and optimized the feature fusion method, thus designing a lightweight small object detection model named TA-YOLO for remote sensing images. It has fewer parameters than the benchmark model YOLOv8, and its mAP on the PASCAL VOC and VisDrone data sets increased by 1.3% and 6.2% respectively. The experimental results prove the powerful function of the trans-attention module and the excellent performance of TA-YOLO.

MFEFNet: A Multi-Scale Feature Information Extraction and Fusion Network for Multi-Scale Object Detection in UAV Aerial Images

Unmanned aerial vehicles (UAVs) are now widely used in many fields. Due to the randomness of UAV flight height and shooting angle, UAV images usually have the following characteristics: many small objects, large changes in object scale, and complex background. Therefore, object detection in UAV aerial images is a very challenging task. To address the challenges posed by these characteristics, this paper proposes a novel UAV image object detection method based on global feature aggregation and context feature extraction named the multi-scale feature information extraction and fusion network (MFEFNet). Specifically, first of all, to extract the feature information of objects more effectively from complex backgrounds, we propose an efficient spatial information extraction (SIEM) module, which combines residual connection to build long-distance feature dependencies and effectively extracts the most useful feature information by building contextual feature relations around objects. Secondly, to improve the feature fusion efficiency and reduce the burden brought by redundant feature fusion networks, we propose a global aggregation progressive feature fusion network (GAFN). This network adopts a three-level adaptive feature fusion method, which can adaptively fuse multi-scale features according to the importance of different feature layers and reduce unnecessary intermediate redundant features by utilizing the adaptive feature fusion module (AFFM). Furthermore, we use the MPDIoU loss function as the bounding-box regression loss function, which not only enhances model robustness to noise but also simplifies the calculation process and improves the final detection efficiency. Finally, the proposed MFEFNet was tested on VisDrone and UAVDT datasets, and the mAP0.5 value increased by 2.7% and 2.2%, respectively.

A convolutional neural network (Seg-CNN) for recognition, classification, and segmentation of apple fruit branches and stems in images

This article reports the results of research studies conducted in 2023–2024 on transfer learning of Segmentation Convolutional Neural Networks (Seg-CNN) models for classification, recognition, and segmentation of branches with apple fruits and stems in images. State-of-the-art convolutional neural network architectures, i.e., YOLOv8(n,s,m,l,x)-seg, were used for a detailed segmentation of biological objects in images of varying complexity and scale at the pixel level. An image dataset collected in the field using a GoPro HERO 11 camera was marked up for transfer model training. Data augmentation was performed, producing a total of 2500 images. Image markup was performed using the polygon annotation tool. As a result, polygonal contours around objects were created, outlines of branches, apple tree fruits, and stems were outlined, and segments of objects in the images were indicated. The objects were assigned the following classes: Apple branch, Apple fruit, and Apple stem. Binary classification metrics, such as Precision and Recall, as well as Mean Average Precision (mAP), were used to evaluate the performance of the trained models in recognizing branches with apple fruits and stems in images. The YOLOv8x-seg (mAP50 0.758) and YOLOv8l-seg (mAP50 0.74) models showed high performance in terms of all metrics in recognizing branches, apple fruit, and fruit stems in images, outperforming the YOLOv8n-seg (mAP50 0.7) model due to their more complex architecture. The YOLOv8n-seg model has a faster frame processing speed (11.39 frames/s), rendering it a preferred choice for computing systems with limited resources. The results obtained confirm the prospects of using machine learning algorithms and convolutional neural networks for segmentation and pixel-by-pixel classification of branches with apple fruits and stems on RGB images for monitoring the condition of plants and determining their geometric characteristics.

Single-pixel imaging based on self-supervised conditional mask classifier-free guidance

Reconstructing high-quality images at a low measurement rate is a pivotal objective of Single-Pixel Imaging (SPI). Currently, deep learning methods achieve this by optimizing the loss between the target image and the original image, thereby constraining the potential of low measurement values. We employ conditional probability to ameliorate this, introducing the classifier-free guidance model (CFG) for enhanced reconstruction. We propose a self-supervised conditional masked classifier-free guidance (SCM-CFG) for single-pixel reconstruction. At a 10% measurement rate, SCM-CFG efficiently completed the training task, achieving an average peak signal-to-noise ratio (PSNR) of 26.17 dB on the MNIST dataset. This surpasses other methods of photon imaging and computational ghost imaging. It demonstrates remarkable generalization performance. Moreover, thanks to the outstanding design of the conditional mask in this paper, it can significantly enhance the accuracy of reconstructed images through overlay. SCM-CFG achieved a notable improvement of an average of 7.3 dB in overlay processing, in contrast to only a 1 dB improvement in computational ghost imaging. Subsequent physical experiments validated the effectiveness of SCM-CFG.

From object obfuscation to contextually-dependent identification: enhancing automated privacy protection in street-level image platforms (SLIPs)

ABSTRACT Street-level image platforms (SLIPs) employ indiscriminate forms of data collection that include potentially privacy invasive images. Both the scale and the indiscriminate nature of data collection means that significant privacy management requirements are needed. Legal risk management is currently operated through obfuscation techniques involving certain image objects. Current SLIP object obfuscation solutions are an indiscriminate and a blunt solution to a similarly indiscriminate data collection concern. A new contextual approach to obfuscation is required that goes beyond object obfuscation. Contextually-dependent identification would seek to identify the contexts, including captured objects, which can give rise to privacy concerns. It is technically more challenging for automated solutions as it requires an assessment of the contextual situation to understand privacy risk. Context-sensitive privacy detection, combined with context-sensitive privacy-by-design processes, potentially offer a risk management solution that better situates and addresses the concerns arising from SLIP data collections.

CrossFormer++: A Versatile Vision Transformer Hinging on Cross-Scale Attention.

While features of different scales are perceptually important to visual inputs, existing vision transformers do not yet take advantage of them explicitly. To this end, we first propose a cross-scale vision transformer, CrossFormer. It introduces a cross-scale embedding layer (CEL) and a long-short distance attention (LSDA). On the one hand, CEL blends each token with multiple patches of different scales, providing the self-attention module itself with cross-scale features. On the other hand, LSDA splits the self-attention module into a short-distance one and a long-distance counterpart, which not only reduces the computational burden but also keeps both small-scale and large-scale features in the tokens. Moreover, through experiments on CrossFormer, we observe another two issues that affect vision transformers' performance, i.e., the enlarging self-attention maps and amplitude explosion. Thus, we further propose a progressive group size (PGS) paradigm and an amplitude cooling layer (ACL) to alleviate the two issues, respectively. The CrossFormer incorporating with PGS and ACL is called CrossFormer++. Extensive experiments show that CrossFormer++ outperforms the other vision transformers on image classification, object detection, instance segmentation, and semantic segmentation tasks.

Efficient tensor decomposition-based filter pruning

In this paper, we present CORING, which is short for effiCient tensOr decomposition-based filteR prunING, a novel filter pruning methodology for neural networks. CORING is crafted to achieve efficient tensor decomposition-based pruning, a stark departure from conventional approaches that rely on vectorized or matricized filter representations. Our approach represents a significant leap forward in the field by introducing tensor decompositions, specifically the HOSVD, which preserves the multidimensional nature of filters while providing a low-rank approximation, thus substantially reducing complexity. Furthermore, we introduce a versatile method for calculating filter similarity by using the low-rank approximation offered by the HOSVD. This obviates the need for using full filters or reshaped versions and enhances the overall efficiency and effectiveness of our approach. Extensive experimentation across diverse architectures and datasets spanning various vision tasks, including image classification, object detection, instance segmentation, and keypoint detection, validates CORING’s prowess. Remarkably, it outperforms state-of-the-art methods in reducing MACs and parameters, consistently enhancing validation accuracy. Furthermore, we supplement our quantitative results with a comprehensive ablation study, providing substantial evidence of the efficiency of our tensor-based approach. Beyond quantitative outcomes, qualitative results vividly illustrate CORING’s ability to retain essential features within pruned neural networks. Our code is available for research purposes.

Heuristic data-driven anchor generation for UAV-based maritime rescue image object detection

The specificity of scenarios and tasks in Unmanned Aerial Vehicles (UAV)-based maritime rescue poses challenges for detecting targets within images captured by drones in such environments. This study focuses on leveraging heuristic methods to extract data features from specific UAV maritime rescue images to optimize the generation of anchor boxes in detection models. Experiments conducted on the large-scale SeaDronesSee maritime rescue dataset, using the MMDetection object detection framework, demonstrated that the optimized anchor boxes, improved model performance by 48.9% to 62.8% compared to the framework's default configuration, with the most proficient model surpassing the official highest SeaDronesSee baseline by over 49.3%. Further analysis of the results revealed the variation in detection difficulty for different objects within the dataset and identified the reasons behind these differences. The methodology and analysis presented in this study hold promise for optimizing UAV-based maritime rescue object detection models as well as refining data analysis and enhancement.

Reducing the volume of computations when building analogs of neural networks for the first stage of an ensemble classifier with stacking

The object of research in this work is ensemble classifiers with stacking, intended for the classification of objects in images with the presence of small sets of labeled data for training. To improve the quality of classification at the first stage of such a classifier, it is necessary to place more primary classifiers that differ in heterogeneous structured processing. However, the number of known neural networks with appropriate characteristics is limited. One approach to solving this problem is to build analogs of known neural networks that make classification errors on other images compared to the base network. The disadvantage of the known methods for constructing such analogs is the need to perform additional floating-point operations. The current paper proposes and investigates a new method to form analogs through random cyclic shifts of rows or columns of input images. This has made it possible to completely eliminate additional floating-point operations. The effectiveness of using this method is explained by the structured processing of input images in basic neural networks. The use of analogs obtained by the proposed method does not impose additional restrictions in practice. This is because the heterogeneity of structured processing in basic neural networks is a typical requirement for them in an ensemble classifier with stacking. The simulation for the CIFAR-10 data set demonstrated that the proposed technique for constructing analogs allows for a comparative quality of classification by the ensemble classifier. Using MLP-Mixer analogs provided an improvement of 4.6 %, and CCT analogs – 5.9 %

An Analysis: Progress in Identifying Objects, Classification and Segmenting them in Satellite Images

Abstract: Recognizing and differentiating items in satellite pictures is crucial for different goals, such as city planning and disaster response. Recent developments in machine learning, particularly deep learning, and the abundance of high-quality satellite images have significantly boosted progress in this area. CNNs have become essential for precisely identifying and categorising elements depicted in satellite images. These models use convolution and pooling methods on a local scale to identify local patterns and far-reaching connections, ultimately enhancing object detection and segmentation accuracy. This review offers a detailed summary of the advanced methods used to identify and separate objects in satellite images. The article discusses the importance of precise object detection and segmentation in today's dynamic environment, showcasing the progress achieved through deep learning methodologies. The article explores the difficulties of pinpointing key and distinct areas in satellite pictures and shows how deep learning models tackle this issue by detecting the connections between objects and various facets of the surroundings.

Defect Detection of Tomato Fruit Using Image Dataset

Abstract: The tomato crop is more susceptible to disease than any other vegetable, and it can be infected with over 200 diseases caused by different pathogens worldwide. Tomato plant diseases have become a challenge to food security globally. Currently, diagnosing and preventing tomato plant diseases is a challenge due to the lack of essential methods or tools.The traditional techniques of detecting plant disease are arduous and error-prone. Utilizing precise or automatic detection methods in spotting early plant disease can improve the quality of food production and reduce adverse effects. Deep learning has significantly increased the recognition accuracy of image classification and object detection systems in recent years. In this study, a 15-layer convolutional neural network is proposed as the backbone for single shot detector (SSD) to improve the detection of healthy, and three classes of tomato fruit diseases. The proposed model performance is compared with ResNet-50, AlexNet, VGG 16, and VGG19 as the backbone for Single shot detector. The findings of the experiment showed that the proposed CNN-SDD achieved 98.87% higher detection accuracy, which outperformed state-of-the-art models.

Vision-Based Asphalt Pavement Defect Detection Using Deep CNN with BIM Integration

Abstract: The major objective of the research is to predict the defects on the asphalt pavement which is the main concern for the smooth movement of vehicles by using deep CNN. CNN consists of different operations that is image classification, object detection, and image segmentation. We use detectron2 for labelling the defective part to detect the defect accurately. The data set which consists of images of defects of pavement is collected and used for training the algorithm and some of the collected images are used for the testing. The suggested approach makes use of deep learning to automatically detect and categorize a range of asphalt pavement defects, such as surface distress, potholes, and cracks. The training, validation, and testing of the model are conducted on an extensive dataset of annotated pavement photographs. The trained model shows resilience and the ability to generalize, which qualifies it for practical uses in infrastructure management and pavement upkeep. This study advances automated pavement inspection systems, which provide a practical and economical means of determining which maintenance tasks should be prioritized to maintain safer and smoother roads and this approach predicted the defects with an accuracy of 98%.

Review on Detecting Pneumonia in Deep Learning

Deep learning is a machine learning technique that has been optimized for image classification and object detection. Deep learning has brought huge advancement to the medical field as it helps to diagnose various diseases through computed tomography (CT) scan or X-ray images. Pneumonia is a respiratory disease, and it is one of the killer diseases that causes numerous death all around the world. In 2019, the outbreak of COVID-19 has increased the number of pneumonia patients tremendously. With the increasing number of patients, the clinical and medical facilities have become insufficient. The lack of doctors and radiologists to diagnose pneumonia has caused a high number of patients to be misdiagnosed. Chest image is one of the most effective methods to diagnose this disease, however, examining the X-ray or CT images requires specialists such as radiologists. Meanwhile, examining chest CT or X-ray images might be subjective as the presence of pneumonia can be unclear in the images. The main objective of this paper is to provide a comprehensive review of recent advancement in the diagnosis of pneumonia with deep learning, including state-of-art methodology, datasets, discussion, challenges, and future improvements.

Digital Image Object Detection with GLCM Multi-Degrees and Ensemble Learning

Object detection in digital images has been implemented in various fields. Object detection faces challenges, one of which is rotation problems, causing objects to become unknown. We need a method that can extract features that do not affect rotation and reliable ensemble-based classification. The proposal uses the GLCM-MD (Gray-Level Co-occurrence Matrix Multi-Degrees) extraction method with classification using K-Nearest Neighbours (K-NN) and Random Forest (RF) learning as well as Voting Ensemble (VE) from two single classifications. The main goal is to overcome the difficulty of detecting objects when the object experiences rotation which results in significant visualization variations. In this research, the GLCM method is used to produce features that are stable against rotation. Furthermore, classification methods such as K-Nearest Neighbours (KNN), Random Forest (RF), and KNN-RF fusion using the Voting ensemble method are evaluated to improve detection accuracy. The experimental results show that the use of multi-degrees and the use of ensemble voting at all degrees can increase the accuracy value, and the highest accuracy for extraction using multi-degrees is 95.95%. Based on test results which show that the use of features of various degrees and the ensemble voting method can increase accuracy for detecting objects experiencing rotation