Objects In Images Research Articles

Brief Study on Convolutional Neural Networks

Convolutional Neural Networks (CNNs) have revolutionized the field of computer vision and pattern recognition, offering state-of-the-art performance in tasks such as image classification, object detection, and segmentation. This study explores the architecture, training strategies, and applications of CNNs, focusing on their ability to automatically learn spatial hierarchies of features through layers of convolutional filters. The research evaluates various CNN architectures, including Lent, Alex Net, and ResNet, highlighting their improvements in accuracy and efficiency. Additionally, the study investigates advanced techniques such as data augmentation, transfer learning, and regularization methods (e. g, dropout and batch normalization), which enhance the model's generalization capabilities. Through empirical experiments, we demonstrate the effectiveness of CNNs in real-world scenarios, including medical image analysis, autonomous driving, and facial recognition. The study concludes with a discussion of the future trends of CNNs, particularly in the context of deep learning optimization, hardware acceleration, and integration with emerging technologies like edge computing and quantum machine learning.

Rapid Aircraft Wake Vortex Identification Model Based on Optimized Image Object Recognition Networks

Wake vortices generated by aircraft during near-ground operations have a significant impact on airport safety during takeoffs and landings. Identifying wake vortices in complex airspaces assists air traffic controllers in making informed decisions, ensuring the safety of aircraft operations at airports, and enhancing the intelligence level of air traffic control. Unlike traditional image recognition, identifying wake vortices using airborne LiDAR data demands a higher level of accuracy. This study proposes the IRSN-WAKE network by optimizing the Inception-ResNet-v2 network. To improve the model’s feature representation capability, we introduce the SE module into the Inception-ResNet-v2 network, which adaptively weights feature channels to enhance the network’s focus on key features. Additionally, we design and incorporate a noise suppression module to mitigate noise and enhance the robustness of feature extraction. Ablation experiments demonstrate that the introduction of the noise suppression module and the SE module significantly improves the performance of the IRSN-WAKE network in wake vortex identification tasks, achieving an accuracy rate of 98.60%. Comparative experimental results indicate that the IRSN-WAKE network has higher recognition accuracy and robustness compared to common recognition networks, achieving high-accuracy aircraft wake vortex identification and providing technical support for the safe operation of flights.

An Efficient UAV Image Object Detection Algorithm Based on Global Attention and Multi-Scale Feature Fusion

An Efficient UAV Image Object Detection Algorithm Based on Global Attention and Multi-Scale Feature Fusion

Accurate pneumoconiosis staging via deep texture encoding and discriminative representation learning.

Accurate pneumoconiosis staging is key to early intervention and treatment planning for pneumoconiosis patients. The staging process relies on assessing the profusion level of small opacities, which are dispersed throughout the entire lung field and manifest as fine textures. While conventional convolutional neural networks (CNNs) have achieved significant success in tasks such as image classification and object recognition, they are less effective for classifying fine-grained medical images due to the need for global, orderless feature representation. This limitation often results in inaccurate staging outcomes for pneumoconiosis. In this study, we propose a deep texture encoding scheme with a suppression strategy designed to capture the global, orderless characteristics of pneumoconiosis lesions while suppressing prominent regions such as the ribs and clavicles within the lung field. To further enhance staging accuracy, we incorporate an ordinal label distribution to capture the ordinal information among profusion levels of opacities. Additionally, we employ supervised contrastive learning to develop a more discriminative feature space for downstream classification tasks. Finally, in accordance with standard practices, we evaluate the profusion levels of opacities in each subregion of the lung, rather than relying on the entire chest X-ray image. Experimental results on the pneumoconiosis dataset demonstrate the superior performance of the proposed method confirming its effectiveness for accurate pneumoconiosis staging.

Group-Based Distinctive Image Captioning with Memory Difference Encoding and Attention

AbstractRecent advances in image captioning have focused on enhancing accuracy by substantially increasing the dataset and model size. While conventional captioning models exhibit high performance on established metrics such as BLEU, CIDEr, and SPICE, the capability of captions to distinguish the target image from other similar images is under-explored. To generate distinctive captions, a few pioneers employed contrastive learning or re-weighted the ground-truth captions. However, these approaches often overlook the relationships among objects in a similar image group (e.g., items or properties within the same album or fine-grained events). In this paper, we introduce a novel approach to enhance the distinctiveness of image captions, namely Group-based Differential Distinctive Captioning Method, which visually compares each image with other images in one similar group and highlights the uniqueness of each image. In particular, we introduce a Group-based Differential Memory Attention (GDMA) module, designed to identify and emphasize object features in an image that are uniquely distinguishable within its image group, i.e., those exhibiting low similarity with objects in other images. This mechanism ensures that such unique object features are prioritized during caption generation for the image, thereby enhancing the distinctiveness of the resulting captions. To further refine this process, we select distinctive words from the ground-truth captions to guide both the language decoder and the GDMA module. Additionally, we propose a new evaluation metric, the Distinctive Word Rate (DisWordRate), to quantitatively assess caption distinctiveness. Quantitative results indicate that the proposed method significantly improves the distinctiveness of several baseline models, and achieves state-of-the-art performance on distinctiveness while not excessively sacrificing accuracy. Moreover, the results of our user study are consistent with the quantitative evaluation and demonstrate the rationality of the new metric DisWordRate.

Enhancing Turbidity Predictions in Coastal Environments by Removing Obstructions from Unmanned Aerial Vehicle Multispectral Imagery Using Inpainting Techniques

High-resolution remote sensing of turbidity in the coastal environment with unmanned aerial vehicles (UAVs) can be adversely affected by the presence of obstructions of vessels and marine objects in images, which can introduce significant errors in turbidity modeling and predictions. This study evaluates the use of two deep-learning-based inpainting methods, namely, Decoupled Spatial–Temporal Transformer (DSTT) and Deep Image Prior (DIP), to recover the obstructed information. Aerial images of turbidity plumes in the coastal environment were first acquired using a UAV system with a multispectral sensor that included obstructions on the water surface at various obstruction percentages. The performance of the two inpainting models was then assessed through both qualitative and quantitative analyses of the inpainted data, focusing on the accuracy of turbidity retrieval. The results show that the DIP model performs well across a wide range of obstruction percentages from 10 to 70%. In comparison, the DSTT model produces good accuracy only with low percentages of less than 20% and performs poorly when the obstruction percentage increases.

Mutually-Guided Hierarchical Multi-Modal Feature Learning for Referring Image Segmentation

Referring image segmentation aims to locate and segment the target region based on a given textual expression query. The primary challenge is to understand semantics from visual and textual modalities and achieve alignment and matching. Prior works have attempted to address this challenge by leveraging separately pretrained unimodal models to extract global visual and textual features, and perform straightforwardly fusion to establish cross-modal semantic associations. However, these methods often concentrate solely on the global semantics, disregarding the hierarchical semantics of expression and image, and struggling with complex and open real scenarios, thus failing to capture critical cross-modal information. To address these limitations, this paper introduces an innovative mutually-guided hierarchical multi-modal feature learning scheme. By leveraging the guidance of global visual features, the model mines hierarchical text features from different stages of the text encoder. Simultaneously, the guidance of global textual features is leveraged to aggregate multi-scale visual features. This mutually guided hierarchical feature learning effectively addresses the semantically inaccurate cause by free-form text and naturally occurring scale variations. Furthermore, a Segment Detail Refinement (SDR) module is designed to enhance the model’s spatial detail awareness through attention mapping of low-level visual features and cross-modal features. To evaluate the effectiveness of the proposed approach, extensive experiments are conducted on three widely used referring image object segmentation datasets. The results demonstrate the superiority of the presented method in accurately locating and segmenting objects in images.

Improved Small Object Detection Algorithm CRL-YOLOv5.

Detecting small objects in images poses significant challenges due to their limited pixel representation and the difficulty in extracting sufficient features, often leading to missed or false detections. To address these challenges and enhance detection accuracy, this paper presents an improved small object detection algorithm, CRL-YOLOv5. The proposed approach integrates the Convolutional Block Attention Module (CBAM) attention mechanism into the C3 module of the backbone network, which enhances the localization accuracy of small objects. Additionally, the Receptive Field Block (RFB) module is introduced to expand the model's receptive field, thereby fully leveraging contextual information. Furthermore, the network architecture is restructured to include an additional detection layer specifically for small objects, allowing for deeper feature extraction from shallow layers. When tested on the VisDrone2019 small object dataset, CRL-YOLOv5 achieved an mAP50 of 39.2%, representing a 5.4% improvement over the original YOLOv5, effectively boosting the detection precision for small objects in images.

Data science basis and influencing factors for the evaluation of environmental safety perception in Macau parishes

In the context of rapid urbanization, accurately identifying the visual factors that influence environmental safety perception is crucial for improving urban transportation environments and enhancing pedestrian safety. With the increase in urban population density and traffic flow, optimizing urban environmental design to elevate residents’ sense of safety has become a key issue in urban planning and management. However, the existing studies face numerous challenges in conducting large-scale quantitative analysis of environmental safety perception in complex scenarios, such as difficulties in data acquisition and limitations in analytical methods. This study addresses these challenges by applying image semantic segmentation and object detection techniques to extract key visual elements from street view images, combined with manual scoring and deep learning methods, to construct a road safety perception dataset. Using a LightGBM model and the SHAP interpretation framework, in this study, we identify the critical visual factors influencing environmental safety perception. An empirical study was conducted in Macau, a modern city where Eastern and Western cultures intersect, and tourism thrives. The findings reveal that: ① The overall environmental safety perception in the eight parishes and surrounding roads of Macau is relatively high, with significant regional differences in safety perception scores around Macau’s parish roads; ② The proportions of buildings, sidewalks, roads, and trees in images are the four primary factors influencing environmental safety perception; ③ The proportions and quantities of visual elements interact with each other, and their reasonable distribution helps form clear spatial visibility and creates conducive activity spaces, thereby enhancing the perception of environmental safety. Through empirical analysis, this study uncovers the mechanisms by which visual elements in urban street scenes affect environmental safety perception, providing scientific evidence for urban planning and transportation environment improvement. The research holds theoretical significance and offers practical references for urban design and management, demonstrating broad application value.

A generic shared attention mechanism for various backbone neural networks

The self-attention mechanism is crucial for enhancing various backbone neural networks’ performance. However, current methods add self-attention modules (SAMs) to each network layer without fully utilizing their potential, resulting in suboptimal performance and higher parameter consumption as network depth increases. In this paper, we reveal an inherent phenomenon: SAMs produce highly correlated attention maps across layers, with an average Pearson correlation coefficient of 0.85. Inspired by this inherent observation, we propose Dense-and-Implicit Attention (DIA), which shares SAMs across layers and uses a long short-term memory module to calibrate and connect these correlated attention maps, improving parameter efficiency. This approach aligns with the neural network’s dynamic system perspective. Extensive experiments show DIA consistently enhances various backbones like ResNet, Transformer, and UNet in tasks such as image classification, object detection, and image generation with diffusion models. Our analysis indicates that DIA’s effectiveness stems from its dense inter-layer information connections, absent in conventional mechanisms, stabilizing training and providing regularization effects. This paper’s insights advance our understanding of attention mechanisms, optimizing them and paving the way for future developments across diverse neural networks.

Integrating Clay Modeling in Implant Fabrication for Craniofacial Defect.

Manufacturing craniofacial implants using 3-dimensional (3D) methods and computed tomography data has become popular. The image object for the defect is produced as the first step, followed by several methods to create the implant. The authors have used a novel method that combines clay modeling with 3D scanning to create implants for craniofacial contour reconstruction. This approach does not require complicated resources. The method allows for high customization and immediate modifications, resulting in implants that achieve an accurate fit and high patient satisfaction. It is particularly beneficial for addressing complex defects and achieving aesthetic improvements. In addition, it reduces the need for cumbersome digital processing and expensive materials, making it a practical and feasible solution for a wide range of craniofacial deformities.

EntroCap: Zero-shot image captioning with entropy-based retrieval

State-of-the-art image captioning often relies on supervised models trained on domain-specific image–text pairs, which can be expensive and time-consuming to annotate. Additionally, these models struggle with performance drops when applied to data from different distributions. To overcome these challenges, zero-shot image captioning has emerged, utilizing models trained on unlabeled image and text data. Pre-trained vision-language and generative language models have shown promise in this area, but existing methods do not fully leverage the cross-modal representation capabilities of large-scale models and tend to overlook smaller objects in images. We propose EntroCap, a novel zero-shot image captioning method that incorporates an Entropy-based Retrieval Strategy. During training, our Hierarchical Projector optimizes CLIP’s joint embeddings to capture comprehensive global signals with multi-level representations. During inference, the Entropy-based Retrieval Strategy adjusts prediction logits to better include smaller, less frequent objects. Additionally, our Balancing Gate mechanism guides the language model to produce more detailed and contextually relevant captions by balancing local and global signals. Extensive experiments on MSCOCO, Flickr30k, and NoCaps demonstrate that EntroCap outperforms previous zero-shot approaches in both in-domain and cross-domain settings. We also conduct experiments to quantitatively validate EntroCap’s effectiveness in describing small objects.

Mitigating Adversarial Attacks in Object Detection through Conditional Diffusion Models

The field of object detection has witnessed significant advancements in recent years, thanks to the remarkable progress in artificial intelligence and deep learning. These breakthroughs have significantly enhanced the accuracy and efficiency of detecting and categorizing objects in digital images. Nonetheless, contemporary object detection technologies have certain limitations, such as their inability to counter white-box attacks, insufficient denoising, suboptimal reconstruction, and gradient confusion. To overcome these hurdles, this study proposes an innovative approach that uses conditional diffusion models to perturb adversarial examples. The process begins with the application of a random chessboard mask to the adversarial example, followed by the addition of a slight noise to fill the masked area during the forward process. The adversarial image is then restored to its original form through a reverse generative process that only considers the masked pixels, not the entire image. Next, we use the complement of the initial mask as the mask for the second stage to reconstruct the image once more. This two-stage masking process allows for the complete removal of global disturbances and aids in image reconstruction. In particular, we employ a conditional diffusion model based on a class-conditional U-Net architecture, with the source image further conditioned through concatenation. Our method outperforms the recently introduced HARP method by 5% and 6.5% in mAP on the COCO2017 and PASCAL VOC datasets, respectively, under non-APT PGD attacks. Comprehensive experimental results confirm that our method can effectively restore adversarial examples, demonstrating its practical utility.

PS-Net: A Learning Strategy for Accurately Exposing the Professional Photoshop Inpainting.

Restoring missing areas without leaving visible traces has become a trivial task with Photoshop inpainting tools. However, such tools have potentially illegal or unethical uses, such as removing specific objects in images to deceive the public. Despite the emergence of many forensics methods of image inpainting, their detection ability is still insufficient when attending to professional Photoshop inpainting. Motivated by this, we propose a novel method termed primary-secondary network (PS-Net) to localize the Photoshop inpainted regions in images. To the best of our knowledge, this is the first forensic method devoted specifically to Photoshop inpainting. The PS-Net is designed to deal with the problems of delicate and professional inpainted images. It consists of two subnetworks: the primary network (P-Net) and the secondary network (S-Net). The P-Net aims at mining the frequency clues of subtle inpainting features through the convolutional network and further identifying the tampered region. The S-Net enables the model to mitigate compression and noise attacks to some extent by increasing the co-occurring feature weights and providing features that are not captured by the P-Net. Furthermore, the dense connection, Ghost modules, and channel attention blocks (C-A blocks) are adopted to further strengthen the localization ability of PS-Net. Extensive experimental results illustrate that PS-Net can successfully distinguish forged regions in elaborate inpainted images, outperforming several state-of-the-art solutions. The proposed PS-Net is also robust against some postprocessing operations commonly used in Photoshop.

Image Enhancement Guided Object Detection in Visually Degraded Scenes.

Object detection accuracy degrades seriously in visually degraded scenes. A natural solution is to first enhance the degraded image and then perform object detection. However, it is suboptimal and does not necessarily lead to the improvement of object detection due to the separation of the image enhancement and object detection tasks. To solve this problem, we propose an image enhancement guided object detection method, which refines the detection network with an additional enhancement branch in an end-to-end way. Specifically, the enhancement branch and detection branch are organized in a parallel way, and a feature guided module is designed to connect the two branches, which optimizes the shallow feature of the input image in the detection branch to be as consistent as possible with that of the enhanced image. As the enhancement branch is frozen during training, such a design plays a role in using the features of enhanced images to guide the learning of object detection branch, so as to make the learned detection branch being aware of both image quality and object detection. When testing, the enhancement branch and feature guided module are removed, and so no additional computation cost is introduced for detection. Extensive experimental results, on underwater, hazy, and low-light object detection datasets, demonstrate that the proposed method can improve the detection performance of popular detection networks (YOLO v3, Faster R-CNN, DetectoRS) significantly in visually degraded scenes.

Novel entropy-based style transfer of the object in the content image using deep learning

Novel entropy-based style transfer of the object in the content image using deep learning

Self-supervised learning-based re-parameterization instance segmentation for hazardous object in terahertz image

Recently, terahertz imaging technology has shown widespread potential applications in the public security screening field. Previous terahertz hazardous object detection techniques primarily relied on manual recognition and image processing methods. Some solutions incorporating deep learning technologies depend on large quantities of high-quality data, making it challenging to achieve low-cost and high-performance detection. To solve the issue of dependency on large amounts of high-quality data, we propose a diversified dynamic structure You Only Look Once (DDS-YOLO) model based on masked image modeling and structural reparameterization for the instance segmentation of hazardous objects in terahertz security inspection images. To address the scarcity and annotating difficulty problems of terahertz security inspection image samples, we combine automatic data strategies with masked image modeling for self-supervised learning. We propose a multilevel feature refinement fusion mechanism to enhance the quality of learned feature representations. The backbone parameter transfer and fine-tuning training strategies are employed to achieve hazardous object instance segmentation on the terahertz dataset. To address the low detection accuracy issue caused by the poor terahertz image quality, we develop a reparameterizable hierarchical structure for the backbone, improve a multi-scale feature-integrating neck, and design a dynamically decoupled head with lower computational requirements to enhance the performance of the instance segmentation model. Experimental results demonstrate that the proposed model accurately outputs detection boxes, categories, and segmentation masks for hazardous objects with minimal training samples. The comparative experimental results indicate that the proposed model outperforms existing state-of-the-art methods in terms of detection performance. The proposed DDS-YOLO model achieves 59.3 % in mask mean Average Precision (mAP) and 61.3 % in box mAP, and the model parameters and computational requirements also meet practical application scenarios.

Adaptive multilevel attention deeplabv3+ with heuristic based frame work for semantic segmentation of aerial images using improved golden jackal optimization algorithm

Semantic segmentation of small-scale objects in aerial images is challenging due to low-level characteristics in neural networks and varying information in feature maps. Deeplab models struggle with poor edge refinement, resulting in rough borders, and fail to fully exploit relationships between pixel categories at different distances. To addressing the issue in deeplab series, an adaptive multi-level attention based deeplabv3+ (AMLA-Deeplabv3+) with improved golden jackal optimization algorithm is implemented in this paper. Multi-level attention unit has been included in the Atrous spatial pyramid pooling module in the encoder section of deeplabv3+ to bridge the semantic feature gap among encoders output. To put more weights on relevant features squeeze and excitation units has been included in the decoder section of deeplabv3+. The improved golden jackal optimization (IGJO) algorithm is deployed to fine tune the hyper parameters such as number of hidden neurons, epochs, learning rate and batch size of deeplabv3+. The proposed AMLA-Deeplabv3+ is tested on two publicly available datasets named semantic segmentation of aerial imagery and aerial image segmentation. The proposed model achieves better segmentation outcomes in terms of accuracy of 99.65%, Precision of 97.21%, F1-score of 99.48% and mean intersection over union of 98.44% with a computation time of 3.0231 minutes. The experimental outcomes shows that the proposed model’s efficiency is superior to other conventional techniques.

Application of deep learning reconstruction in abdominal magnetic resonance cholangiopancreatography for image quality improvement and acquisition time reduction

PurposeTo compare deep learning (DL)-based and conventional reconstruction through subjective and objective analysis and ascertain whether DL-based reconstruction improves the quality and acquisition speed of clinical abdominal magnetic resonance imaging (MRI). MethodsThe 124 patients who underwent abdominal MRI between January and July 2021 were retrospectively studied. For each patient, two-dimensional axial T2-weighted single-shot fast spin-echo MRI images with or without fat saturation were reconstructed using DL-based and conventional methods. The subjective image quality scores and objective metrics, including signal-to-noise ratios (SNRs) and contrast-to-noise ratios (CNRs) of the images were analysed. An explorative analysis was performed to compare 20 patients’ MRI images with site routine settings, high-resolution settings and high-speed settings. Paired t tests and Wilcoxon signed-rank tests were used for subjective and objective comparisons. ResultsA total of 144 patients were evaluated (mean age, 62.2 ± 14.1 years; 83 men). The MRI images reconstructed using DL-based methods had higher SNRs and CNRs than did those reconstructed using conventional methods (all p < 0.01). The subjective scores of the images reconstructed using DL-based methods were higher than those of the images reconstructed using conventional methods (p < 0.01), with significantly lower variation (p < 0.01). Exploratory analysis revealed that the DL-based reconstructions with thin slice thickness and higher temporal resolution had the highest image quality and were associated with the shortest scan times. ConclusionsDL-based reconstruction methods can be used to improve the quality with higher stability and accelerate the acquisition of abdominal MRI.

Deep convolutional neural networks for ship detection using refined DOTA and TGRS-HRRSD high-resolution image datasets

Automated identification of ships in satellite imagery is of utmost significance in many military and civilian applications, including monitoring maritime traffic and maintaining maritime security. However, ship detection from remotely sensed imagery is a challenging task because of their varying sizes, orientations, types, and various interferences. In recent years, deep learning algorithms have become a powerful tool in image processing applications, including image classification and object detection due to their strong feature representation capabilities. In this study, a refined ship dataset, containing a total of 13,735 instances and representing different ship types, was generated using state-of-the-art datasets for the ship detection task. In a comparative framework, the performance of five deep learning algorithms (Faster R-CNN, YOLOv8, CenterNet, EfficientDet, and SSD) in ship detection was investigated using the created dataset. The performance of the deep learning models was compared using MS COCO accuracy metrics. In addition, the generalization capabilities of the models were tested on an independent image acquired by Göktürk-1 having a mucilage effect that undermines the spectral discrimination. Results revealed that the YOLOv8 model had the greatest ship detection performance (AP@0.50 = 96.33 %), followed by CenterNet (AP@0.50 = 88.80 %), Faster R-CNN (AP@0.50 = 85.30 %), EfficientDet (AP@0.50 = 78.91 %) and SSD (AP@0.5 = 75.98 %), and obtained the highest accuracies in terms of other COCO metrics. In addition, generalization capabilities on the test image confirmed the superiority of the YOLOv8 model with AP@0.50 score of 63.93 %. Overall, the results of this study validated the effectiveness of the YOLOv8 model in identifying ship targets in remotely sensed images.