Low-level Features Research Articles

An Effective and Lightweight Full-Scale Target Detection Network for UAV Images Based on Deformable Convolutions and Multi-Scale Contextual Feature Optimization

Currently, target detection on unmanned aerial vehicle (UAV) images is a research hotspot. Due to the significant scale variability of targets and the interference of complex backgrounds, current target detection models face challenges when applied to UAV images. To address these issues, we designed an effective and lightweight full-scale target detection network, FSTD-Net. The design of FSTD-Net is based on three principal aspects. Firstly, to optimize the extracted target features at different scales while minimizing background noise and sparse feature representations, a multi-scale contextual information extraction module (MSCIEM) is developed. The multi-scale information extraction module (MSIEM) in MSCIEM can better capture multi-scale features, and the contextual information extraction module (CIEM) in MSCIEM is designed to capture long-range contextual information. Secondly, to better adapt to various target shapes at different scales in UAV images, we propose the feature extraction module fitting different shapes (FEMFDS), based on deformable convolutions. Finally, considering low-level features contain rich details, a low-level feature enhancement branch (LLFEB) is designed. The experiments demonstrate that, compared to the second-best model, the proposed FSTD-Net achieves improvements of 3.8%, 2.4%, and 2.0% in AP50, AP, and AP75 on the VisDrone2019, respectively. Additionally, FSTD-Net achieves enhancements of 3.4%, 1.7%, and 1% on the UAVDT dataset. Our proposed FSTD-Net has better detection performance compared to state-of-the-art detection models. The experimental results indicate the effectiveness of the FSTD-Net for target detection in UAV images.

Tripartite evolutionary game and simulation analysis of electric bus charging facility sharing under the governmental reward and punishment mechanism

Electric vehicles (EVs) are environmentally friendly with the features of low noise levels and zero tailpipe emissions. However, limited charging facilities have restricted the large-scale development of EVs, and electric bus charging facility sharing is a promising solution to this issue. While reducing charging time and improving facility utilization, this measure involves the interests of the government, bus operators, and electric car owners, requiring a systemic analysis of their interactions. Therefore, this study constructs a tripartite evolutionary game model under the governmental reward and punishment mechanism. The model examines the strategic decision-making process, influential factors, and evolutionary stability of three players, followed by simulation analysis. The results show that: (1) The evolutionary system may exhibit three stable states: (0,0,0), (1,0,0) and (1,1,1), and enhance the initial ratio of the three stakeholders will help evolve towards the ideal state. (2) Increasing rewards and penalties can both be meaningful, when penalties exceed rewards, the measures can be more effective. (3) Bus operators are more sensitive to incentive and penalty measures than electric car owners, and governments should pay more attention to them in the implementation process. The findings can provide references for the policy formulation on electric bus charging facility sharing.

AerialFormer: Multi-Resolution Transformer for Aerial Image Segmentation

When performing remote sensing image segmentation, practitioners often encounter various challenges, such as a strong imbalance in the foreground–background, the presence of tiny objects, high object density, intra-class heterogeneity, and inter-class homogeneity. To overcome these challenges, this paper introduces AerialFormer, a hybrid model that strategically combines the strengths of Transformers and Convolutional Neural Networks (CNNs). AerialFormer features a CNN Stem module integrated to preserve low-level and high-resolution features, enhancing the model’s capability to process details of aerial imagery. The proposed AerialFormer is designed with a hierarchical structure, in which a Transformer encoder generates multi-scale features and a multi-dilated CNN (MDC) decoder aggregates the information from the multi-scale inputs. As a result, information is taken into account in both local and global contexts, so that powerful representations and high-resolution segmentation can be achieved. The proposed AerialFormer was benchmarked on three benchmark datasets, including iSAID, LoveDA, and Potsdam. Comprehensive experiments and extensive ablation studies show that the proposed AerialFormer remarkably outperforms state-of-the-art methods.

A Hybrid Scene Text Script Identification Network for Regional Indian Languages

In this work, we introduce WAFFNet, an attention-centric feature fusion architecture tailored for word-level multi-lingual scene text script identification. Motivated by the limitations of traditional approaches that rely exclusively on feature-based methods or deep learning strategies, our approach amalgamates statistical and deep features to bridge the gap. At the core of WAFFNet, we utilized the merits of Local Binary Pattern—a prominent descriptor capturing low-level texture features with high-dimensional, semantically-rich convolutional features. This fusion is judiciously augmented by a spatial attention mechanism, ensuring targeted emphasis on semantically critical regions of the input image. To address the class imbalance problem in multi-class classification scenarios, we employed a weighted objective function. This not only regularizes the learning process but also addresses the class imbalance problem. The architectural integrity of WAFFNet is preserved through an end-to-end training paradigm, leveraging transfer learning to expedite convergence and optimize performance metrics. Considering the under-representation of regional Indian languages in current datasets, we meticulously curated IIITG-STLI2023, a comprehensive dataset encapsulating English alongside six under-represented Indian languages: Hindi, Kannada, Malayalam, Telugu, Bengali, and Manipuri. Rigorous evaluation of the IIITG-STLI2023, as well as the established MLe2e and SIW-13 datasets, underscores WAFFNet’s supremacy over both traditional feature-engineering approaches as well as state-of-the-art deep learning frameworks. Thus, the proposed WAFFNet framework offers a robust and effective solution for language identification in scene text images.

Deep Learning-Based Defects Detection in Keyhole TIG Welding with Enhanced Vision.

Keyhole tungsten inert gas (keyhole TIG) welding is renowned for its advanced efficiency, necessitating a real-time defect detection method that integrates deep learning and enhanced vision techniques. This study employs a multi-layer deep neural network trained on an extensive welding image dataset. Neural networks can capture complex nonlinear relationships through multi-layer transformations without manual feature selection. Conversely, the nonlinear modeling ability of support vector machines (SVM) is limited by manually selected kernel functions and parameters, resulting in poor performance for recognizing burn-through and good welds images. SVMs handle only lower-level features such as porosity and excel only in detecting simple edges and shapes. However, neural networks excel in processing deep feature maps of "molten pools" and can encode deep defects that are often confused in keyhole TIG. Applying a four-class classification task to weld pool images, the neural network adeptly distinguishes various weld states, including good welds, burn-through, partial penetration, and undercut. Experimental results demonstrate high accuracy and real-time performance. A comprehensive dataset, prepared through meticulous preprocessing and augmentation, ensures reliable results. This method provides an effective solution for quality control and defect prevention in keyhole TIG welding process.

Lightweight Water Surface Object Detection Network for Unmanned Surface Vehicles

The detection algorithms for water surface objects considerably assist unmanned surface vehicles in rapidly perceiving their surrounding environment, providing essential environmental information and evaluating object attributes. This study proposes a lightweight water surface target detection algorithm called YOLO-WSD (water surface detection), based on YOLOv8n, to address the need for real-time, high-precision, and lightweight target detection algorithms that can adapt to the rapid changes in the surrounding environment during specific tasks. Initially, we designed the C2F-E module, enriched in gradient flow compared to the conventional C2F module, enabling the backbone network to extract richer multi-level features while maintaining lightness. Additionally, this study redesigns the feature fusion network structure by introducing low-level features and achieving multi-level fusion to enhance the network’s capability of integrating multiple levels. Meanwhile, it investigates the impact of channel number differences in the Concat module fusion on model performance, thereby optimizing the neural network structure. Lastly, it introduces the WIOU localization loss function to bolster model robustness. Experiments demonstrated that YOLO-WSD achieves a 4.6% and 3.4% improvement in mAP0.5 on the water surface object detection dataset and Seaship public dataset, respectively, with recall rates improving by 5.4% and 8.5% relative to the baseline YOLOv8n model. The model’s parameter size is 3.3 M. YOLO-WSD exhibits superior performance compared to other mainstream lightweight algorithms.

Joint pyramidal perceptual attention and hierarchical consistency constraint for gaze estimation

Eye gaze provides valuable cues about human intent, making gaze estimation a hot topic. Extracting multi-scale information has recently proven effective for gaze estimation in complex scenarios. However, existing methods for estimating gaze based on multi-scale features tend to focus only on information from single-level feature maps. Furthermore, information across different scales may also lack relevance. To address these issues, we propose a novel joint pyramidal perceptual attention and hierarchical consistency constraint (PaCo) for gaze estimation. The proposed PaCo consists of two main components: pyramidal perceptual attention module (PPAM) and hierarchical consistency constraint (HCC). Specifically, PPAM first extracts multi-scale spatial features using a pyramid structure, and then aggregates information from coarse granularity to fine granularity. In this way, PPAM enables the model to simultaneously focus on both the eye region and facial region at multiple scales. Then, HCC makes constrains consistency on low-level and high-level features, aiming to enhance the gaze semantic consistency between different feature levels. With the combination of PPAM and HCC, PaCo can learn more discriminative features in complex situations. Extensive experimental results show that PaCo achieves significant performance improvements on challenging datasets such as Gaze360, MPIIFaceGaze, and RT-GENE,reducing errors to 10.27°, 3.23°, 6.46°, respectively.

Contextual coherence increases perceived numerosity independent of semantic content.

Number perception emerges from multiple stages of visual processing. Understanding how systematic biases in number perception occur within a hierarchy of increasingly complex feature representations helps uncover the multistage processing underlying our visual number sense. Recent work demonstrated that reducing coherence of low-level visual attributes, such as color and orientation, systematically reduces perceived number. Here, we ask when in the visual processing hierarchy coherence affects numerosity perception and specifically whether the coherence effect is exclusive to low-level visual features or instead whether it can be driven by contextual or semantic relationships. We tested adults in an ordinal numerical comparison task with contextual coherence mathematically manipulated using a statistical model of visual object co-occurrence. Across several experiments, we found that arrays with high contextual coherence were perceived as numerically larger than arrays with low contextual coherence. This contextual coherence effect was not attenuated even when we reduced objects to texforms (unrecognizable images that preserve midlevel visual features) or removed semantic content from the images through box scrambling and diffeomorphic warping. Together, these results suggest that visual coherence derived from natural statistics of object co-occurrence systematically alters perceived numerosity at low-level visual processing, even before later stages at which items can be explicitly categorized and identified. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

AI-IMAGE REPRESENTATION AND LINEAR REPRENDER RENDERING

Image representation and rendering have become critical in numerous applications such as virtual reality, medical imaging, and computer graphics. Traditional rendering techniques often face challenges in efficiently handling complex scenes and achieving photorealistic results while maintaining low computational costs. The problem lies in the high-dimensional nature of image data, leading to slow processing times and reduced scalability. This research presents an AI-enhanced technique called Linear RepRender, which leverages deep learning to transform high-dimensional image representations into simplified linear forms for faster rendering. The proposed method employs a combination of convolutional neural networks (CNNs) and linear regression models to reduce image complexity. Specifically, the CNN extracts low-level and high-level features from the image, while the linear regression step approximates the scene’s core visual elements. This hybrid approach significantly improves rendering speed without sacrificing image quality. Furthermore, the method incorporates a loss function optimized for minimizing discrepancies between the rendered and ground truth images. Experimental results demonstrate that Linear RepRender outperforms traditional rendering algorithms, such as ray tracing and rasterization, in terms of computational efficiency and visual accuracy. On a dataset of complex 3D scenes, the proposed method achieved a 35% reduction in rendering time and a 22% improvement in peak signal-to-noise ratio (PSNR) compared to state-of-the-art methods. Additionally, Linear RepRender was able to handle up to 1.5 million polygons per scene with minimal visual artifacts, making it suitable for real-time applications.

BRAIN TUMOR CLASSIFICATION USING INCEPTIONRESNET-V2 AND TRANSFER LEARNING APPROACH

Brain, a highly intricate organ within the central nervous system, plays a fundamental role in information processing, cognition, motor control, and consciousness. Brain tumors pose severe threats to brain function and overall human well-being. Timely detection of these tumors is imperative for life-saving interventions. A dataset comprising four categories: no tumors, meningioma tumors, glioma tumors, and pituitary tumors was regarded in this research. The employed of the InceptionResNet-V2 architecture combined with Transfer Learning and data augmentation proposed to obtain optimal results on brain tumor classification types. Transfer learning act as fine tuning, enabling the model to acquire fundamental low-level image features from a comprehensive dataset. It then leverages higher-level features to become more tailored to the specific training data. This method is employed to improve the model's adaptability to the training data. The InceptionResNet-V2 architecture utilized in the evaluation using test data, in Scenario 1, achieved 94.18% accuracy. Scenario 2, which combined augmentation with InceptionResNetV2, showed an improvement in accuracy to 95.10%. Furthermore, in Scenario 3, the combination of InceptionResNetV2 with Transfer Learning and augmentation resulted in an impressive accuracy of 96.63%, demonstrating its effectiveness in brain tumor classification. Transfer learning aligns the model by acquiring low-level image features and utilizing higher-level features to improve adaptability to the training data.

Subject review: White balance techniques of digital image

Due to un control environment during creating of the digital image make the resulting image have un real color cast, for that a white balance algorithm has been found to correct the unreal color cast create by non-canonical Lighting sources, In order to achieve satisfactory color of the digital image, a white balance required to adjust the image color either automatically or manually. for that researchers found an automatic and manual white balance algorithms, and even the Automatic white balance algorithm depending on analysis either low level features or high Level features, explaining An examples for Automatic white balance algorithm ( Gray World Algorithm and The white patch Algorithm ) and clarify each Algorithm suits different kind of Image.

Foreign Object Detection Network for Transmission Lines from Unmanned Aerial Vehicle Images

Foreign objects such as balloons and nests often lead to widespread power outages by coming into contact with transmission lines. The manual detection of these is labor-intensive work. Automatic foreign object detection on transmission lines is a crucial task for power safety and is becoming the mainstream method, but the lack of datasets is a restriction. In this paper, we propose an advanced model termed YOLOv8 Network with Bidirectional Feature Pyramid Network (YOLOv8_BiFPN) to detect foreign objects on power transmission lines. Firstly, we add a weighted cross-scale connection structure to the detection head of the YOLOv8 network. The structure is bidirectional. It provides interaction between low-level and high-level features, and allows information to spread across feature maps of different scales. Secondly, in comparison to the traditional concatenation and shortcut operations, our method integrates information between different scale features through weighted settings. Moreover, we created a dataset of Foreign Object detection on Transmission Lines from a Drone-view (FOTL_Drone). It consists of 1495 annotated images with six types of foreign object. To our knowledge, FOTL_Drone stands out as the most comprehensive dataset in the field of foreign object detection on transmission lines, which encompasses a wide array of geographic features and diverse types of foreign object. Experimental results showcase that YOLOv8_BiFPN achieves an average precision of 90.2% and an mAP@.50 of 0.896 across various categories of foreign objects, surpassing other models.

FSPDD: A double-branch attention guided network for few-shot PCB defect detection

AbstractDuring the production of printed circuit board (PCB), there will be defects due to inappropriate operations, which will affect the use of electronic products. Majority defect detection methods cost a large number of annotated samples to train detection models. However, PCB defect samples are difficult to collect. Moreover, existing few-shot object detection methods tend to extracting low-level features from support and query images via the shared backbone such as ResNet-50. However, it is not sufficient to obtain fine-grained prior guidance. To address the above issues, we propose a few-shot PCB defect detection model with double-branch attention. Specifically, the joint attention enhancement (JAE) module is proposed to fully mine effective information of query PCB images in multiple dimensions to enhance the representation of latent defects. Then, the multi-scale guidance (MSG) module is proposed to integrate prior knowledge within support PCB images into vectors to reweight query PCB images. Experiments on the PCB defect dataset demonstrate that AP of FSPDD outperforms state-of-the-art methods under different shot settings (k=1,2,3,5,10,30) and our proposed FSPDD has a good generalization ability, in which AP reachs 0.273 when $$k=30$$ k = 30 and is 5.28% higher than SOTA methods.

Size adaptation: Do you know it when you see it?

The visual system adapts to a wide range of visual features, from lower-level features like color and motion to higher-level features like causality and, perhaps, number. According to some, adaptation is a strictly perceptual phenomenon, such that the presence of adaptation licenses the claim that a feature is truly perceptual in nature. Given the theoretical importance of claims about adaptation, then, it is important to understand exactly when the visual system does and does not exhibit adaptation. Here, we take as a case study one specific kind of adaptation: visual adaptation to size. Supported by evidence from four experiments, we argue that, despite robust effects of size adaptation in the lab, (1) size adaptation effects are phenomenologically underwhelming (in some cases, hardly appreciable at all), (2) some effects of size adaptation appear contradictory, and difficult to explain given current theories of size adaptation, and (3) prior studies on size adaptation may have failed to isolate size as the adapted dimension. Ultimately, we argue that while there is evidence to license the claim that size adaptation is genuine, size adaptation is a puzzling and poorly understood phenomenon.

Pest-ConFormer: A hybrid CNN-Transformer architecture for large-scale multi-class crop pest recognition

Crop pests are acknowledged as the major factors in reducing the yield and quality of agricultural production worldwide. It is an urgent necessity to recognize crop pests accurately to protect the crop in the early stage to reduce the loss for the agricultural economy. Due to the ecological characteristics of the crop pests and the complex background in fields, the crop pests show high inter-class similarity and significant intra-class variation in external morphology appearance, which makes current recognition methods suffer from low classification accuracy and poor generalization ability in complex natural environment recognition tasks. To tackle this problem, a hybrid convolutional neural network and transformer-based model, namely Pest-ConFormer, featured with multi-scale weakly supervised feature selection mechanisms is proposed, which has shown excellent multi-scale discriminative feature extraction in fine-grained visual classification (FGVC) tasks. This method employs a hybrid convolution-transformer encoder architecture pre-training in a self-supervised masked autoencoder manner as a backbone to learn pests’ highly discriminative features across various scales. Next, a dual-path feature aggregation structure with a top-down FPN-like feature pathway and a bottom-up PANet-like feature pathway based on attention mechanisms is designed to learn high-level global context information and low-level local detailed feature representation. Thirdly, a fine-grained classification module using weakly supervised learning is introduced to select the discriminative feature points in different pyramidal levels. Then, these feature points are fed into a graph convolutional network to accomplish classification. Several experiments are conducted on the large-scale multi-class IP102 benchmark dataset, and the proposed method achieves an accuracy of 77.81 % regarding crop pest recognition. The experimental results indicate that our approach outperforms other state-of-the-art methods by nearly 2 percent points, demonstrating that the proposed hybrid architecture with dual-path feature aggregation and fine-grained classification modules can be more effective in the crop pest recognition field than CNN-based methods and can be deployed in the practical natural environment. The source code will be available at https://github.com/mwfang/pestconformer.

Generative innovations for paleography: enhancing character image synthesis through unconditional single image models

Data scarcity in paleographic image datasets poses a significant challenge to researchers and scholars in the field. Unlike modern printed texts, historical manuscripts and documents are often scarce and fragile, making them difficult to digitize and create comprehensive datasets. Recently many innovations have been arrived on single image generative models for natural images but none of them are focused on paleographic character images and other handwritten datasets. In paleographic images like stone inscription characters, maintaining exact shape and structure of character is important unlike natural images. In this paper we propose an unconditional single image generative model, CharGAN for isolated paleographic character images. In the proposed system, augmented images are generated from a single image using generative adversarial networks, while maintaining their structure. Specifically, an external augmentation inducer is used to create higher-level augmentations in the generated images. In addition, the input to the generator is replaced with dynamic sampling from a Gaussian mixture model to make changes to the low-level features. From our experimental results, we infer that these two enhancements make single-image generative models suitable not only for natural images, but also for paleographic character images and other handwritten character datasets, the AHCD dataset, and EMNIST, where the global structure is important. Both the qualitative and quantitative results show that our approach is effective and superior in single-image generative tasks, particularly in isolated character image generation.

Trans-DCN: A High-Efficiency and Adaptive Deep Network for Bridge Cable Surface Defect Segmentation

Cables are vital load-bearing components of cable-stayed bridges. Surface defects can lead to internal corrosion and fracturing, significantly impacting the stability of the bridge structure. The detection of surface defects from bridge cable images faces numerous challenges, including shadow disturbances due to uneven lighting and difficulties in addressing multiscale defect features. To address these challenges, this paper proposes a novel and cost-effective deep learning segmentation network, named Trans-DCN, to detect defects in the surface of the bridge cable. The network leverages an efficient Transformer-based encoder and integrates multiscale features to overcome the limitations associated with local feature inadequacy. The decoder implements an atrous Deformable Convolution (DCN) pyramid and dynamically fuses low-level feature information to perceive the complex distribution of defects. The effectiveness of Trans-DCN is evaluated by comparing it with state-of-the-art segmentation baseline models using a dataset comprising cable bridge defect images. Experimental results demonstrate that our network outperforms the state-of-the-art network SegFormer, achieving a 27.1% reduction in GFLOPs, a 1.2% increase in mean Intersection over Union, and a 1.5% increase in the F1 score. Ablation experiments confirmed the effectiveness of each module within our network, further substantiating the significant validity and advantages of Trans-DCN in the task of bridge cable defect segmentation. The network proposed in this paper provides an effective solution for downstream cable bridge image analysis.

Symbolic Task Inference in Deep Reinforcement Learning

This paper proposes DeepSynth, a method for effective training of deep reinforcement learning agents when the reward is sparse or non-Markovian, but at the same time progress towards the reward requires achieving an unknown sequence of high-level objectives. Our method employs a novel algorithm for synthesis of compact finite state automata to uncover this sequential structure automatically. We synthesise a human-interpretable automaton from trace data collected by exploring the environment. The state space of the environment is then enriched with the synthesised automaton, so that the generation of a control policy by deep reinforcement learning is guided by the discovered structure encoded in the automaton. The proposed approach is able to cope with both high-dimensional, low-level features and unknown sparse or non-Markovian rewards. We have evaluated DeepSynth’s performance in a set of experiments that includes the Atari game Montezuma’s Revenge, known to be challenging. Compared to approaches that rely solely on deep reinforcement learning, we obtain a reduction of two orders of magnitude in the iterations required for policy synthesis, and a significant improvement in scalability.

Dual-stage feedback network for lightweight color image compression artifact reduction

Lossy image coding techniques usually result in various undesirable compression artifacts. Recently, deep convolutional neural networks have seen encouraging advances in compression artifact reduction. However, most of them focus on the restoration of the luma channel without considering the chroma components. Besides, most deep convolutional neural networks are hard to deploy in practical applications because of their high model complexity. In this article, we propose a dual-stage feedback network (DSFN) for lightweight color image compression artifact reduction. Specifically, we propose a novel curriculum learning strategy to drive a DSFN to reduce color image compression artifacts in a luma-to-RGB manner. In the first stage, the DSFN is dedicated to reconstructing the luma channel, whose high-level features containing rich structural information are then rerouted to the second stage by a feedback connection to guide the RGB image restoration. Furthermore, we present a novel enhanced feedback block for efficient high-level feature extraction, in which an adaptive iterative self-refinement module is carefully designed to refine the low-level features progressively, and an enhanced separable convolution is advanced to exploit multiscale image information fully. Extensive experiments show the notable advantage of our DSFN over several state-of-the-art methods in both quantitative indices and visual effects with lower model complexity.

UNet-like network fused swin transformer and CNN for semantic image synthesis

Semantic image synthesis approaches has been dominated by the modelling of Convolutional Neural Networks (CNN). Due to the limitations of local perception, their performance improvement seems to have plateaued in recent years. To tackle this issue, we propose the SC-UNet model, which is a UNet-like network fused Swin Transformer and CNN for semantic image synthesis. Photorealistic image synthesis conditional on the given semantic layout depends on the high-level semantics and the low-level positions. To improve the synthesis performance, we design a novel conditional residual fusion module for the model decoder to efficiently fuse the hierarchical feature maps extracted at different scales. Moreover, this module combines the opposition-based learning mechanism and the weight assignment mechanism for enhancing and attending the semantic information. Compared to pure CNN-based models, our SC-UNet combines the local and global perceptions to better extract high- and low-level features and better fuse multi-scale features. We have conducted an extensive amount of comparison experiments, both in quantitative and qualitative terms, to validate the effectiveness of our proposed SC-UNet model for semantic image synthesis. The outcomes illustrate that SC-UNet distinctively outperforms the state-of-the-art model on three benchmark datasets (Citysacpes, ADE20K, and COCO-Stuff) including numerous real-scene images.