Low-level Image Features Research Articles

Priors for natural image statistics inform confidence in perceptual decisions.

Decision confidence plays a critical role in humans' ability to make adaptive decisions in a noisy perceptual world. Despite its importance, there is currently little consensus about the computations underlying confidence judgements in perceptual decisions. To better understand these mechanisms, we addressed the extent to which confidence is informed by a naturalistic prior distribution. Contrary to previous research, we did not require participants to internalise parameters of an arbitrary prior distribution. We instead used a novel psychophysical paradigm leveraging probability distributions of low-level image features in natural scenes, which are well-known to influence perception. Participants reported the subjective upright of naturalistic image patches, targets, and then reported their confidence in their orientation responses. We used computational modelling to relate the statistics of the low-level features in the targets to the average distribution of these features across many naturalistic images, a prior. Our results showed that participants' perceptual and importantly, their confidence judgments aligned with an internalised prior for image statistics. Overall, our study highlights the importance of naturalistic task designs that capitalise on existing, long-term priors to further understand the computational basis of confidence.

CPDM: Content-preserving diffusion model for underwater image enhancement

Underwater image enhancement (UIE) is challenging since image degradation in aquatic environments is complicated and changing over time. Existing mainstream methods rely on either physical-model or data-driven, suffering from performance bottlenecks due to changes in imaging conditions or training instability. In this article, we attempt to adapt the diffusion model to the UIE task and propose a Content-Preserving Diffusion Model (CPDM) to address the above challenges. CPDM first leverages a diffusion model as its fundamental model for stable training and then designs a content-preserving framework to deal with changes in imaging conditions. Specifically, we construct a conditional input module by adopting both the raw image and the difference between the raw and noisy images as the input at each time step of the diffusion process, which can enhance the model’s adaptability by considering the changes involving the raw images in underwater environments. To preserve the essential content of the raw images, we construct a content compensation module for content-aware training by extracting low-level image features of the raw images as compensation for each down block. We conducted tests on the LSUI, UIEB, and EUVP datasets, and the results show that CPDM outperforms state-of-the-art methods in both subjective and objective metrics, achieving the best overall performance. The GitHub link for the code is https://github.com/GZHU-DVL/CPDM.

Beauty ideals modulate recognition of filtered images

Social media has affected how we relate to our body image. Digital makeovers have both reinforced existing beauty ideals and created new ones. This project investigated whether young adults’ recognition of image filters was associated by the beliefs of beauty ideals and gender. Participants completed a visual detection task (forced choice paradigm) where contrast filter correction was assessed for images of female and male bodies that were thin, average, or curvaceous/muscular. Results showed that people respond faster, and accuracy is higher when filters are applied to bodies that represent the historical beauty ideals: thin female bodies and muscular male bodies. These findings suggest that the perception of low-level image features is associated to fit the beliefs about beauty.

Digital Architectural Form Generation Through Pixel System-Driven Image Feature Information

This study analyzes valuable information (image feature information) from pixel-based images to systematize the form generation process utilizing this information. Information in architecture is mainly used as an analytical tool for functional design optimization. However, the Fourth Industrial Revolution and advances in information and communication technology have positioned information as a core value-creating tool. Consequently, there is a growing need to develop methodologies for effectively applying information to architectural design generation. This study combines digital technology with images, which directly influence architectural design, transforming them into numerical information. By systematizing the three-dimensional form generation process using low-level image feature information, it analyzes the relationship between the characteristics of valuable information and the generated forms. This approach helps identify the interrelationships between the characteristics of valuable information and the generated forms, the complementarity needed to mitigate the limitations of valuable information, and the potential for architectural applications of the three-dimensional form generation methodology utilizing image information. This approach reduces labor and enhances architectural aesthetics by instantly generating diverse early design alternatives. Such research can provide a more logical basis for avoiding intuitive judgments during the design decision-making process. It can also lay the groundwork for information-centric design systems in architecture.

Gaze behaviors during free viewing revealed differences in visual salience processing across four major psychiatric disorders: a mega-analysis study of 1012 individuals.

Aberrant salience processing has been proposed as a pathophysiological mechanism underlying psychiatric symptoms in patients with schizophrenia. The gaze trajectories of individuals with schizophrenia have been reported to be abnormal when viewing an image, suggesting anomalous visual salience as one possible pathophysiological mechanism associated with psychiatric diseases. This study was designed to determine whether visual salience is affected in individuals with schizophrenia, and whether this abnormality is unique to patients with schizophrenia. We examined the gaze behaviors of 1012 participants recruited from seven institutes (550 healthy individuals and 238, 41, 50 and 133 individuals with schizophrenia, bipolar disorder, major depressive disorder and autism spectrum disorder, respectively) when they looked at stationary images as they liked, i.e., free-viewing condition. We used an established computational model of salience maps derived from low-level visual features to measure the degree to which the gaze trajectories of individuals were guided by visual salience. The analysis revealed that the saliency at the gaze of individuals with schizophrenia were higher than healthy individuals, suggesting that patients' gazes were guided more by low-level image salience. Among the low-level image features, orientation salience was most affected. Furthermore, a general linear model analysis of the data for the four psychiatric disorders revealed a significant effect of disease. This abnormal salience processing depended on the disease and was strongest in patients with schizophrenia, followed by patients with bipolar disorder, major depressive disorder, and autism spectrum disorder, suggesting a link between abnormalities in salience processing and strength/frequency for psychosis of these disorders.

Snow Cover Extraction from Landsat 8 OLI Based on Deep Learning with Cross-Scale Edge-Aware and Attention Mechanism

Snow cover distribution is of great significance for climate change and water resource management. Current deep learning-based methods for extracting snow cover from remote sensing images face challenges such as insufficient local detail awareness and inadequate utilization of global semantic information. In this study, a snow cover extraction algorithm integrating cross-scale edge perception and an attention mechanism on the U-net model architecture is proposed. The cross-scale edge perception module replaces the original jump connection of U-net, enhances the low-level image features by introducing edge detection on the shallow feature scale, and enhances the detail perception via branch separation and fusion features on the deep feature scale. Meanwhile, parallel channel and spatial attention mechanisms are introduced in the model encoding stage to adaptively enhance the model’s attention to key features and improve the efficiency of utilizing global semantic information. The method was evaluated on the publicly available CSWV_S6 optical remote sensing dataset, and the accuracy of 98.14% indicates that the method has significant advantages over existing methods. Snow extraction from Landsat 8 OLI images of the upper reaches of the Irtysh River was achieved with satisfactory accuracy rates of 95.57% (using two, three, and four bands) and 96.65% (using two, three, four, and six bands), indicating its strong potential for automated snow cover extraction over larger areas.

Context-aware Optimization for Bandwidth-Efficient Image Analytics Offloading

Convolutional Neural Networks (CNN) have given rise to numerous visual analytics applications at the edge of the Internet. The image is typically captured by cameras and then live-streamed to edge servers for analytics due to the prohibitive cost of running CNN on computation-constrained end devices. A critical component to ensure low-latency and accurate visual analytics offloading over low bandwidth networks is image compression which minimizes the amount of visual data to offload and maximizes the decoding quality of salient pixels for analytics. Despite the wide adoption, JPEG standards and traditional image compression techniques do not address the accuracy of analytics tasks, leading to ineffective compression for visual analytics offloading. Although recent machine-centric image compression techniques leverage sophisticated neural network models or hardware architecture to support the accuracy-bandwidth tradeoff, they introduce excessive latency in the visual analytics offloading pipeline. This article presents CICO, a Context-aware Image Compression Optimization framework to achieve low-bandwidth and low-latency visual analytics offloading. CICO contextualizes image compression for offloading by employing easily-computable low-level image features to understand the importance of different image regions for a visual analytics task. Accordingly, CICO can optimize the tradeoff between compression size and analytics accuracy. Extensive real-world experiments demonstrate that CICO reduces the bandwidth consumption of existing compression methods by up to 40% under comparable analytics accuracy. Regarding the low-latency support, CICO achieves up to a 2× speedup over state-of-the-art compression techniques.

BRAIN TUMOR CLASSIFICATION USING INCEPTIONRESNET-V2 AND TRANSFER LEARNING APPROACH

Brain, a highly intricate organ within the central nervous system, plays a fundamental role in information processing, cognition, motor control, and consciousness. Brain tumors pose severe threats to brain function and overall human well-being. Timely detection of these tumors is imperative for life-saving interventions. A dataset comprising four categories: no tumors, meningioma tumors, glioma tumors, and pituitary tumors was regarded in this research. The employed of the InceptionResNet-V2 architecture combined with Transfer Learning and data augmentation proposed to obtain optimal results on brain tumor classification types. Transfer learning act as fine tuning, enabling the model to acquire fundamental low-level image features from a comprehensive dataset. It then leverages higher-level features to become more tailored to the specific training data. This method is employed to improve the model's adaptability to the training data. The InceptionResNet-V2 architecture utilized in the evaluation using test data, in Scenario 1, achieved 94.18% accuracy. Scenario 2, which combined augmentation with InceptionResNetV2, showed an improvement in accuracy to 95.10%. Furthermore, in Scenario 3, the combination of InceptionResNetV2 with Transfer Learning and augmentation resulted in an impressive accuracy of 96.63%, demonstrating its effectiveness in brain tumor classification. Transfer learning aligns the model by acquiring low-level image features and utilizing higher-level features to improve adaptability to the training data.

Expansive Receptive Field and Local Feature Extraction Network: Advancing Multiscale Feature Fusion for Breast Fibroadenoma Segmentation in Sonography.

Fibroadenoma is a common benign breast disease that affects women of all ages. Early diagnosis can greatly improve the treatment outcomes and reduce the associated pain. Computer-aided diagnosis (CAD) has great potential to improve diagnosis accuracy and efficiency. However, its application in sonography is limited. A network that utilizes expansive receptive fields and local information learning was proposed for the accurate segmentation of breast fibroadenomas in sonography. The architecture comprises the Hierarchical Attentive Fusion module, which conducts local information learning through channel-wise and pixel-wise perspectives, and the Residual Large-Kernel module, which utilizes multiscale large kernel convolution for global information learning. Additionally, multiscale feature fusion in both modules was included to enhance the stability of our network. Finally, an energy function and a data augmentation method were incorporated to fine-tune low-level features of medical images and improve data enhancement. The performance of our model is evaluated using both our local clinical dataset and a public dataset. Mean pixel accuracy (MPA) of 93.93% and 86.06% and mean intersection over union (MIOU) of 88.16% and 73.19% were achieved on the clinical and public datasets, respectively. They are significantly improved over state-of-the-art methods such as SegFormer (89.75% and 78.45% in MPA and 83.26% and 71.85% in MIOU, respectively). The proposed feature extraction strategy, combining local pixel-wise learning with an expansive receptive field for global information perception, demonstrates excellent feature learning capabilities. Due to this powerful and unique local-global feature extraction capability, our deep network achieves superior segmentation of breast fibroadenoma in sonography, which may be valuable in early diagnosis.

Survey of Techniques and Applications in Object Recognition

Abstract: Object detection, which is closely related to video analysis and image comprehension, has received attention in years from researchers. Traditional approaches to object detection rely on crafted features and shallow trainable architectures. However, these methods face challenges. Often reach a plateau, in performance improvement. To overcome these limitations complex ensembles are often created by combining low-level image features with high-level context from object detectors and scene classifiers. The emergence of learning has introduced powerful tools that can learn semantics high level and deeper features. These tools address the limitations of architectures by introducing network structures training strategies and optimization functions. This paper provides a review of learning-based frameworks for object detection. The review starts with an overview of the history of learning with a particular emphasis on the Convolutional Neural Network (CNN) as an exemplary tool. It then delves into generic object detection architectures exploring modifications and effective strategies for improving detection performance. Recognizing the characteristics of detection tasks, the paper also briefly surveys several tasks such, as salient object detection, face detection, crowd analysis, and pedestrian detection.

Contribution of low-level image statistics to EEG decoding of semantic content in multivariate and univariate models with feature optimization

Spatio-temporal patterns of evoked brain activity contain information that can be used to decode and categorize the semantic content of visual stimuli. However, this procedure can be biased by low-level image features independently of the semantic content present in the stimuli, prompting the need to understand the robustness of different models regarding these confounding factors. In this study, we trained machine learning models to distinguish between concepts included in the publicly available THINGS-EEG dataset using electroencephalography (EEG) data acquired during a rapid serial visual presentation paradigm. We investigated the contribution of low-level image features to decoding accuracy in a multivariate model, utilizing broadband data from all EEG channels. Additionally, we explored a univariate model obtained through data-driven feature selection applied to the spatial and frequency domains. While the univariate models exhibited better decoding accuracy, their predictions were less robust to the confounding effect of low-level image statistics. Notably, some of the models maintained their accuracy even after random replacement of the training dataset with semantically unrelated samples that presented similar low-level content. In conclusion, our findings suggest that model optimization impacts sensitivity to confounding factors, regardless of the resulting classification performance. Therefore, the choice of EEG features for semantic decoding should ideally be informed by criteria beyond classifier performance, such as the neurobiological mechanisms under study.

PortLaneNet: A Scene-Aware Model for Robust Lane Detection in Container Terminal Environments

In this paper, we introduce PortLaneNet, an optimized lane detection model specifically designed for the unique challenges of enclosed container terminal environments. Unlike conventional lane detection scenarios, this model addresses complexities such as intricate ground markings, tire crane lane lines, and various types of regional lines that significantly complicate detection tasks. Our approach includes the novel Scene Prior Perception Module, which leverages pre-training to provide essential prior information for more accurate lane detection. This module capitalizes on the enclosed nature of container terminals, where images from similar area scenes offer effective prior knowledge to enhance detection accuracy. Additionally, our model significantly improves understanding by integrating both high- and low-level image features through attention mechanisms, focusing on the critical components of lane detection. Through rigorous experimentation, PortLaneNet has demonstrated superior performance in port environments, outperforming traditional lane detection methods. The results confirm the effectiveness and superiority of our model in addressing the complex challenges of lane detection in such specific settings. Our work provides a valuable reference for solving lane detection issues in specialized environments and proposes new ideas and directions for future research.

EAD-Net: Efficiently Asymmetric Network for Semantic Labeling of High-Resolution Remote Sensing Images with Dynamic Routing Mechanism

Semantic labeling of high-resolution remote sensing images (HRRSIs) holds a significant position in the remote sensing domain. Although numerous deep-learning-based segmentation models have enhanced segmentation precision, their complexity leads to a significant increase in parameters and computational requirements. While ensuring segmentation accuracy, it is also crucial to improve segmentation speed. To address this issue, we propose an efficient asymmetric deep learning network for HRRSIs, referred to as EAD-Net. First, EAD-Net employs ResNet50 as the backbone without pooling, instead of the RepVGG block, to extract rich semantic features while reducing model complexity. Second, a dynamic routing module is proposed in EAD-Net to adjust routing based on the pixel occupancy of small-scale objects. Concurrently, a channel attention mechanism is used to preserve their features even with minimal occupancy. Third, a novel asymmetric decoder is introduced, which uses convolutional operations while discarding skip connections. This not only effectively reduces redundant features but also allows using low-level image features to enhance EAD-Net’s performance. Extensive experimental results on the ISPRS 2D semantic labeling challenge benchmark demonstrate that EAD-Net achieves state-of-the-art (SOTA) accuracy performance while reducing model complexity and inference time, while the mean Intersection over Union (mIoU) score reaching 87.38% and 93.10% in the Vaihingen and Potsdam datasets, respectively.

MKTN: Adversarial-Based Multifarious Knowledge Transfer Network from Complementary Teachers

With the demands for light deep networks models in various front-end devices, network compression has attracted increasing interest for reducing model sizes yet without sacrificing much model accuracy. This paper presents a multifarious knowledge transfer network (MKTN) that aims to produce a compact yet powerful student network from two complementary teacher networks. Instead of learning homogeneous features, the idea is to pre-train one teacher to capture generative and low-level image features under a reconstruction objective, and another teacher to capture discriminative and task-specific features under the same objective as the student network. During knowledge transfer, the student learns multifarious and complementary knowledge from the two teacher networks under the guidance of the proposed adversarial loss and feature loss respectively. Experimental results indicate that the proposed training losses can effectively guide the student to learn spatial-level and pixel-level information as distilled from teacher networks. On the other hand, our study over a number of widely used datasets shows that transferring multifarious features from complementary teachers equipped with different types of knowledge helps to teach a compact yet powerful student effectively.

DNNAM: Image inpainting algorithm via deep neural networks and attention mechanism

Most image inpainting algorithms have problems such as fuzzy images, texture distortion and semantic inaccuracy, and the image inpainting effect is limited when processing photos with large missing sections and resolution levels. The paper proposes an effective image inpainting algorithm via partial multi-scale channel attention mechanism and deep neural networks to address the above phenomenon that existing image inpainting methods using deep learning modules have insufficient perception and representation capabilities for multi-scale features with high proportion of irregular defects. Initially, we used the Res-U-Net module as a generator. The U-Net-like backbone network topology can achieve the encoding and decoding stages of damaged images. Secondly, the residual network structure was built in the encoder and decoder to improve the ability of the proposed network to extract and display the features of the damaged images. Finally, the partial multi-scale channel attention module was inserted in the skip connection with the decoder to increase the efficiency of using the low-level features of the original images. The experimental results of the research can show that the proposed method outperforms state-of-the-art methods in terms of subjective visual perception and objective evaluation indicators on the CelebA, Places2 and Paris Street View datasets.

Broadband and large-aperture metasurface edge encoders for incoherent infrared radiation.

The prevalence of computer vision systems necessitates hardware-based approaches to relieve the high computational demand of deep neural networks in resource-limited applications. One solution would be to off-load low-level image feature extraction, such as edge detection, from the digital network to the analog imaging system. To that end, this work demonstrates incoherent, broadband, low-noise optical edge detection of real-world scenes by combining the wavefront shaping of a 24-mm aperture metasurface with a refractive lens. An inverse design approach is used to optimize the metasurface for Laplacian-based edge detection across the 7.5- to 13.5-μm LWIR imaging band, allowing for facile integration with uncooled microbolometer-based LWIR imagers to encode edge information. A polarization multiplexed approach leveraging a birefringent metasurface is also demonstrated as a single-aperture implementation. This work could be applied to improve computer vision capabilities of resource-constrained systems by leveraging optical preprocessing to alleviate the computational requirements for high-accuracy image segmentation and classification.

Patterns of saliency and semantic features distinguish gaze of expert and novice viewers of surveillance footage

When viewing the actions of others, we not only see patterns of body movements, but we also "see" the intentions and social relations of people. Experienced forensic examiners – Closed Circuit Television (CCTV) operators – have been shown to convey superior performance in identifying and predicting hostile intentions from surveillance footage than novices. However, it remains largely unknown what visual content CCTV operators actively attend to, and whether CCTV operators develop different strategies for active information seeking from what novices do. Here, we conducted computational analysis for the gaze-centered stimuli captured by experienced CCTV operators and novices' eye movements when viewing the same surveillance footage. Low-level image features were extracted by a visual saliency model, whereas object-level semantic features were extracted by a deep convolutional neural network (DCNN), AlexNet, from gaze-centered regions. We found that the looking behavior of CCTV operators differs from novices by actively attending to visual contents with different patterns of saliency and semantic features. Expertise in selectively utilizing informative features at different levels of visual hierarchy may play an important role in facilitating the efficient detection of social relationships between agents and the prediction of harmful intentions.

Machine learning and similar image-based techniques based on Nash game theory

The use of computer vision techniques to address the task of image retrieval is known as a Content-Based Image Retrieval (CBIR) system. It is a system designed to locate and retrieve the appropriate digital image from a large database by utilizing a query image. Over the last few years, machine learning algorithms have achieved impressive results in image retrieval tasks due to their ability to learn from large amounts of diverse data and improve their accuracy in image recognition and retrieval. Our team has developed a CBIR system that is reinforced by two machine learning algorithms and employs multiple clustering and low-level image feature extraction, such as color, shape, and texture, to formulate a Nash game. Consequently, we are faced with a multicriteria optimization problem. To solve this problem, we have formulated a three-player static Nash game, where each player utilizes a different strategy (color descriptor, Zernike descriptor, and SFTA descriptor) based on their objective function. The Nash equilibrium is defined as the membership classes of the query image.

Face spoofing detection using Heterogeneous Auto-Similarities of Characteristics

Facial recognition systems are widely used in various daily applications due to their high accuracy rate and ease of use. Unfortunately, their performance drops drastically when encountering spoofing attacks. In this paper, a novel pipeline for face spoofing detection based on the extraction of Heterogeneous Auto-Similarities of Characteristics (HASC) descriptor from the HSV and YCbCr color spaces is proposed. To detect print and replay spoofing attacks, the HASC descriptor encodes linear and non-linear dependencies between low-level dense features of the color face image using covariance and information-theoretic measures. The combination of covariance and information-theoretic measures makes HASC an efficient descriptor to explore how the features are interrelated in real and fake faces. The HASC texture descriptions are extracted separately from each color channel of the HSV and YCbCr color spaces, and the resulting feature vectors are concatenated into an extended feature vector to better discriminate between real and fake faces. Extensive experiments are conducted on publicly available databases namely, Replay-Attack, CASIA FASD, and MSU MFSD. The obtained results demonstrate that while being of low dimension, the HASC descriptor when fed into a binary Support Vector Machine (SVM) classifier, achieves competitive results compared to the state-of-the-art.

Adaptive learning point cloud and image diversity feature fusion network for 3D object detection

3D object detection is a critical task in the fields of virtual reality and autonomous driving. Given that each sensor has its own strengths and limitations, multi-sensor-based 3D object detection has gained popularity. However, most existing methods extract high-level image semantic features and fuse them with point cloud features, focusing solely on consistent information from both sensors while ignoring their complementary information. In this paper, we present a novel two-stage multi-sensor deep neural network, called the adaptive learning point cloud and image diversity feature fusion network (APIDFF-Net), for 3D object detection. Our approach employs the fine-grained image information to complement the point cloud information by combining low-level image features with high-level point cloud features. Specifically, we design a shallow image feature extraction module to learn fine-grained information from images, instead of relying on deep layer features with coarse-grained information. Furthermore, we design a diversity feature fusion (DFF) module that transforms low-level image features into point-wise image features and explores their complementary features through an attention mechanism, ensuring an effective combination of fine-grained image features and point cloud features. Experiments on the KITTI benchmark show that the proposed method outperforms state-of-the-art methods.