Low-level Image Features Research Articles

Gaze behaviors during free viewing revealed differences in visual salience processing across four major psychiatric disorders: a mega-analysis study of 1012 individuals.

Aberrant salience processing has been proposed as a pathophysiological mechanism underlying psychiatric symptoms in patients with schizophrenia. The gaze trajectories of individuals with schizophrenia have been reported to be abnormal when viewing an image, suggesting anomalous visual salience as one possible pathophysiological mechanism associated with psychiatric diseases. This study was designed to determine whether visual salience is affected in individuals with schizophrenia, and whether this abnormality is unique to patients with schizophrenia. We examined the gaze behaviors of 1012 participants recruited from seven institutes (550 healthy individuals and 238, 41, 50 and 133 individuals with schizophrenia, bipolar disorder, major depressive disorder and autism spectrum disorder, respectively) when they looked at stationary images as they liked, i.e., free-viewing condition. We used an established computational model of salience maps derived from low-level visual features to measure the degree to which the gaze trajectories of individuals were guided by visual salience. The analysis revealed that the saliency at the gaze of individuals with schizophrenia were higher than healthy individuals, suggesting that patients' gazes were guided more by low-level image salience. Among the low-level image features, orientation salience was most affected. Furthermore, a general linear model analysis of the data for the four psychiatric disorders revealed a significant effect of disease. This abnormal salience processing depended on the disease and was strongest in patients with schizophrenia, followed by patients with bipolar disorder, major depressive disorder, and autism spectrum disorder, suggesting a link between abnormalities in salience processing and strength/frequency for psychosis of these disorders.

Snow Cover Extraction from Landsat 8 OLI Based on Deep Learning with Cross-Scale Edge-Aware and Attention Mechanism

Snow cover distribution is of great significance for climate change and water resource management. Current deep learning-based methods for extracting snow cover from remote sensing images face challenges such as insufficient local detail awareness and inadequate utilization of global semantic information. In this study, a snow cover extraction algorithm integrating cross-scale edge perception and an attention mechanism on the U-net model architecture is proposed. The cross-scale edge perception module replaces the original jump connection of U-net, enhances the low-level image features by introducing edge detection on the shallow feature scale, and enhances the detail perception via branch separation and fusion features on the deep feature scale. Meanwhile, parallel channel and spatial attention mechanisms are introduced in the model encoding stage to adaptively enhance the model’s attention to key features and improve the efficiency of utilizing global semantic information. The method was evaluated on the publicly available CSWV_S6 optical remote sensing dataset, and the accuracy of 98.14% indicates that the method has significant advantages over existing methods. Snow extraction from Landsat 8 OLI images of the upper reaches of the Irtysh River was achieved with satisfactory accuracy rates of 95.57% (using two, three, and four bands) and 96.65% (using two, three, four, and six bands), indicating its strong potential for automated snow cover extraction over larger areas.

BRAIN TUMOR CLASSIFICATION USING INCEPTIONRESNET-V2 AND TRANSFER LEARNING APPROACH

Brain, a highly intricate organ within the central nervous system, plays a fundamental role in information processing, cognition, motor control, and consciousness. Brain tumors pose severe threats to brain function and overall human well-being. Timely detection of these tumors is imperative for life-saving interventions. A dataset comprising four categories: no tumors, meningioma tumors, glioma tumors, and pituitary tumors was regarded in this research. The employed of the InceptionResNet-V2 architecture combined with Transfer Learning and data augmentation proposed to obtain optimal results on brain tumor classification types. Transfer learning act as fine tuning, enabling the model to acquire fundamental low-level image features from a comprehensive dataset. It then leverages higher-level features to become more tailored to the specific training data. This method is employed to improve the model's adaptability to the training data. The InceptionResNet-V2 architecture utilized in the evaluation using test data, in Scenario 1, achieved 94.18% accuracy. Scenario 2, which combined augmentation with InceptionResNetV2, showed an improvement in accuracy to 95.10%. Furthermore, in Scenario 3, the combination of InceptionResNetV2 with Transfer Learning and augmentation resulted in an impressive accuracy of 96.63%, demonstrating its effectiveness in brain tumor classification. Transfer learning aligns the model by acquiring low-level image features and utilizing higher-level features to improve adaptability to the training data.

Expansive Receptive Field and Local Feature Extraction Network: Advancing Multiscale Feature Fusion for Breast Fibroadenoma Segmentation in Sonography.

Fibroadenoma is a common benign breast disease that affects women of all ages. Early diagnosis can greatly improve the treatment outcomes and reduce the associated pain. Computer-aided diagnosis (CAD) has great potential to improve diagnosis accuracy and efficiency. However, its application in sonography is limited. A network that utilizes expansive receptive fields and local information learning was proposed for the accurate segmentation of breast fibroadenomas in sonography. The architecture comprises the Hierarchical Attentive Fusion module, which conducts local information learning through channel-wise and pixel-wise perspectives, and the Residual Large-Kernel module, which utilizes multiscale large kernel convolution for global information learning. Additionally, multiscale feature fusion in both modules was included to enhance the stability of our network. Finally, an energy function and a data augmentation method were incorporated to fine-tune low-level features of medical images and improve data enhancement. The performance of our model is evaluated using both our local clinical dataset and a public dataset. Mean pixel accuracy (MPA) of 93.93% and 86.06% and mean intersection over union (MIOU) of 88.16% and 73.19% were achieved on the clinical and public datasets, respectively. They are significantly improved over state-of-the-art methods such as SegFormer (89.75% and 78.45% in MPA and 83.26% and 71.85% in MIOU, respectively). The proposed feature extraction strategy, combining local pixel-wise learning with an expansive receptive field for global information perception, demonstrates excellent feature learning capabilities. Due to this powerful and unique local-global feature extraction capability, our deep network achieves superior segmentation of breast fibroadenoma in sonography, which may be valuable in early diagnosis.

Survey of Techniques and Applications in Object Recognition

Abstract: Object detection, which is closely related to video analysis and image comprehension, has received attention in years from researchers. Traditional approaches to object detection rely on crafted features and shallow trainable architectures. However, these methods face challenges. Often reach a plateau, in performance improvement. To overcome these limitations complex ensembles are often created by combining low-level image features with high-level context from object detectors and scene classifiers. The emergence of learning has introduced powerful tools that can learn semantics high level and deeper features. These tools address the limitations of architectures by introducing network structures training strategies and optimization functions. This paper provides a review of learning-based frameworks for object detection. The review starts with an overview of the history of learning with a particular emphasis on the Convolutional Neural Network (CNN) as an exemplary tool. It then delves into generic object detection architectures exploring modifications and effective strategies for improving detection performance. Recognizing the characteristics of detection tasks, the paper also briefly surveys several tasks such, as salient object detection, face detection, crowd analysis, and pedestrian detection.

Contribution of low-level image statistics to EEG decoding of semantic content in multivariate and univariate models with feature optimization

Spatio-temporal patterns of evoked brain activity contain information that can be used to decode and categorize the semantic content of visual stimuli. However, this procedure can be biased by low-level image features independently of the semantic content present in the stimuli, prompting the need to understand the robustness of different models regarding these confounding factors. In this study, we trained machine learning models to distinguish between concepts included in the publicly available THINGS-EEG dataset using electroencephalography (EEG) data acquired during a rapid serial visual presentation paradigm. We investigated the contribution of low-level image features to decoding accuracy in a multivariate model, utilizing broadband data from all EEG channels. Additionally, we explored a univariate model obtained through data-driven feature selection applied to the spatial and frequency domains. While the univariate models exhibited better decoding accuracy, their predictions were less robust to the confounding effect of low-level image statistics. Notably, some of the models maintained their accuracy even after random replacement of the training dataset with semantically unrelated samples that presented similar low-level content. In conclusion, our findings suggest that model optimization impacts sensitivity to confounding factors, regardless of the resulting classification performance. Therefore, the choice of EEG features for semantic decoding should ideally be informed by criteria beyond classifier performance, such as the neurobiological mechanisms under study.

PortLaneNet: A Scene-Aware Model for Robust Lane Detection in Container Terminal Environments

In this paper, we introduce PortLaneNet, an optimized lane detection model specifically designed for the unique challenges of enclosed container terminal environments. Unlike conventional lane detection scenarios, this model addresses complexities such as intricate ground markings, tire crane lane lines, and various types of regional lines that significantly complicate detection tasks. Our approach includes the novel Scene Prior Perception Module, which leverages pre-training to provide essential prior information for more accurate lane detection. This module capitalizes on the enclosed nature of container terminals, where images from similar area scenes offer effective prior knowledge to enhance detection accuracy. Additionally, our model significantly improves understanding by integrating both high- and low-level image features through attention mechanisms, focusing on the critical components of lane detection. Through rigorous experimentation, PortLaneNet has demonstrated superior performance in port environments, outperforming traditional lane detection methods. The results confirm the effectiveness and superiority of our model in addressing the complex challenges of lane detection in such specific settings. Our work provides a valuable reference for solving lane detection issues in specialized environments and proposes new ideas and directions for future research.

EAD-Net: Efficiently Asymmetric Network for Semantic Labeling of High-Resolution Remote Sensing Images with Dynamic Routing Mechanism

Semantic labeling of high-resolution remote sensing images (HRRSIs) holds a significant position in the remote sensing domain. Although numerous deep-learning-based segmentation models have enhanced segmentation precision, their complexity leads to a significant increase in parameters and computational requirements. While ensuring segmentation accuracy, it is also crucial to improve segmentation speed. To address this issue, we propose an efficient asymmetric deep learning network for HRRSIs, referred to as EAD-Net. First, EAD-Net employs ResNet50 as the backbone without pooling, instead of the RepVGG block, to extract rich semantic features while reducing model complexity. Second, a dynamic routing module is proposed in EAD-Net to adjust routing based on the pixel occupancy of small-scale objects. Concurrently, a channel attention mechanism is used to preserve their features even with minimal occupancy. Third, a novel asymmetric decoder is introduced, which uses convolutional operations while discarding skip connections. This not only effectively reduces redundant features but also allows using low-level image features to enhance EAD-Net’s performance. Extensive experimental results on the ISPRS 2D semantic labeling challenge benchmark demonstrate that EAD-Net achieves state-of-the-art (SOTA) accuracy performance while reducing model complexity and inference time, while the mean Intersection over Union (mIoU) score reaching 87.38% and 93.10% in the Vaihingen and Potsdam datasets, respectively.

MKTN: Adversarial-Based Multifarious Knowledge Transfer Network from Complementary Teachers

With the demands for light deep networks models in various front-end devices, network compression has attracted increasing interest for reducing model sizes yet without sacrificing much model accuracy. This paper presents a multifarious knowledge transfer network (MKTN) that aims to produce a compact yet powerful student network from two complementary teacher networks. Instead of learning homogeneous features, the idea is to pre-train one teacher to capture generative and low-level image features under a reconstruction objective, and another teacher to capture discriminative and task-specific features under the same objective as the student network. During knowledge transfer, the student learns multifarious and complementary knowledge from the two teacher networks under the guidance of the proposed adversarial loss and feature loss respectively. Experimental results indicate that the proposed training losses can effectively guide the student to learn spatial-level and pixel-level information as distilled from teacher networks. On the other hand, our study over a number of widely used datasets shows that transferring multifarious features from complementary teachers equipped with different types of knowledge helps to teach a compact yet powerful student effectively.

DNNAM: Image inpainting algorithm via deep neural networks and attention mechanism

Most image inpainting algorithms have problems such as fuzzy images, texture distortion and semantic inaccuracy, and the image inpainting effect is limited when processing photos with large missing sections and resolution levels. The paper proposes an effective image inpainting algorithm via partial multi-scale channel attention mechanism and deep neural networks to address the above phenomenon that existing image inpainting methods using deep learning modules have insufficient perception and representation capabilities for multi-scale features with high proportion of irregular defects. Initially, we used the Res-U-Net module as a generator. The U-Net-like backbone network topology can achieve the encoding and decoding stages of damaged images. Secondly, the residual network structure was built in the encoder and decoder to improve the ability of the proposed network to extract and display the features of the damaged images. Finally, the partial multi-scale channel attention module was inserted in the skip connection with the decoder to increase the efficiency of using the low-level features of the original images. The experimental results of the research can show that the proposed method outperforms state-of-the-art methods in terms of subjective visual perception and objective evaluation indicators on the CelebA, Places2 and Paris Street View datasets.

Broadband and large-aperture metasurface edge encoders for incoherent infrared radiation.

The prevalence of computer vision systems necessitates hardware-based approaches to relieve the high computational demand of deep neural networks in resource-limited applications. One solution would be to off-load low-level image feature extraction, such as edge detection, from the digital network to the analog imaging system. To that end, this work demonstrates incoherent, broadband, low-noise optical edge detection of real-world scenes by combining the wavefront shaping of a 24-mm aperture metasurface with a refractive lens. An inverse design approach is used to optimize the metasurface for Laplacian-based edge detection across the 7.5- to 13.5-μm LWIR imaging band, allowing for facile integration with uncooled microbolometer-based LWIR imagers to encode edge information. A polarization multiplexed approach leveraging a birefringent metasurface is also demonstrated as a single-aperture implementation. This work could be applied to improve computer vision capabilities of resource-constrained systems by leveraging optical preprocessing to alleviate the computational requirements for high-accuracy image segmentation and classification.

Patterns of saliency and semantic features distinguish gaze of expert and novice viewers of surveillance footage

When viewing the actions of others, we not only see patterns of body movements, but we also "see" the intentions and social relations of people. Experienced forensic examiners – Closed Circuit Television (CCTV) operators – have been shown to convey superior performance in identifying and predicting hostile intentions from surveillance footage than novices. However, it remains largely unknown what visual content CCTV operators actively attend to, and whether CCTV operators develop different strategies for active information seeking from what novices do. Here, we conducted computational analysis for the gaze-centered stimuli captured by experienced CCTV operators and novices' eye movements when viewing the same surveillance footage. Low-level image features were extracted by a visual saliency model, whereas object-level semantic features were extracted by a deep convolutional neural network (DCNN), AlexNet, from gaze-centered regions. We found that the looking behavior of CCTV operators differs from novices by actively attending to visual contents with different patterns of saliency and semantic features. Expertise in selectively utilizing informative features at different levels of visual hierarchy may play an important role in facilitating the efficient detection of social relationships between agents and the prediction of harmful intentions.

Machine learning and similar image-based techniques based on Nash game theory

The use of computer vision techniques to address the task of image retrieval is known as a Content-Based Image Retrieval (CBIR) system. It is a system designed to locate and retrieve the appropriate digital image from a large database by utilizing a query image. Over the last few years, machine learning algorithms have achieved impressive results in image retrieval tasks due to their ability to learn from large amounts of diverse data and improve their accuracy in image recognition and retrieval. Our team has developed a CBIR system that is reinforced by two machine learning algorithms and employs multiple clustering and low-level image feature extraction, such as color, shape, and texture, to formulate a Nash game. Consequently, we are faced with a multicriteria optimization problem. To solve this problem, we have formulated a three-player static Nash game, where each player utilizes a different strategy (color descriptor, Zernike descriptor, and SFTA descriptor) based on their objective function. The Nash equilibrium is defined as the membership classes of the query image.

Face spoofing detection using Heterogeneous Auto-Similarities of Characteristics

Facial recognition systems are widely used in various daily applications due to their high accuracy rate and ease of use. Unfortunately, their performance drops drastically when encountering spoofing attacks. In this paper, a novel pipeline for face spoofing detection based on the extraction of Heterogeneous Auto-Similarities of Characteristics (HASC) descriptor from the HSV and YCbCr color spaces is proposed. To detect print and replay spoofing attacks, the HASC descriptor encodes linear and non-linear dependencies between low-level dense features of the color face image using covariance and information-theoretic measures. The combination of covariance and information-theoretic measures makes HASC an efficient descriptor to explore how the features are interrelated in real and fake faces. The HASC texture descriptions are extracted separately from each color channel of the HSV and YCbCr color spaces, and the resulting feature vectors are concatenated into an extended feature vector to better discriminate between real and fake faces. Extensive experiments are conducted on publicly available databases namely, Replay-Attack, CASIA FASD, and MSU MFSD. The obtained results demonstrate that while being of low dimension, the HASC descriptor when fed into a binary Support Vector Machine (SVM) classifier, achieves competitive results compared to the state-of-the-art.

Context-aware Optimization for Bandwidth-Efficient Image Analytics Offloading

Convolutional Neural Networks (CNN) have given rise to numerous visual analytics applications at the edge of the Internet. The image is typically captured by cameras and then live-streamed to edge servers for analytics due to the prohibitive cost of running CNN on computation-constrained end devices. A critical component to ensure low-latency and accurate visual analytics offloading over low bandwidth networks is image compression which minimizes the amount of visual data to offload and maximizes the decoding quality of salient pixels for analytics. Despite the wide adoption, JPEG standards and traditional image compression techniques do not address the accuracy of analytics tasks, leading to ineffective compression for visual analytics offloading. Although recent machine-centric image compression techniques leverage sophisticated neural network models or hardware architecture to support the accuracy-bandwidth trade-off, they introduce excessive latency in the visual analytics offloading pipeline. This paper presents CICO, a Context-aware Image Compression Optimization framework to achieve low-bandwidth and low-latency visual analytics offloading. CICO contextualizes image compression for offloading by employing easily-computable low-level image features to understand the importance of different image regions for a visual analytics task. Accordingly, CICO can optimize the trade-off between compression size and analytics accuracy. Extensive real-world experiments demonstrate that CICO reduces the bandwidth consumption of existing compression methods by up to 40% under comparable analytics accuracy. Regarding the low-latency support, CICO achieves up to a 2x speedup over state-of-the-art compression techniques.

Adaptive learning point cloud and image diversity feature fusion network for 3D object detection

3D object detection is a critical task in the fields of virtual reality and autonomous driving. Given that each sensor has its own strengths and limitations, multi-sensor-based 3D object detection has gained popularity. However, most existing methods extract high-level image semantic features and fuse them with point cloud features, focusing solely on consistent information from both sensors while ignoring their complementary information. In this paper, we present a novel two-stage multi-sensor deep neural network, called the adaptive learning point cloud and image diversity feature fusion network (APIDFF-Net), for 3D object detection. Our approach employs the fine-grained image information to complement the point cloud information by combining low-level image features with high-level point cloud features. Specifically, we design a shallow image feature extraction module to learn fine-grained information from images, instead of relying on deep layer features with coarse-grained information. Furthermore, we design a diversity feature fusion (DFF) module that transforms low-level image features into point-wise image features and explores their complementary features through an attention mechanism, ensuring an effective combination of fine-grained image features and point cloud features. Experiments on the KITTI benchmark show that the proposed method outperforms state-of-the-art methods.

Ontology-Based Semantic Construction Image Interpretation

Image-based techniques have become integral to the construction sector, aiding in project planning, progress monitoring, quality control, and documentation. In this paper, we address two key challenges that limit our ability to fully exploit the potential of images. The first is the “semantic gap” between low-level image features and high-level semantic descriptions. The second is the lack of principled integration between images and other digital systems used in construction, such as construction schedules and building information modeling (BIM). These challenges make it difficult to effectively incorporate images into digital twins of construction (DTC), a critical concept that addresses the construction industry’s need for more efficient project management and decision-making. To address these challenges, we first propose an ontology-based construction image interpretation (CII) framework to formalize the interpretation and integration workflow. Then, the DiCon-SII ontology is developed to provide a formalized vocabulary for visual construction contents and features. DiCon-SII also acts as a bridge between images and other digital systems to help construct an image-involved DTC. To evaluate the practical application of DiCon-SII and CII in supporting construction management tasks and as a precursor to DTC, we conducted a case study involving drywall installation. Via this case study, we demonstrate how the proposed methods can be used to infer the operational stage of a construction process, estimate labor productivity, and retrieve specific images based on user queries.

Detecting Ocular Tb With Modified Co-Occurrence Matrix

Image mapping is a method both for identifying and retrieving similar images from a large database of digital images. This system utilizes the low level image features such as color, texture and shape for retrieving similar images. Among the low level image features, color and texture are perceptually important low level features. Hence this paper entitled “Detecting ocular TB with modified co-occurrence matrix ” uses the color and texture information for identifying similar images from the database. The texture features are extracted from images using the modified co-occurrence matrix which is called texton co-occurrence matrix and the RGB color model is used for color texture.

Optic disc detection based on fully convolutional network and weighted matrix recovery model.

Eye diseases often affect human health. Accurate detection of the optic disc contour is one of the important steps in diagnosing and treating eye diseases. However, the structure of fundus images is complex, and the optic disc region is often disturbed by blood vessels. Considering that the optic disc is usually a saliency region in fundus images, we propose a weakly-supervised optic disc detection method based on the fully convolution neural network (FCN) combined with the weighted low-rank matrix recovery model (WLRR). Firstly, we extract the low-level features of the fundus image and cluster the pixels using the Simple Linear Iterative Clustering (SLIC) algorithm to generate the feature matrix. Secondly, the top-down semantic prior information provided by FCN and bottom-up background prior information of the optic disc region are used to jointly construct the prior information weighting matrix, which more accurately guides the decomposition of the feature matrix into a sparse matrix representing the optic disc and a low-rank matrix representing the background. Experimental results on the DRISHTI-GS dataset and IDRiD dataset show that our method can segment the optic disc region accurately, and its performance is better than existing weakly-supervised optic disc segmentation methods. Graphical abstract of optic disc segmentation.

Content-Based Image Retrieval using Hard Voting Ensemble Method of Inception, Xception, and Mobilenet Architectures

Advancements in internet accessibility and the affordability of digital picture sensors have led to the proliferation of extensive image databases utilized across a multitude of applications. Addressing the semantic gap between low-level attributes and human visual perception has become pivotal in refining Content-Based Image Retrieval (CBIR) methodologies, especially within this context. As this field is intensely researched, numerous efficient algorithms for CBIR systems have surfaced, precipitating significant progress in the artificial intelligence field. In this study, we propose employing a hard voting ensemble approach on features derived from three robust deep learning architectures: Inception, Exception, and Mobilenet. This is aimed at bridging the divide between low-level image features and human visual perception. The Euclidean method is adopted to determine the similarity metric between the query image and the features database. The outcome was a noticeable improvement in image retrieval accuracy. We applied our approach to a practical dataset named CBIR 50, which encompasses categories such as mobile phones, cars, cameras, and cats. The effectiveness of our method was thereby validated. Our approach outshone existing CBIR algorithms with superior accuracy (ACC), precision (PREC), recall (REC), and F1-score (F1-S), proving to be a noteworthy addition to the field of CBIR. Our proposed methodology could be potentially extended to various other sectors, including medical imaging and surveillance systems, where image retrieval accuracy is of paramount importance.