Set Of Visual Features Research Articles

Spike desensitisation as a mechanism for high-contrast selectivity in retinal ganglion cells.

In the vertebrate retina, several dozens of parallel channels relay information about the visual world to the brain. These channels are represented by the different types of retinal ganglion cells (RGCs), whose responses are rendered selective for distinct sets of visual features by various mechanisms. These mechanisms can be roughly grouped into synaptic interactions and cell-intrinsic mechanisms, with the latter including dendritic morphology as well as ion channel complement and distribution. Here, we investigate how strongly ion channel complement can shape RGC output by comparing two mouse RGC types, the well-described ON alpha cell and a little-studied ON cell that is EGFP-labelled in the Igfbp5 mouse line and displays an unusual selectivity for stimuli with high contrast. Using patch-clamp recordings and computational modelling, we show that a higher activation threshold and a pronounced slow inactivation of the voltage-gated Na+ channels contribute to the distinct contrast tuning and transient responses in ON Igfbp5 RGCs, respectively. In contrast, such a mechanism could not be observed in ON alpha cells. This study provides an example for the powerful role that the last stage of retinal processing can play in shaping RGC responses.

Carry on Caesar

Abstract The variations on broadly fixed formulas used to represent such characters as the one portraying the Roman emperor, Julius Caesar, are all the more conspicuous for their, sometimes, almost imperceptible nuances. This article will illustrate some meaningful examples of filmic Caesars, focusing on the linguistic representation of this character, often handled as a stock character. The larger definition of stock characters, which includes but is not limited to stereotypes, encompasses a set of both visual and linguistic formulaic features which depend on identity constructions or social positions, to put it in Quantz’s terms (2015). The character Julius Caesar is often made to follow what film historians have called a general ‘linguistic paradigm’, by which British actors with posh accents, in post-war Hollywood epics of the 1950s, are frequently cast as wicked Roman tyrants or simply as members of the establishment opposed by the ‘hero’ of the tale. Departures from a schema can be however just as revealing in order to pinpoint recurrent themes, and they will be explored by focusing on the comic rendition of Caesar as interpreted by Kenneth Williams in the 1964 British comedy Carry on Cleo, and on the analysis of its outrageously racist and sexist Italian translation.

Unsupervised video anomaly detection in UAVs: a new approach based on learning and inference

In this paper, an innovative approach to detecting anomalous occurrences in video data without supervision is introduced, leveraging contextual data derived from visual characteristics and effectively addressing the semantic discrepancy that exists between visual information and the interpretation of atypical incidents. Our work incorporates Unmanned Aerial Vehicles (UAVs) to capture video data from a different perspective and to provide a unique set of visual features. Specifically, we put forward a technique for discerning context through scene comprehension, which entails the construction of a spatio-temporal contextual graph to represent various aspects of visual information. These aspects encompass the manifestation of objects, their interrelations within the spatio-temporal domain, and the categorization of the scenes captured by UAVs. To encode context information, we utilize Transformer with message passing for updating the graph's nodes and edges. Furthermore, we have designed a graph-oriented deep Variational Autoencoder (VAE) approach for unsupervised categorization of scenes, enabling the extraction of the spatio-temporal context graph across diverse settings. In conclusion, by utilizing contextual data, we ascertain anomaly scores at the frame-level to identify atypical occurrences. We assessed the efficacy of the suggested approach by employing it on a trio of intricate data collections, specifically, the UCF-Crime, Avenue, and ShanghaiTech datasets, which provided substantial evidence of the method's successful performance.

A stimulus-driven approach reveals vertical luminance gradient as a stimulus feature that drives human cortical scene selectivity

Human neuroimaging studies have revealed a dedicated cortical system for visual scene processing. But what is a “scene”? Here, we use a stimulus-driven approach to identify a stimulus feature that selectively drives cortical scene processing. Specifically, using fMRI data from BOLD5000, we examined the images that elicited the greatest response in the cortical scene processing system, and found that there is a common “vertical luminance gradient” (VLG), with the top half of a scene image brighter than the bottom half; moreover, across the entire set of images, VLG systematically increases with the neural response in the scene-selective regions (Study 1). Thus, we hypothesized that VLG is a stimulus feature that selectively engages cortical scene processing, and directly tested the role of VLG in driving cortical scene selectivity using tightly controlled VLG stimuli (Study 2). Consistent with our hypothesis, we found that the scene-selective cortical regions—but not an object-selective region or early visual cortex—responded significantly more to images of VLG over control stimuli with minimal VLG. Interestingly, such selectivity was also found for images with an “inverted” VLG, resembling the luminance gradient in night scenes. Finally, we also tested the behavioral relevance of VLG for visual scene recognition (Study 3); we found that participants even categorized tightly controlled stimuli of both upright and inverted VLG to be a place more than an object, indicating that VLG is also used for behavioral scene recognition. Taken together, these results reveal that VLG is a stimulus feature that selectively engages cortical scene processing, and provide evidence for a recent proposal that visual scenes can be characterized by a set of common and unique visual features.

PA-Tran: Learning to Estimate 3D Hand Pose with Partial Annotation.

This paper tackles a novel and challenging problem-3D hand pose estimation (HPE) from a single RGB image using partial annotation. Most HPE methods ignore the fact that the keypoints could be partially visible (e.g., under occlusions). In contrast, we propose a deep-learning framework, PA-Tran, that jointly estimates the keypoints status and 3D hand pose from a single RGB image with two dependent branches. The regression branch consists of a Transformer encoder which is trained to predict a set of target keypoints, given an input set of status, position, and visual features embedding from a convolutional neural network (CNN); the classification branch adopts a CNN for estimating the keypoints status. One key idea of PA-Tran is a selective mask training (SMT) objective that uses a binary encoding scheme to represent the status of the keypoints as observed or unobserved during training. In addition, by explicitly encoding the label status (observed/unobserved), the proposed PA-Tran can efficiently handle the condition when only partial annotation is available. Investigating the annotation percentage ranging from 50-100%, we show that training with partial annotation is more efficient (e.g., achieving the best 6.0 PA-MPJPE when using about 85% annotations). Moreover, we provide two new datasets. APDM-Hand, is for synthetic hands with APDM sensor accessories, which is designed for a specific hand task. PD-APDM-Hand, is a real hand dataset collected from Parkinson's Disease (PD) patients with partial annotation. The proposed PA-Tran can achieve higher estimation accuracy when evaluated on both proposed datasets and a more general hand dataset.

Classification of Paintings by Artistic Style Using Color and Texture Features

In this paper, an approach for the classification of paintings by artistic style using color and texture features is proposed. Our approach automatically extracts a set of visual features that effectively discriminate among diverse artistic styles. Additionally, our proposal performs an effective selection of the most relevant features to be used in an artificial neural network architecture. Using the most important features allows our system to achieve an efficient learning process. The proposed system analyzes digitized paintings using acombination of color and texture features, which have shown to be highly discriminatory. Our approach consists of two main stages: training and testing. Firstly, in the training stage, the features from seven artistic styles are extracted to train a multi-layer perceptron. Secondly, the learned model is utilized to determine the artistic style of a given incoming painting to our system. The experimental results, on an extensive dataset of digitized paintings, show that our method obtains a higher accuracy in comparison with those obtained by the state-of-the-art methods. Moreover, our proposal attains a higher accuracy rate using fewer features descriptors.

Visual Aesthetic Quality Assessment of Urban Forests: A Conceptual Framework

Visual aesthetic quality is the visual pleasure level that attracts people and makes them prefer certain areas. Visual aesthetic quality is valued and considered for urban forests but remains challenging. This could be due to a lack of understanding of visual aesthetic quality assessment variables based on visual aesthetic theories. This study supports an integrated conceptual framework based on the result of a systematic literature review study to describe and measure aesthetics that incorporates objective and subjective factors through urban forest visual character and urban forest visual quality. The results include defining and understanding a description of visual aesthetic factors and variables as well as a thorough explanation of visual aesthetic theories to comprehend how to assess the visual aesthetic quality of urban forests. This study agrees with and supports the visual aesthetic theoretical framework, and we believe that due to our shared evolutionary history, humans have a standard set of urban forest visual aesthetic features with preferences that change according to cultural and personal variances. Furthermore, this research provides a foundation of visual aesthetic variables of urban forests that will assist urban forest researchers, urban forest managers, and decision-makers in managing and protecting the visual aesthetics of urban forests.

Enhancement of email spam detection using improved deep learning algorithms for cyber security

Email has sustained to be an essential part of our lives and as a means for better communication on the internet. The challenge pertains to the spam emails residing a large amount of space and bandwidth. The defect of state-of-the-art spam filtering methods like misclassification of genuine emails as spam (false positives) is the rising challenge to the internet world. Depending on the classification techniques, literature provides various algorithms for the classification of email spam. This paper tactics to develop a novel spam detection model for improved cybersecurity. The proposed model involves several phases like dataset acquisition, feature extraction, optimal feature selection, and detection. Initially, the benchmark dataset of email is collected that involves both text and image datasets. Next, the feature extraction is performed using two sets of features like text features and visual features. In the text features, Term Frequency-Inverse Document Frequency (TF-IDF) is extracted. For the visual features, color correlogram and Gray-Level Co-occurrence Matrix (GLCM) are determined. Since the length of the extracted feature vector seems to the long, the optimal feature selection process is done. The optimal feature selection is performed by a new meta-heuristic algorithm called Fitness Oriented Levy Improvement-based Dragonfly Algorithm (FLI-DA). Once the optimal features are selected, the detection is performed by the hybrid learning technique that is composed of two deep learning approaches named Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN). For improving the performance of existing deep learning approaches, the number of hidden neurons of RNN and CNN is optimized by the same FLI-DA. Finally, the optimized hybrid learning technique having CNN and RNN classifies the data into spam and ham. The experimental outcomes show the ability of the proposed method to perform the spam email classification based on improved deep learning.

Vision Processing for Assistive Vision: A Deep Reinforcement Learning Approach

There is increasing interest in using computer vision and machine learning to enhance human decision making with computer-mediated assistive vision systems. In particular, retinal implants are a rapidly advancing technology offering individuals suffering vision loss due to retinal dystrophies, an opportunity to restore partial vision. However, the visual representations achievable with current and near-term implants are severely limited in resolution and contrast, placing high importance on the selection of visual features to convey via the implant. Using vision processing algorithms on camera-captured input, functional outcomes can be enhanced with such devices. To this end, we propose a novel end-to-end vision processing pipeline for prosthetic vision that learns task-salient visual filters in simulation offline via deep reinforcement learning (DRL). Once learnt, these filters are deployable on a prosthetic vision device to process camera-captured images and produce task-guiding scene representations in real-time. We show how a set of learnt visual features enabling a virtual agent to optimally perform the task of navigation in a 3-D environment can be extracted and applied to enhance the same features in real world images. We evaluate and validate our proposed approach quantitatively and qualitatively using simulated prosthetic vision. To our knowledge, this is the first application of DRL to the derivation of scene representations for human-centric computer-mediated displays such as prosthetic vision devices.

Large-scale extraction of interpretable features provides new insights into kidney histopathology – A proof-of-concept study

Whole slide images contain a magnitude of quantitative information that may not be fully explored in qualitative visual assessments. We propose: (1) a novel pipeline for extracting a comprehensive set of visual features, which are detectable by a pathologist, as well as sub-visual features, which are not discernible by human experts and (2) perform detailed analyses on renal images from mice with experimental unilateral ureteral obstruction. An important criterion for these features is that they are easy to interpret, as opposed to features obtained from neural networks. We extract and compare features from pathological and healthy control kidneys to learn how the compartments (glomerulus, Bowman's capsule, tubule, interstitium, artery, and arterial lumen) are affected by the pathology. We define feature selection methods to extract the most informative and discriminative features. We perform statistical analyses to understand the relation of the extracted features, both individually, and in combinations, with tissue morphology and pathology. Particularly for the presented case-study, we highlight features that are affected in each compartment. With this, prior biological knowledge, such as the increase in interstitial nuclei, is confirmed and presented in a quantitative way, alongside with novel findings, like color and intensity changes in glomeruli and Bowman's capsule. The proposed approach is therefore an important step towards quantitative, reproducible, and rater-independent analysis in histopathology.

Detection of Cotton Leaf Disease Using Image Processing Techniques

In the area of research, diagnosis of disease symptoms in the plants duly applying image processing methods is a matter of big concern. The need of the hour is to prepare an efficient plant disease diagnosis system that can help the farmers in their cultivation and farming. This work is an attempt to prepare a framework of plant disease diagnosis system by using the cotton plant leaves. The digital pictures of cotton leaves are obtained to undergo a set of image processing techniques. Thresholding based segmentation techniques are used to remove the region of interest (ROI) i.e., infected part from the enhanced images. Consequently, diseases are detected from the region of interest by using an accurate set of visual texture features. At last treatment actions are taken to supervise the diseases found in the plants. This work will help the farmer’s society to take effective measures to protect their crops from diseases.

Intelligent Splicing Method of Virtual Reality Lingnan Cultural Heritage Panorama Based on Automatic Machine Learning

With the increasing expansion of virtual reality application fields and the complexity of application content, the demand for real-time rendering of realistic graphics has increased sharply. This research mainly discusses the intelligent mosaic method of virtual reality Lingnan cultural heritage panorama based on automatic machine learning. In order to effectively make up for the impact of the insufficiency of the collection process on the quality of the final panoramic image of Lingnan cultural heritage, it is necessary to minimize the irregular rotation of the camera and collect images according to the overlapping area between adjacent images of appropriate size. In order to make Lingnan cultural heritage panoramic images have better visual effects, it is necessary to preprocess the images before image registration and fusion. Image preprocessing mainly includes image denoising and image projection transformation. In this study, cylindrical projection is used to construct the panorama of Lingnan cultural heritage. For each Lingnan cultural heritage training image, we first perform image segmentation to obtain multiple regions and extract the visual features of each region. We use automatic machine learning models to train the visual feature set and use the bagging method to generate different training subsets. In order to generate each component classifier, we determine the overlap area of the two images according to the matched SIFT feature points and determine the best stitching line during the implementation of stitching. In this paper, the number of pixels in the first row of the overlapping area is used to determine the candidate stitching line column, and the best stitching line position should be determined in consideration of the smallest color difference in the stitching area and the most similar texture on both sides. This article uses a Java Applet-based approach to realize virtual roaming of viewing panoramic images of Lingnan cultural heritage in IE browser. The highest accuracy of SIFT is 82.22%, and the lowest recognition time is 0.01 s. This research will promote the development of Lingnan cultural heritage.

Multimodal video-text matching using a deep bifurcation network and joint embedding of visual and textual features

Video-text matching is of high importance in the field of machine vision and artificial intelligence. The main challenging issue in video-text matching is the projection of the video and textual features into a common semantic space, which is called video-text joint embedding. The proper functionality of video-text joint embedding depends on two important factors: the effectiveness of the extracted information for video-text matching and the suitability of network structure for the projection of the extracted features into a common space. Generally, existing approaches do not leverage all the audio and visual information of a video for video-text matching. This study proposes a new approach for video-text matching by extracting a comprehensive set of visual and textual features and projecting them into a common semantic space using an effective structure. We use a new deep network with two textual and video branches that extracts several informative high-level visual and textual features and maps them into a shared space. In the video branch, we extract several descriptors comprising appearance-based, concept-based, and action-based features as well as an audio encoder to extract sound features in the video clip. We also utilize several feature extraction approaches in the textual branch for the effective transformation of an input sentence to a common semantic space. Furthermore, an image description database is used to pre-train and initialize network weights. We evaluated the proposed architecture with two popular video description datasets and compared the results with the results of several state-of-the-art approaches. The comparison results showed the effectiveness of the proposed algorithm.

Topography of Visual Features in the Human Ventral Visual Pathway.

Visual object recognition in humans and nonhuman primates is achieved by the ventral visual pathway (ventral occipital-temporal cortex, VOTC), which shows a well-documented object domain structure. An on-going question is what type of information is processed in the higher-order VOTC that underlies such observations, with recent evidence suggesting effects of certain visual features. Combining computational vision models, fMRI experiment using a parametric-modulation approach, and natural image statistics of common objects, we depicted the neural distribution of a comprehensive set of visual features in the VOTC, identifying voxel sensitivities with specific feature sets across geometry/shape, Fourier power, and color. The visual feature combination pattern in the VOTC is significantly explained by their relationships to different types of response-action computation (fight-or-flight, navigation, and manipulation), as derived from behavioral ratings and natural image statistics. These results offer a comprehensive visual feature map in the VOTC and a plausible theoretical explanation as a mapping onto different types of downstream response-action systems.

Prediction of complex stimuli across saccades.

The visual system can predict visual features across saccades based on learned transsaccadic associations between peripheral and foveal input. This has been shown for simple visual features such as shape, size, and spatial frequency. The present study investigated whether transsaccadic predictions are also made for more complex visual stimuli. In an acquisition phase, new transsaccadic associations were established. In the first experiment, pictures of real-world objects changed category during the saccade (fruits were changed into balls or vice versa). In the second experiment, the gender of faces was manipulated during the saccade (faces changed from male to female or vice versa). In the following test phase, the stimuli were briefly presented in the periphery, and participants had to indicate which object or face, respectively, they had perceived. In both experiments, peripheral perception was biased toward the acquired associated foveal input. These results demonstrate that transsaccadic predictions are not limited to a small set of simple visual features but can also be made for more complex and realistic stimuli. Multiple new associations can be learned within a short time frame, and the resulting predictions appear to be object specific.

Analyzing the effect of adding temporal features to an autoencoder-based video quality model

According to Cisco, most Internet traffic is currently comprised of videos. Therefore, developing a quality assessment method for assuring that those videos are received and displayed with quality at the user side is an important and challenging task. As a consequence, over the last decades, several no-reference video quality metrics have been proposed with the goal of blindly predicting (with no access to the original signal) the quality of videos in streaming applications. One of such metrics is NAVE, whose architecture includes an auto-encoder module that produces a compact set of visual features with a higher descriptive capacity. Nevertheless, the visual features in NAVE do not include descriptive temporal features that are sensitive to temporal degradation. In this work, we analyze the effect on accuracy performance of using a new type of temporal features, based on natural scene statistics. This approach has the goal of making the tested video quality metric more generic, i.e. sensitive to both spatial and temporal distortions and therefore adequate for video streaming applications.

Predictive Control‐Based Completeness Analysis and Global Calibration of Robot Vision Features

This paper provides an in-depth study and analysis of robot vision features for predictive control and a global calibration of their feature completeness. The acquisition and use of the complete macrofeature set are studied in the context of a robot task by defining the complete macrofeature set at the level of the overall purpose and constraints of the robot vision servo task. The visual feature set that can fully characterize the macropurpose and constraints of a vision servo task is defined as the complete macrofeature set. Due to the complexity of the task, a part of the features of the complete macrofeature set is obtained directly from the image, and another part of the features is obtained from the image by inference. The task is guaranteed to be completely based on a robust calibration-free visual serving strategy based on interference observer that is proposed to complete the visual serving task with high performance. To address the problems of singular values, local minima, and insufficient robustness in the traditional scale-free vision servo algorithm, a new scale-free vision servo method is proposed to construct a dual closed-loop vision servo structure based on interference observer, which ensures the closed-loop stability of the system through the Q-filter-based interference observer, while estimating and eliminating the interference consisting of hand-eye mapping model uncertainty and controlled robot input interference. The equivalent interference consisting of hand-eye mapping model uncertainty, controlled robot input interference, and detection noise is estimated and eliminated to obtain an inner-loop structure that presents a nominal model externally, and then an outer-loop controller is designed according to the nominal model to achieve the best performance of the system dynamic performance and robustness to optimally perform the vision servo task.

A multi-modal approach to predict the strength of doctor–patient relationships

Advances in healthcare social media and information about the doctor-patient (D–P) communication regarding the prior patients’ treatment experience, can positively influence the D–P relationship. In pace with prior patients’ photo-sharing on healthcare social media websites from personal computers and smartphones regarding their treatment experience, the amount of multi-modal content has been growing exponentially. Therefore, there is an increasing need for coping with such information to mine useful knowledge about the D–P communication. Scraping 68,610 reviews, including 4618 photos from a popular physician-rating site, Yelp.com , this study proposes a novel, real-time, multi-modal classification framework, which uses textual and visual modalities as a source of information. Furthermore, this work suggests a social media image filtering mechanism that filters duplicate and irrelevant information from the data. Results show that the data filtering enhances the information reliability, whereas the addition of novel text and visual feature sets improves the classification accuracy up to 16.94%. In addition, fusing textual and visual features enhance the performance of the classifier by 18.24%, which produces better results than considering them separately. The findings also revealed that deep learning algorithms outperformed the classical machine learning algorithms across the entire novel features model, indicating the usefulness and suitability of the proposed methodology. Lastly, the findings from extensive experiments on the physicians’ reviews dataset will guide the doctors to demonstrate the implication of the proposed system for improving the D–P relationship.

Appearance and shape-based hybrid visual feature extraction: toward audio–visual automatic speech recognition

Nowadays, audio–visual automatic speech recognition (AV-ASR) is an emerging field of research, but there is still lack of proper visual features for visual speech recognition. Visual features are mainly categorized into shape based and appearance based. Based on the different information embedded in shape and appearance features, this paper proposes a new set of hybrid visual features which lead to a better visual speech recognition system. Pseudo-Zernike Moment (PZM) is calculated for shape-based visual feature while Local Bnary Pattern-three orthogonal planes (LBP-TOP) and Discrete Cosine Transform (DCT) are calculated for the appearance-based feature. Moreover, our proposed method also gathers global and local visual information. Thus, the objective of the proposed system is to embed all this visual information into a compact features set. Here, for audio speech recognition, the proposed system uses Mel-frequency cepstral coefficients (MFCC). We also propose a hybrid classification method to carry out all the experiments of AV-ASR. Artificial Neural Network (ANN), multiclass Support Vector Machine (SVM) and Naive Bayes (NB) classifiers are used for classifier hybridization. It is shown that the proposed AV-ASR system with a hybrid classifier significantly improves the recognition rate.

Multi-Instance Learning Algorithm Based on LSTM for Chinese Painting Image Classification

Aiming at the problem of weakly supervised learning in traditional Chinese painting image classification, a novel multi-instance learning algorithm based on Long and Short-Term Memory neural network with attention mechanism (ALSTM-MIL) is proposed. Firstly, by using the Pyramid Overlapping Grid Division (POGP), a multi-instance modeling scheme is designed to convert Chinese painting images into multi-instance bag, thereby transforming the problem of Chinese painting image classification into a MIL problem. Secondly, an efficient sequence generator is designed. It selects discriminative instances from the positive bags, construct a discriminative instance set (DIS), and convert multi-instance bags into equal-length ordered sequences. Thirdly, an LSTM network model with an attention mechanism is designed to perform semantic analysis on multi-instance bags to obtain their memory coding features, and then combined with the Softmax classifier to achieve semantic classification of traditional Chinese painting images. Experimental results on the Chinese painting (CP) image set show that the LSTM network built on the visual feature set is feasible, and the performance of the proposed MIL algorithm is also superior to other classification algorithms.