Set Of Low-level Features Research Articles

High-level and Low-level Feature Set for Image Caption Generation with Optimized Convolutional Neural Network

Automatic creation of image descriptions, i.e. captioning of images, is an important topic in artificial intelligence (AI) that bridges the gap between computer vision (CV) and natural language processing (NLP). Currently, neural networks are becoming increasingly popular in captioning images and researchers are looking for more efficient models for CV and sequence-sequence systems. This study focuses on a new image caption generation model that is divided into two stages. Initially, low-level features, such as contrast, sharpness, color and their high-level counterparts, such as motion and facial impact score, are extracted. Then, an optimized convolutional neural network (CNN) is harnessed to generate the captions from images. To enhance the accuracy of the process, the weights of CNN are optimally tuned via spider monkey optimization with sine chaotic map evaluation (SMO-SCME). The development of the proposed method is evaluated with a diversity of metrics.

Estimating the degree of conflict in speech by employing Bag-of-Audio-Words and Fisher Vectors

The automatic detection of conflict situations from human speech has several straightforward applications such as the surveillance of public spaces, providing feedback about employees in call centers, and other roles in human–computer interactions. In this study we examine the potential of different state-of-the-art feature extraction techniques, all developed to be able to efficiently represent a variable-length speech utterance by a fixed-length feature vector. Besides the ‘ComParE functionals’ attribute set, which became the de facto standard feature set in the area of computational paralinguistics (which focuses on the automatic assessment of non-verbal phenomena being present in human speech), we experiment with two methods introduced quite recently: Bag-of-Audio-Words (BoAW) and Fisher Vectors (FV). Using three standard basic, low-level feature sets, we found that, while BoAW proved to be quite sensitive to its meta-parameter settings, with Fisher Vectors we were able to achieve state-of-the-art conflict intensity estimation performance on a public and widely-used corpus. Furthermore, by applying Principal Component Analysis on the frame-level attributes, we managed to achieve a 30% speed-up in the feature extraction step. Interestingly, in contrast with our previous paralinguistic studies, combining the different predictions with these feature extraction approaches, we were unable to achieve any further significant improvement. The highest correlation coefficient values we got on the test set lay in the range 0.850–0.860, while the authors of several previous studies were able to achieve similar values (i.e. 0.849, 0.856 and 0.853). Considering that in this task the target score to be estimated (i.e. the intensity of the conflict being present in the actual clip) is definitely prone to subjectivity and therefore to label noise, current efforts have probably achieved the highest correlation coefficients attainable, and match human performance.

Distance Metric Learnt Kernel-Based Music Classification Using Timbral Descriptors

Automatic music genre classification based on distance metric learning (DML) is proposed in this paper. Three types of timbral descriptors, namely, mel-frequency cepstral coefficient (MFCC) features, modified group delay features (MODGDF) and low-level timbral feature sets are combined at the feature level. We experimented with k nearest neighbor (kNN) and support vector machine (SVM)-based classifiers for standard and DML kernels (DMLK) using GTZAN and Folk music dataset. Standard kernel-based kNN and SVM-based classifiers report classification accuracy (in%) of 79.03 and 90.16, respectively, on GTZAN dataset and 86.60 and 92.26, respectively, for Folk music dataset, with the best performing RBF kernel. A further improvement was observed when DML kernels were used in place of standard kernels in the kernel kNN and SVM-based classifiers with an accuracy of 84.46%, 92.74% (GTZAN), 90.00 and 96.23 (Folk music dataset) for DMLK-kNN and DMLK-SVM, respectively. The results demonstrate the potential of DML kernels in music genre classification task.

Restraining Bolts for Reinforcement Learning Agents

In this work we have investigated the concept of “restraining bolt”, inspired by Science Fiction. We have two distinct sets of features extracted from the world, one by the agent and one by the authority imposing some restraining specifications on the behaviour of the agent (the “restraining bolt”). The two sets of features and, hence the model of the world attainable from them, are apparently unrelated since of interest to independent parties. However they both account for (aspects of) the same world. We have considered the case in which the agent is a reinforcement learning agent on a set of low-level (subsymbolic) features, while the restraining bolt is specified logically using linear time logic on finite traces f/f over a set of high-level symbolic features. We show formally, and illustrate with examples, that, under general circumstances, the agent can learn while shaping its goals to suitably conform (as much as possible) to the restraining bolt specifications.1

Research on the Method of Color Fundus Image Optic Cup Segmentation Based on Deep Learning

The optic cup is a physiological structure in the fundus and is a small central depression in the eye. It has a normal proportion in the optic papilla. If the ratio is large, its size may be used to determine diseases such as glaucoma or congenital myopia. The occurrence of glaucoma is generally accompanied by physical changes to the optic cup, optic disc, and optic nerve fiber layer. Therefore, accurate measurement of the optic cup is important for the detection of glaucoma. The accurate segmentation of the optic cup is essential for the measurement of the size of the optic cup relative to other structures in the eye. This paper proposes a new network architecture we call Segmentation-ResNet Seg-ResNet that takes a residual network structure as the main body, introduces a channel weighting structure that automatically adjusts the dependence of the feature channels, re-calibrates the feature channels, and introduces a set of low-level features that are combined with high-level features to improve network performance. Pre-fusion features and fused features are symmetrical. Hence, this work correlates with the concept of symmetry. Combined with the training strategy of migration learning, the segmentation accuracy is improved while speeding up network convergence. The robustness and effectiveness of the proposed method are demonstrated by testing data from the GlaucomaRepo and Drishti-GS fundus image databases.

An Unsupervised Feature Learning Approach to Reduce False Alarm Rate in ICUs.

The high rate of false alarms in intensive care units (ICUs) is one of the top challenges of using medical technology in hospitals. These false alarms are often caused by patients' movements, detachment of monitoring sensors, or different sources of noise and interference that impact the collected signals from different monitoring devices. In this paper, we propose a novel set of high-level features based on unsupervised feature learning technique in order to effectively capture the characteristics of different arrhythmia in electrocardiogram (ECG) signal and differentiate them from irregularity in signals due to different sources of signal disturbances. This unsupervised feature learning technique, first extracts a set of low-level features from all existing heart cycles of a patient, and then clusters these segments for each individual patient to provide a set of prominent high-level features. The objective of the clustering phase is to enable the classification method to differentiate between the high-level features extracted from normal and abnormal cycles (i.e., either due to arrhythmia or different sources of distortions in signal) in order to put more attention to the features extracted from abnormal portion of the signal that contribute to the alarm.

Instant Stress: Detection of Perceived Mental Stress Through Smartphone Photoplethysmography and Thermal Imaging.

BackgroundA smartphone is a promising tool for daily cardiovascular measurement and mental stress monitoring. A smartphone camera–based photoplethysmography (PPG) and a low-cost thermal camera can be used to create cheap, convenient, and mobile monitoring systems. However, to ensure reliable monitoring results, a person must remain still for several minutes while a measurement is being taken. This is cumbersome and makes its use in real-life situations impractical.ObjectiveWe proposed a system that combines PPG and thermography with the aim of improving cardiovascular signal quality and detecting stress responses quickly.MethodsUsing a smartphone camera with a low-cost thermal camera added on, we built a novel system that continuously and reliably measures 2 different types of cardiovascular events: (1) blood volume pulse and (2) vasoconstriction/dilation-induced temperature changes of the nose tip. 17 participants, involved in stress-inducing mental workload tasks, measured their physiological responses to stressors over a short time period (20 seconds) immediately after each task. Participants reported their perceived stress levels on a 10-cm visual analog scale. For the instant stress inference task, we built novel low-level feature sets representing cardiovascular variability. We then used the automatic feature learning capability of artificial neural networks to improve the mapping between the extracted features and the self-reported ratings. We compared our proposed method with existing hand-engineered features-based machine learning methods.ResultsFirst, we found that the measured PPG signals presented high quality cardiac cyclic information (mean pSQI: 0.755; SD 0.068). We also found that the measured thermal changes of the nose tip presented high-quality breathing cyclic information and filtering helped extract vasoconstriction/dilation-induced patterns with fewer respiratory effects (mean pSQI: from 0.714 to 0.157). Second, we found low correlations between the self-reported stress scores and the existing metrics of the cardiovascular signals (ie, heart rate variability and thermal directionality) from short measurements, suggesting they were not very dependent upon one another. Third, we tested the performance of the instant perceived stress inference method. The proposed method achieved significantly higher accuracies than existing precrafted features-based methods. In addition, the 17-fold leave-one-subject-out cross-validation results showed that combining both modalities produced higher accuracy than using PPG or thermal imaging only (PPG+Thermal: 78.33%; PPG: 68.53%; Thermal: 58.82%). The multimodal results are comparable to the state-of-the-art stress recognition methods that require long-term measurements. Finally, we explored effects of different data labeling strategies on the sensitivity of our inference methods. Our results showed the need for separation of and normalization between individual data.ConclusionsThe results demonstrate the feasibility of using smartphone-based imaging for instant stress detection. Given that this approach does not need long-term measurements requiring attention and reduced mobility, we believe it is more suitable for mobile mental health care solutions in the wild.

Visual tracking in video sequences based on biologically inspired mechanisms

Visual tracking is the process of locating one or more objects based on their appearance. The high variation in the conditions and states of a moving object and presence of challenges such as background clutter, illumination variation, occlusion, etc. makes this problem extremely complex, and hard to achieve a robust algorithm in this field. However, unlike the machine vision, in the biological vision, the task of visual tracking is ideally conducted even in the worst conditions. Consequently, in this paper, taking into account the superior performance of biological vision in visual tracking, a biologically inspired visual tracking algorithm is introduced. The proposed algorithm inspiring the task-driven recognition procedure of the primary layers of the ventral pathway, and visual cortex mechanisms including spatial–temporal processing, motion perception, attention, and saliency to track a single object in the video sequence. For this purpose, a set of low-level features including the oriented-edges, color, and motion information (inspired by the layer V1) extracted from the target area and based on the discrimination rate that each feature creates with the background (inspired by the saliency mechanism), a subset of these features are employed to generate the appearance model and identify the target location. Moreover, by memorizing the shape and motion information (inspired by the short-term memory) scale variation and occlusion are handled. The experimental results showed that the proposed algorithm can well handle most of the visual tracking challenges, achieve high precision in target locating and act in a real-time manner.

Action unit detection and key frame selection for human activity prediction

Abstract Human activity prediction aims to recognize an unfinished activity with limited appearance and motion information. In this paper, we propose to predict an incomplete activity by combining the mid-level action units and the discriminative key frames exploited from each activity class. Specifically, we extract a great deal of action-related volumes from activity videos. Based on a set of low-level powerful features, similar volumes are aggregated into a mid-level feature, named action unit. Then, we detect these action units in each activity video and generate the frame feature by computing the distribution of concurrent action units in a single frame. Notice that human can easily recognize an incomplete activity using scanty key frames composed of representative interrelated action units together. The key frames in each activity class are selected by computing the entropy of each single frame feature. Finally, a structured SVM is trained to recognize activities with different observation ratios. The proposed approach is evaluated on several publicly available datasets in comparison with state-of-the-art approaches. The experimental results and analysis clearly demonstrate the effectiveness of the proposed approach.

CONTENT BASED IMAGE RETRIEVAL USING HYBRID-SVM

The goal of content based image retrieval is to retrieve the images that users want to search. Content based Image retrieval systems attempt to search through a database to find images that are perceptually similar to a query image. Set of low-Level visual features (Color, Shape and Texture) are used to represent an image in most modern content based image retrieval systems. Therefore, between high-level information and low-level features a gap exists, which are the main reason that down the improvement of the image retrieval accuracy.In this paper Hybrid support vector machine (SVM) method proposed to retrieve several features and shorten the semantic gap between low-level visual feature and high-level perception. Image data set is taken from coral image data set. MATLAB 2009a is used as a simulator to analysis the proposed work.

Detection of Melanoma Skin Disease by Extracting High Level Features for Skin Lesions

Melanoma is a very dangerous type of skin cancer as compared to others. It can be cured, when diagnosed in its early stage. The detection and diagnosis of skin cancer is difficult using earlier conventional methods. The accurate detection and diagnosis of melanoma is possible using suitable image processing techniques. High level features, measures asymmetry of skin lesion images. These features can be used to diagnose lesions as skin cancer (melanoma). This paper presents large set of low level features for analysing skin lesions. The best classification is obtained by combining the low level feature set with the high level feature set. The result shows that this method can be used and further developed as a tool for detection and classification of skin cancer (melanoma).

High-Level Feature Extraction for Classification and Person Re-Identification

The need to understand behaviors and identify individuals across surveillance cameras has led to a growing interest in research on feature extraction. High-level feature extraction aims at discriminating behaviors and individuals with high accuracy. In this paper, instead of using low-level image features, a novel method for extracting high-level features, which can be used for classification and person re-identification, is proposed. For this purpose, a set of low-level features is first extracted for each local of image. Then, a set of prototypes is sampled in each low-level feature space. These prototypes represent a different category of features which have the maximum inter-distance. Using canonical correlation analysis, new distance criteria are set to measure the mutual distance between each pair of sampled prototypes. This allows the proposed method to evaluate the similarity of the prototypes more accurately. As a result, the sample prototypes have the maximum inter-distance and their discriminative capacity is high. Once the set of the prototypes is achieved in each low-level feature space, the set is used to project the low-level feature space onto a new feature space. To serve this purpose, the correlation between the feature extracted for each local of image in the low-level feature space and each prototype in the set is computed and the resulting correlation is considered as one element of the new feature vector for that local. Now, there are two feature vectors for each local of image which are then combined to obtain a single feature vector. The deep belief network (DBN) is employed as a final stage whereby the single feature vector is fed into the DBN to train it in order to output a discriminant feature with a reduced dimension. This new high-level feature can be efficiently used for such applications as data classification and person re-identification.

Semantic Video Concept Detection using Novel Mixed-Hybrid-Fusion Approach for Multi-Label Data

The performance of the semantic concept detection method depends on, the selection of the low-level visual features used to represent key-frames of a shot and the selection of the feature-fusion method used. This paper proposes a set of low-level visual features of considerably smaller size and also proposes novel ‘hybrid-fusion’ and ‘mixed-hybrid-fusion’, approaches which are formulated by combining early and late-fusion strategies proposed in the literature. In the initially proposed hybrid-fusion approach, the features from the same feature group are combined using early-fusion before classifier training; and the concept probability scores from multiple classifiers are merged using late-fusion approach to get final detection scores. A feature group is defined as the features from the same feature family such as color moment. The hybrid-fusion approach is refined and the “mixed-hybrid-fusion” approach is proposed to further improve detection rate. This paper presents a novel video concept detection system for multi-label data using a proposed mixed-hybrid-fusion approach. Support Vector Machine (SVM) is used to build classifiers that produce concept probabilities for a test frame. The proposed approaches are evaluated on multi-label TRECVID2007 development dataset. Experimental results show that, the proposed mixed-hybrid-fusion approach performs better than other proposed hybrid-fusion approach and outperforms all conventional early-fusion and late-fusion approaches by large margins with respect to feature set dimensionality and Mean Average Precision (MAP) values.

Bridging the semantic gap with human perception based features for scene categorization

PurposeThe purpose of this paper is to build a classification system which mimics the perceptual ability of human vision, in gathering knowledge about the structure, content and the surrounding environment of a real-world natural scene, at a quick glance accurately. This paper proposes a set of novel features to determine the gist of a given scene based on dominant color, dominant direction, openness and roughness features.Design/methodology/approachThe classification system is designed at two different levels. At the first level, a set of low level features are extracted for each semantic feature. At the second level the extracted features are subjected to the process of feature evaluation, based on inter-class and intra-class distances. The most discriminating features are retained and used for training the support vector machine (SVM) classifier for two different data sets.FindingsAccuracy of the proposed system has been evaluated on two data sets: the well-known Oliva-Torralba data set and the customized image data set comprising of high-resolution images of natural landscapes. The experimentation on these two data sets with the proposed novel feature set and SVM classifier has provided 92.68 percent average classification accuracy, using ten-fold cross validation approach. The set of proposed features efficiently represent visual information and are therefore capable of narrowing the semantic gap between low-level image representation and high-level human perception.Originality/valueThe method presented in this paper represents a new approach for extracting low-level features of reduced dimensionality that is able to model human perception for the task of scene classification. The methods of mapping primitive features to high-level features are intuitive to the user and are capable of reducing the semantic gap. The proposed feature evaluation technique is general and can be applied across any domain.

Semantic attributes for people’s appearance description: an appearance modality for video surveillance applications

Using semantic attributes such as gender, clothes, and accessories to describe people’s appearance is an appealing modeling method for video surveillance applications. We proposed a midlevel appearance signature based on extracting a list of nameable semantic attributes describing the body in uncontrolled acquisition conditions. Conventional approaches extract the same set of low-level features to learn the semantic classifiers uniformly. Their critical limitation is the inability to capture the dominant visual characteristics for each trait separately. The proposed approach consists of extracting low-level features in an attribute-adaptive way by automatically selecting the most relevant features for each attribute separately. Furthermore, relying on a small training-dataset would easily lead to poor performance due to the large intraclass and interclass variations. We annotated large scale people images collected from different person reidentification benchmarks covering a large attribute sample and reflecting the challenges of uncontrolled acquisition conditions. These annotations were gathered into an appearance semantic attribute dataset that contains 3590 images annotated with 14 attributes. Various experiments prove that carefully designed features for learning the visual characteristics for an attribute provide an improvement of the correct classification accuracy and a reduction of both spatial and temporal complexities against state-of-the-art approaches.

Counting and Classification of Highway Vehicles by Regression Analysis

In this paper, we describe a novel algorithm that counts and classifies highway vehicles based on regression analysis. This algorithm requires no explicit segmentation or tracking of individual vehicles, which is usually an important part of many existing algorithms. Therefore, this algorithm is particularly useful when there are severe occlusions or vehicle resolution is low, in which extracted features are highly unreliable. There are mainly two contributions in our proposed algorithm. First, a warping method is developed to detect the foreground segments that contain unclassified vehicles. The common used modeling and tracking (e.g., Kalman filtering) of individual vehicles are not required. In order to reduce vehicle distortion caused by the foreshortening effect, a nonuniform mesh grid and a projective transformation are estimated and applied during the warping process. Second, we extract a set of low-level features for each foreground segment and develop a cascaded regression approach to count and classify vehicles directly, which has not been used in the area of intelligent transportation systems. Three different regressors are designed and evaluated. Experiments show that our regression-based algorithm is accurate and robust for poor quality videos, from which many existing algorithms could fail to extract reliable features.

Quantitative Study of Music Listening Behavior in a Smartphone Context

Context-based services have attracted increasing attention because of the prevalence of sensor-rich mobile devices such as smartphones. The idea is to recommend information that a user would be interested in according to the user’s surrounding context. Although remarkable progress has been made to contextualize music playback, relatively little research has been made using a large collection of real-life listening records collected in situ . In light of this fact, we present in this article a quantitative study of the personal, situational, and musical factors of musical preference in a smartphone context, using a new dataset comprising the listening records and self-report context annotation of 48 participants collected over 3wk via an Android app. Although the number of participants is limited and the population is biased towards students, the dataset is unique in that it is collected in a daily context, with sensor data and music listening profiles recorded at the same time. We investigate 3 core research questions evaluating the strength of a rich set of low-level and high-level audio features for music usage auto-tagging (i.e., music preference in different user activities), the strength of time-domain and frequency-domain sensor features for user activity classification, and how user factors such as personality traits are correlated with the predictability of music usage and user activity, using a closed set of 8 activity classes. We provide an in-depth discussion of the main findings of this study and their implications for the development of context-based music services for smartphones.

A Hybrid Approach for CBIR using SVM Classifier, Partical Swarm Optimizer with Mahalanobis Formula

The goal of content-based image retrieval is to retrieve the images that as per the user query. Mainly the Content based Image retrieval technique attempt to search through the database that finds images that are perceptually similar to a given query image. Set of low-Level visual features (Color, Shape and Texture) are used to represent an image in most modern content based image retrieval systems. Therefore, a gap exists between low-level visual features and information of high-level perception, which is the main reason that down the improvement of the image retrieval accuracy. To retrieve several features of images and shorten the semantic gap between low-level visual feature and high-level perception a Hybrid support vector machine (SVM) scheme is proposed in this paper. Image data set is taken from coral image data set.

An Efficient Approach for Content based Image Retrieval using SVM, KNN-GA as Multilayer Classifier

technology is growing day by day in various fields and image retrieval is one of the most of them, it is more interesting and fastest growing research areas. It is an effective and efficient tool for managing large image databases. In most Content-Based Image Retrieval (CBIR) systems, images are represented and differentiated by a set of low-level visual features; hence a direct correlation with high- level semantic information will be absent. Therefore, a gap exists between high-level information. In this paper they proposed novel approach for content based image retrieval was two tier architecture model is used for most accurate retrieval. In the first tier first feature extraction process done using PSO with SVM classifier, after successful classification in first tier the retrieved result has been passed into the second tier classifier. And in the second tier KNN classifier is used but as they knew that GA is one of the optimization technique and it produces the best optimized result in maximum cases so it is applied with the KNN classifier, and it produces more accurate and efficient compared result. Keywordsbased image retrieval (CBIR), feature extraction, SVM, PSO, KNN, GA, object optimization.

High-level intuitive features (HLIFs) for intuitive skin lesion description.

A set of high-level intuitive features (HLIFs) is proposed to quantitatively describe melanoma in standard camera images. Melanoma is the deadliest form of skin cancer. With rising incidence rates and subjectivity in current clinical detection methods, there is a need for melanoma decision support systems. Feature extraction is a critical step in melanoma decision support systems. Existing feature sets for analyzing standard camera images are comprised of low-level features, which exist in high-dimensional feature spaces and limit the system's ability to convey intuitive diagnostic rationale. The proposed HLIFs were designed to model the ABCD criteria commonly used by dermatologists such that each HLIF represents a human-observable characteristic. As such, intuitive diagnostic rationale can be conveyed to the user. Experimental results show that concatenating the proposed HLIFs with a full low-level feature set increased classification accuracy, and that HLIFs were able to separate the data better than low-level features with statistical significance. An example of a graphical interface for providing intuitive rationale is given.