Multiple Low-level Features Research Articles

Semantic-Consistency-guided Learning on Deep Features for Unsupervised Salient Object Detection

Unsupervised salient object detection is an important task in many real-world scenarios where pixel-wise label information is of scarce availability. Despite its significance, this problem remains rarely explored, with a few works that consider unsupervised salient object detection methods based on the fused graph from the sum fusion of multiple deep feature similarity matrices. However, these methods ignore the interrelation of the low-level feature similarity matrices and the high-level semantic similarity matrice, which degrades the quality of the fused graph. In this article, we propose a semantic-consistency-guided multi-graph fusion learning algorithm for unsupervised saliency detection, where the consistency and inconsistency between multiple low-level feature similarity matrices and the high-level semantic similarity matrice are explored to promote the robustness and quality of the fused graph. In the first stage, a semantic-consistency-guided multi-graph fusion learning method is proposed to exploit consistency and inconsistency of multiple low-level deep features and the high-level semantic feature. The semantic-consistency-guided similarity matrices are computed for preliminary saliency ranking. In the following saliency refinement stage, the semantic-enhanced similarity matrices are built by the cross diffusion to fuse the multiple low-level deep features and the high semantic deep feature. Based on the semantic-enhanced similarity matrices, the refinement saliency maps are calculated in a semantic-enhanced cellular automata manner. Furthermore, the final ensemble stage of the large margin semi-supervised classification views the preliminary ranking results and refinement results as features, adopts the large margin graphs for saliency ensemble. Extensive evaluations over four benchmark datasets show that the proposed unsupervised method performs favorably against the state-of-the-art approaches and is competitive with some supervised deep learning-based methods.

Smart Video Surveillance Using YOLO Algorithm and OpenCV

Abstract: Due to object detection’s close relationship with video analysis and image understanding, it has attracted much research attention in recent years. Traditional object detection methods are built on handcrafted features and shallow trainable architectures. Their performance easily stagnates by constructing complex ensembles that combine multiple low-level image features with highlevel context from object detectors and scene classifiers. With the rapid development in deep learning, more powerful tools, which can learn semantic, high-level, deeper features, are introduced to address the problems existing in traditional architectures.

Configuration perceptual learning and its relationship with element perceptual learning.

Visual perceptual learning has been studied extensively and reported to enhance the perception of almost all types of training stimuli, from low- to high-level visual stimuli. Notably, high-level stimuli are often composed of multiple low-level features. Therefore, it is natural to ask whether training of high-level stimuli affects the perception of low-level stimuli and vice versa. In the present study, we trained subjects with either a high-level configuration stimulus or a low-level element stimulus. The high-level configuration stimulus consisted of two Gabors in the left and right visual fields, respectively, and the low-level element stimulus was the Gabor in the right visual field of the configuration stimulus. We measured the perceptual learning effects using the configuration stimulus and the element stimuli in both left and right visual fields. We found that the configuration perceptual learning equally improved the perception of the configuration stimulus and both element stimuli. In contrast, the element perceptual learning was confined to the trained element stimulus. These findings demonstrate an asymmetric relationship between perceptual learning of the configuration and the element stimuli and suggest a hybrid mechanism of the configuration perceptual learning. Our findings also offer a promising paradigm to promote the efficiency of perceptual learning-that is, gaining more learning effect with less training time.

ImDeeplabV3plus with instance selective whitening loss in domain generalization semantic segmentation

Semantic segmentation is a classical problem in computer vision, which is important in the field of autonomous driving. Although significant progress has been achieved in semantic segmentation, its generalization ability to unknown domains is still challenging. To effectively solve this problem, a semantic segmentation method ImDeeplabV3plus with instance selective whitening loss is proposed in this paper. DeeplabV3plus is selected as the baseline. In order to enhance the representation of the region of interest, the coordinate attention (CA) mechanism is added. To better integrate multiple low-level features, the adaptively spatial feature fusion (ASFF) is employed to adaptively learn the importance of features at different levels for each location. For preferably coping with the domain changes, an instance selective whitening (ISW) loss is introduced in the early stage of the backbone. The model is trained with the Cityscapes dataset and then applied to the unknown domain RobotCar dataset. Compared with DeeplabV3plus, the authors’ ImDeeplabV3plus model shows 1.29% mIoU improvement. When ISW loss is added, 2.08% improvement in mIoU is achieved compared with ImDeeplabV3plus. Experimental results show that the proposed method is simple and improves the domain generalization ability.

Advances in Object Detection Algorithms – A Brief Survey

One of the key senses mankind has always desired to impart to an Intelligent system is Computer Vision, as it enables the system to sense its environment, automatically understand the context and decide whether to take the next action or not. This action can be observation, navigation, motion, manoeuvre, etc., Over the last few decades, we have seen tremendous advances in object detection, and with the advent of Deep learning it is possible to accurately process images/videos, classify, segment, and detect all the objects in the environment, leading to contextual understanding of the image/video for further actions to be taken based on the context. In this paper I explore the various advances made in Object Detection and present how algorithms have evolved over time, giving us near accurate outcomes. In this I evaluate key algorithms before the invent of CNNs and then few after them. I also explore the one-stage and two-stage algorithms including R-CNN family of algorithms, YOLO algorithms, and few others. I start with an overview of object detection and the stages involved, discuss a brief working of the algorithm and showcase its detection accuracy results, offering the advantages and disadvantages of a specific algorithm. Post 2014, with deep learning coming to the fore, object detection has transformed computer’s ability to classify, and today’s algorithms offer very high accuracy in detection as they heavily deploy selective search algorithms combining multiple low-level image features with high-level context. Selective search algorithms assist in extracting ROI (Region of Interest) creating a boundary around the object and for each ROI, SVM (Support Vector Machine) classifiers are used to determine the type of object in the ROI. R-CNN and YOLO models have varied applications in the field of tracking

Robust acoustic scene classification using a multi-spectrogram encoder-decoder framework

This article proposes an encoder-decoder network model for Acoustic Scene Classification (ASC), the task of identifying the scene of an audio recording from its acoustic signature. We make use of multiple low-level spectrogram features at the front-end, transformed into higher level features through a well-trained CNN-DNN front-end encoder. The high-level features and their combination (via a trained feature combiner) are then fed into different decoder models comprising random forest regression, DNNs and a mixture of experts, for back-end classification. We conduct extensive experiments to evaluate the performance of this framework on various ASC datasets, including LITIS Rouen and IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events (DCASE) 2016 Task 1, 2017 Task 1, 2018 Tasks 1A & 1B and 2019 Tasks 1A & 1B. The experimental results highlight two main contributions; the first is an effective method for high-level feature extraction from multi-spectrogram input via the novel CNN-DNN architecture encoder network, and the second is the proposed decoder which enables the framework to achieve competitive results on various datasets. The fact that a single framework is highly competitive for several different challenges is an indicator of its robustness for performing general ASC tasks.

Machine learning applied in natural language processing

Machine learning applied in natural language processing

Review on Medical Image Retrieval Based on Wavelet, Bag of Features and Relevance Feedback

Technological advances have evolved in all the directions including the biomedical, because of which a record number of lives are saved every day. The advancement has now surpassed the tools level, now the doctors with the help of new tools can also detect diseases, which saves the response time. In this paper, we will work on one such technique which will help in retrieving the similar type of images with the help of their features. In this paper, the features such as Texture features, LBP features, Retrieval feature, which are processed with hash coding and relevance feedback to get the final results. The framework provides the output utilizing a hash coding classifiers which predict the image from the database of the images. The images are classified on a global level with the help of multiple low-level features.

A modified Efficient Manifold Ranking Algorithm for Large Database Image Retrieval

The efficient manifold ranking (EMR) algorithm is used quite effectively in content-based image retrieval (CBIR) for large image databases where images are represented by multiple low-level features to describe about the color, texture and shape. The EMR ranking algorithm requires steps to determine anchor points of the image database by using the k-means hard clustering and the accuracy of the ranking depends strongly on the selected anchor points. This paper describes a new result based on a modified Fuzzy C-Means (FCM) clustering algorithm to select anchor points in the large database in order to increase the efficiency of manifold ranking specially for the large database cases. Experiments have demonstrated the effectiveness of the proposed algorithm for the issue of building an anchor graph, the set of anchor points determined by this novel lvdc-FCM algorithm has actually increased the effective of manifold ranking and the quality of images query results which retrieved of the CBIR.

Cross-Level Parallel Network for Crowd Counting

Automated people counting in crowd scenes is challenging due to large variations in scale, density, and background clutter. To tackle them, we propose a novel cross-level parallel network (CLPNet) by extracting multiple low-level features from VGG16 and fusing them with specific scale aggregation modules in the high-level stage. To deal with scale variation, we design five different aggregation modules for multiscale fusion. Furthermore, the ground truth is processed skillfully to eliminate the mismatches caused by the scale variation between heads and density maps. To cope with background clutter, cross-level feature fusion is implemented. Higher-level semantic information could effectively separate head from background and regain the lost low-level detailed information. To address the variation of density, we design a parallel network, in which two separate channels focus on different density-level estimation, and attain more accurate counting results. Finally, we evaluate the proposed CLPNet on four representative crowd counting datasets, i.e., ShanghaiTech, UCF_CC_50, WorldExpo’10, and UCF_QNRF. The experimental results demonstrate that with the cross-level and multiscale structure CLPNet achieves superior performance compared with the state-of-the-art crowd counting methods.

A novel two-stage scene classification model based on feature variable significance in high-resolution remote sensing

In this article, we focus on the high-resolution remote sensing scene classification, which aims to label the scene image according to its content. While the high similarity of the between-class and variety of the inner class in the scene images make it challenging. To address the issue, we propose a unified two-stage classification framework named variable-weighted multi-feature fusing (VWMF) method. In the first stage, we fuse multiple low-level features into a concatenated holistic histogram feature via pre-trained unsupervised models. Then, we use a kernel collaborative representation-based classification method to predict the two most possible categories. In the second stage, for the two possible categories, we calculate the variable-weighted feature through a pre-calculated feature variable weight set of the corresponding two categories, and then put the variable-weighted feature into the pre-trained SVM for one-versus-one classification. Comprehensive experiments on two publicly available data sets demonstrate the superiority of our VWMF method compared with state-of-the-art remote sensing scene classification methods.

Object Detection With Deep Learning: A Review.

Due to object detection's close relationship with video analysis and image understanding, it has attracted much research attention in recent years. Traditional object detection methods are built on handcrafted features and shallow trainable architectures. Their performance easily stagnates by constructing complex ensembles that combine multiple low-level image features with high-level context from object detectors and scene classifiers. With the rapid development in deep learning, more powerful tools, which are able to learn semantic, high-level, deeper features, are introduced to address the problems existing in traditional architectures. These models behave differently in network architecture, training strategy, and optimization function. In this paper, we provide a review of deep learning-based object detection frameworks. Our review begins with a brief introduction on the history of deep learning and its representative tool, namely, the convolutional neural network. Then, we focus on typical generic object detection architectures along with some modifications and useful tricks to improve detection performance further. As distinct specific detection tasks exhibit different characteristics, we also briefly survey several specific tasks, including salient object detection, face detection, and pedestrian detection. Experimental analyses are also provided to compare various methods and draw some meaningful conclusions. Finally, several promising directions and tasks are provided to serve as guidelines for future work in both object detection and relevant neural network-based learning systems.

Structured learning of metric ensembles with application to person re-identification

Matching individuals across non-overlapping camera networks, known as person re-identification, is a fundamentally challenging problem due to the large visual appearance changes caused by variations of viewpoints, lighting, and occlusion. Approaches in literature can be categorized into two streams: The first stream is to develop reliable features against realistic conditions by combining several visual features in a pre-defined way; the second stream is to learn a metric from training data to ensure strong inter-class differences and intra-class similarities. However, seeking an optimal combination of visual features which is generic yet adaptive to different benchmarks is an unsolved problem, and metric learning models easily get over-fitted due to the scarcity of training data in person re-identification. In this paper, we propose two effective structured learning based approaches which explore the adaptive effects of visual features in recognizing persons in different benchmark data sets. Our framework is built on the basis of multiple low-level visual features with an optimal ensemble of their metrics. We formulate two optimization algorithms, CMC triplet and CMC top, which directly optimize evaluation measures commonly used in person re-identification, also known as the Cumulative Matching Characteristic (CMC) curve. The more standard CMC triplet formulation works on the triplet information by maximizing the relative distance between a matched pair and a mismatched pair in each triplet unit. The CMC top formulation, modeled on a structured learning of maximizing the correct identification among top candidates, is demonstrated to be more beneficial to person re-identification by directly optimizing an objective closer to the actual testing criteria. The combination of these factors leads to a person re-identification system which outperforms most existing algorithms. More importantly, we advance state-of-the-art results by improving the rank-1 recognition rates from 40% to 61% on the iLIDS benchmark, 16% to 22% on the PRID2011 benchmark, 43% to 50% on the VIPeR benchmark, 34% to 55% on the CUHK01 benchmark and 21% to 68% on the CUHK03 benchmark.

Region-Based Convolutional Networks for Accurate Object Detection and Segmentation.

Object detection performance, as measured on the canonical PASCAL VOC Challenge datasets, plateaued in the final years of the competition. The best-performing methods were complex ensemble systems that typically combined multiple low-level image features with high-level context. In this paper, we propose a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 50 percent relative to the previous best result on VOC 2012-achieving a mAP of 62.4 percent. Our approach combines two ideas: (1) one can apply high-capacity convolutional networks (CNNs) to bottom-up region proposals in order to localize and segment objects and (2) when labeled training data are scarce, supervised pre-training for an auxiliary task, followed by domain-specific fine-tuning, boosts performance significantly. Since we combine region proposals with CNNs, we call the resulting model an R-CNN or Region-based Convolutional Network. Source code for the complete system is available at http://www.cs.berkeley.edu/~rbg/rcnn.

Relation Based Bayesian Network for NBNN

Under the conditional independence assumption among local features, the Naive Bayes Nearest Neighbor (NBNN) classifier has been recently proposed and performs classification without any training or quantization phases. While the original NBNN shows high classification accuracy without adopting an explicit training phase, the conditional independence among local features is against the compositionality of objects indicating that different, but related parts of an object appear together. As a result, the assumption of the conditional independence weakens the accuracy of classification techniques based on NBNN. In this work, we look into this issue, and propose a novel Bayesian network for an NBNN based classification to consider the conditional dependence among features. To achieve our goal, we extract a high-level feature and its corresponding, multiple low-level features for each image patch. We then represent them based on a simple, two-level layered Bayesian network, and design its classification function considering our Bayesian network. To achieve low memory requirement and fast query-time performance, we further optimize our representation and classification function, named relation-based Bayesian network, by considering and representing the relationship between a high-level feature and its low-level features into a compact relation vector, whose dimensionality is the same as the number of low-level features, e.g., four elements in our tests. We have demonstrated the benefits of our method over the original NBNN and its recent improvement, and local NBNN in two different benchmarks. Our method shows improved accuracy, up to 27% against the tested methods. This high accuracy is mainly due to consideration of the conditional dependences between high-level and its corresponding low-level features.

Prototype-Based Discriminative Feature Learning for Kinship Verification

In this paper, we propose a new prototype-based discriminative feature learning (PDFL) method for kinship verification. Unlike most previous kinship verification methods which employ low-level hand-crafted descriptors such as local binary pattern and Gabor features for face representation, this paper aims to learn discriminative mid-level features to better characterize the kin relation of face images for kinship verification. To achieve this, we construct a set of face samples with unlabeled kin relation from the labeled face in the wild dataset as the reference set. Then, each sample in the training face kinship dataset is represented as a mid-level feature vector, where each entry is the corresponding decision value from one support vector machine hyperplane. Subsequently, we formulate an optimization function by minimizing the intraclass samples (with a kin relation) and maximizing the neighboring interclass samples (without a kin relation) with the mid-level features. To better use multiple low-level features for mid-level feature learning, we further propose a multiview PDFL method to learn multiple mid-level features to improve the verification performance. Experimental results on four publicly available kinship datasets show the superior performance of the proposed methods over both the state-of-the-art kinship verification methods and human ability in our kinship verification task.

Combining segmentation and attention: a new foveal attention model.

Artificial vision systems cannot process all the information that they receive from the world in real time because it is highly expensive and inefficient in terms of computational cost. Inspired by biological perception systems, artificial attention models pursuit to select only the relevant part of the scene. On human vision, it is also well established that these units of attention are not merely spatial but closely related to perceptual objects (proto-objects). This implies a strong bidirectional relationship between segmentation and attention processes. While the segmentation process is the responsible to extract the proto-objects from the scene, attention can guide segmentation, arising the concept of foveal attention. When the focus of attention is deployed from one visual unit to another, the rest of the scene is perceived but at a lower resolution that the focused object. The result is a multi-resolution visual perception in which the fovea, a dimple on the central retina, provides the highest resolution vision. In this paper, a bottom-up foveal attention model is presented. In this model the input image is a foveal image represented using a Cartesian Foveal Geometry (CFG), which encodes the field of view of the sensor as a fovea (placed in the focus of attention) surrounded by a set of concentric rings with decreasing resolution. Then multi-resolution perceptual segmentation is performed by building a foveal polygon using the Bounded Irregular Pyramid (BIP). Bottom-up attention is enclosed in the same structure, allowing to set the fovea over the most salient image proto-object. Saliency is computed as a linear combination of multiple low level features such as color and intensity contrast, symmetry, orientation and roundness. Obtained results from natural images show that the performance of the combination of hierarchical foveal segmentation and saliency estimation is good in terms of accuracy and speed.

Learning a discriminative mid-level feature for action recognition

In this paper, we address the problem of recognizing human actions from videos. Most of the existing approaches employ low-level features (e.g., local features and global features) to represent an action video. However, algorithms based on low-level features are not robust to complex environments such as cluttered background, camera movement and illumination change. Therefore, we propose a novel random forest learning framework to construct a discriminative and informative mid-level feature from low-level features of densely sampled 3D cuboids. Each cuboid is classified by the corresponding random forests with a novel fusion scheme, and the cuboid’s posterior probabilities of all categories are normalized to generate a histogram. After that, we obtain our mid-level feature by concatenating histograms of all the cuboids. Since a single low-level feature is not enough to capture the variations of human actions, multiple complementary low-level features (i.e., optical flow and histogram of gradient 3D features) are employed to describe 3D cuboids. Moreover, temporal context between local cuboids is exploited as another type of low-level feature. The above three low-level features (i.e., optical flow, histogram of gradient 3D features and temporal context) are effectively fused in the proposed learning framework. Finally, the mid-level feature is employed by a random forest classifier for robust action recognition. Experiments on the Weizmann, UCF sports, Ballet, and multi-view IXMAS datasets demonstrate that out mid-level feature learned from multiple low-level features can achieve a superior performance over state-of-the-art methods.

Multifeature analysis and semantic context learning for image classification

This article introduces an image classification approach in which the semantic context of images and multiple low-level visual features are jointly exploited. The context consists of a set of semantic terms defining the classes to be associated to unclassified images. Initially, a multiobjective optimization technique is used to define a multifeature fusion model for each semantic class. Then, a Bayesian learning procedure is applied to derive a context model representing relationships among semantic classes. Finally, this context model is used to infer object classes within images. Selected results from a comprehensive experimental evaluation are reported to show the effectiveness of the proposed approaches.

A novel framework for efficient automated singer identification in large music databases

Over the past decade, there has been explosive growth in the availability of multimedia data, particularly image, video, and music. Because of this, content-based music retrieval has attracted attention from the multimedia database and information retrieval communities. Content-based music retrieval requires us to be able to automatically identify particular characteristics of music data. One such characteristic, useful in a range of applications, is the identification of the singer in a musical piece. Unfortunately, existing approaches to this problem suffer from either low accuracy or poor scalability. In this article, we propose a novel scheme, called Hybrid Singer Identifier (HSI), for efficient automated singer recognition. HSI uses multiple low-level features extracted from both vocal and nonvocal music segments to enhance the identification process; it achieves this via a hybrid architecture that builds profiles of individual singer characteristics based on statistical mixture models. An extensive experimental study on a large music database demonstrates the superiority of our method over state-of-the-art approaches in terms of effectiveness, efficiency, scalability, and robustness.