Object-level Features Research Articles

Prediction of Heart Transplant Rejection Using Histopathological Whole-Slide Imaging.

Endomyocardial biopsies are the current gold standard for monitoring heart transplant patients for signs of cardiac allograft rejection. Manually analyzing the acquired tissue samples can be costly, time-consuming, and subjective. Computer-aided diagnosis, using digitized whole-slide images, has been used to classify the presence and grading of diseases such as brain tumors and breast cancer, and we expect it can be used for prediction of cardiac allograft rejection. In this paper, we first create a pipeline to normalize and extract pixel-level and object-level features from histopathological whole-slide images of endomyocardial biopsies. Then, we develop a two-stage classification algorithm, where we first cluster individual tiles and then use the frequency of tiles in each cluster for classification of each whole-slide image. Our results show that the addition of an unsupervised clustering step leads to higher classification accuracy, as well as the importance of object-level features based on the pathophysiology of rejection. Future expansion of this study includes the development of a multiclass classification pipeline for subtypes and grades of cardiac allograft rejection.

GLA: Global–Local Attention for Image Description

In recent years, the task of automatically generating image description has attracted a lot of attention in the field of artificial intelligence. Benefitting from the development of convolutional neural networks (CNNs) and recurrent neural networks (RNNs), many approaches based on the CNN-RNN framework have been proposed to solve this task and achieved remarkable process. However, two problems remain to be tackled in which the most existing methods use only the image-level representation. One problem is object missing, in which some important objects may be missing when generating the image description and the other is misprediction, when one object may be recognized in a wrong category. In this paper, to address these two problems, we propose a new method called global–local attention (GLA) for generating image description. The proposed GLA model utilizes an attention mechanism to integrate object-level features with image-level feature. Through this manner, our model can selectively pay attention to objects and context information concurrently. Therefore, our proposed GLA method can generate more relevant image description sentences and achieve the state-of-the-art performance on the well-known Microsoft COCO caption dataset with several popular evaluation metrics—CIDEr, METEOR, ROUGE-L and BLEU-1, 2, 3, 4.

Salient object detection via boosting object-level distinctiveness and saliency refinement

Many salient object detection approaches share the common drawback that they cannot uniformly highlight heterogeneous regions of salient objects, and thus, parts of the salient objects are not discriminated from background regions in a saliency map. In this paper, we focus on this drawback and accordingly propose a novel algorithm that more uniformly highlights the entire salient object as compared to many approaches. Our method consists of two stages: boosting the object-level distinctiveness and saliency refinement. In the first stage, a coarse object-level saliency map is generated based on boosting the distinctiveness of the object proposals in the test images, using a set of object-level features and the Modest AdaBoost algorithm. In the second stage, several saliency refinement steps are executed to obtain a final saliency map in which the boundaries of salient objects are preserved. Quantitative and qualitative comparisons with state-of-the-art approaches demonstrate the superior performance of our approach.

Automated grading of breast cancer histopathology using cascaded ensemble with combination of multi-level image features

We present a novel image-analysis based method for automatically distinguishing low, intermediate, and high grades of breast cancer in digitized histopathology. A multiple level feature set, including pixel-, object-, and semantic-level features derived from convolutional neural networks (CNN), is extracted from 106 hematoxylin and eosin stained breast biopsy tissue studies from 106 women patients. These multi-level features allow not only characterization of cancer morphology, but also extraction of structural and interpretable information within the histopathological images. In this study, an improved hybrid active contour model based segmentation method was used to segment nuclei from the images. The semantic-level features were extracted by a CNN approach, which described the proportions of nuclei belonging to the different grades, in conjunction with pixel-level (texture) and object-level (architecture) features, to create an integrated set of image attributes that can potentially outperform either subtype of features individually. We utilized a cascaded approach to train multiple support vector machine (SVM) classifiers using combinations of feature subtypes to enable the possibility of maximizing the performance by leveraging different feature sets extracted from multiple levels. Final class (cancer grade) was determined by combining the scores produced by the individual SVM classifiers. By employing a light (three-layer) CNN model and parallel computing, the presented approach is computationally efficient and applicable to large-scale datasets. The method achieved an accuracy of 0.92 for low versus high, 0.77 for low versus intermediate, and 0.76 for intermediate versus high, and an overall accuracy of 0.69 when discriminating low, intermediate, and high grades of histopathological breast cancer images. This suggested that our grading method could be useful in developing a computational diagnostic tool for differentiating breast cancer grades, which might enable objective and reproducible alternative for diagnosis.

A Study on Nuclei Segmentation, Feature Extraction and Disease Stage Classification for Human Brain Histopathological Images

Computer aided diagnosis (CAD) systems are important in obtaining precision medicine and patient driven solutions for various diseases. One of the main brain tumor is the Glioblastoma multiforme (GBM) and histopathological tissue images can provide unique insights into identifying and grading disease stages. In this study, we consider nuclei segmentation method, feature extraction and disease stage classification for brain tumor histopathological images using automatic image analysis methods. In particular we utilized automatic nuclei segmentation and labeling for histopathology image data obtained from The Cancer Genome Atlas (TCGA) and check for significance of feature descriptors using K-S test and classification accuracy using support vector machine (SVM) and Random Forests (RF). Our results indicate that we obtain classification accuracy 98.6% and 99.8% in the case of Object-Level features and 82.1% and 86.1% in the case of Spatial Arrangement features, respectively.

Predicting Eye Fixations With Higher-Level Visual Features

Saliency map and object map are the two contrasting hypotheses for the mechanisms utilized by the visual system to guide eye fixations when humans are freely viewing natural images. Most computational studies define saliency as outliers of distributions of low-level features, and propose saliency as an important factor for predicting eye fixations. Psychophysical studies, however, suggest that high-level objects predict eye fixations more accurately and the early saliency only has a minor effect. But this view has been challenged by a study which shows opposite results, suggesting that the role of object-level features needs further investigations. In addition, little is known about the role of intermediate features between the low-level and the object-level features. In this paper, we construct two models based on mid-level and object-level features, respectively, and compare their performances against those based on low-level features. Quantitative evaluation on three benchmark natural image fixation data sets demonstrates that the mid-level model outperforms the state-of-the-art low-level models by a significant margin and the object-level model is inferior to most low-level models. Quantitative evaluation on a video fixation data set demonstrates that both the mid-level and object-level models outperform the state-of-the-art low-level models, and the latter performs better under three out of four standard metrics. When combined together the two proposed models achieve even higher performance. However, incorporating the best low-level model yields negligible improvements on all of the data sets. Taken together, these results indicate that higher level features may be more effective than low-level features for predicting eye fixations on natural images in the free viewing condition.

An Object-Level Feature Representation Model for the Multi-target Retrieval of Remote Sensing Images

To address the problem of multi-target retrieval (MTR) of remote sensing images, this study proposes a new object-level feature representation model. The model provides an enhanced application image representation that improves the efficiency of MTR. Generating the model in our scheme includes processes, such as object-oriented image segmentation, feature parameter calculation, and symbolic image database construction. The proposed model uses the spatial representation method of the extended nine-direction lower-triangular (9DLT) matrix to combine spatial relationships among objects, and organizes the image features according to MPEG-7 standards. A similarity metric method is proposed that improves the precision of similarity retrieval. Our method provides a trade-off strategy that supports flexible matching on the target features, or the spatial relationship between the query target and the image database. We implement this retrieval framework on a dataset of remote sensing images. Experimental results show that the proposed model achieves competitive and high-retrieval precision.

A Petri Net-based Approach for Event Detection in Pedestrian Crossing Sequence

In this paper we present an approach to automatically detect anomalous traffic events like pedestrians crossing the junction, based on traffic video of low-level features such as size of the blob, spatial location, and velocity. The construction of Petri-Nets was used for both semantic description and event detection within traffic videos. The major novelties of this paper are extensions to both the modeling and the recognition capacities of Object Petri-Nets (PN). The detection of object level features are done with the help of state of art techniques like Gaussian Mixture of Models (GMM), and a series of Petri-Nets composed of various objects is proposed to describe the video content. The expected outcome of the proposed framework is that we can easily build semantic detectors based on PNs to search within traffic videos and identify interesting events. Experimental results based on recorded traffic video data sets and synthetic data sets are used to illustrate the potential of this framework.

Automated Detection of Synapses in Serial Section Transmission Electron Microscopy Image Stacks

We describe a method for fully automated detection of chemical synapses in serial electron microscopy images with highly anisotropic axial and lateral resolution, such as images taken on transmission electron microscopes. Our pipeline starts from classification of the pixels based on 3D pixel features, which is followed by segmentation with an Ising model MRF and another classification step, based on object-level features. Classifiers are learned on sparse user labels; a fully annotated data subvolume is not required for training. The algorithm was validated on a set of 238 synapses in 20 serial 7197×7351 pixel images (4.5×4.5×45 nm resolution) of mouse visual cortex, manually labeled by three independent human annotators and additionally re-verified by an expert neuroscientist. The error rate of the algorithm (12% false negative, 7% false positive detections) is better than state-of-the-art, even though, unlike the state-of-the-art method, our algorithm does not require a prior segmentation of the image volume into cells. The software is based on the ilastik learning and segmentation toolkit and the vigra image processing library and is freely available on our website, along with the test data and gold standard annotations (http://www.ilastik.org/synapse-detection/sstem).

Segmentation of cell nuclei in heterogeneous microscopy images: A reshapable templates approach

Histological tissue images typically exhibit very sophisticated spatial color patterns. It is of great clinical importance to extract qualitative and quantitative information from these images. As an ad hoc solution, various unsupervised approaches address the object detection and segmentation problem which are suitable for limited classes of histology images. In this paper, we propose a general purpose localization and segmentation method which utilizes reshapable templates. The method combines both pixel- and object-level features for detecting regions of interest. Segmentation is carried out in two levels including both the coarse and fine ones. A set of simple-shaped templates is used for coarse segmentation. A content based template reshaping algorithm is proposed for fine segmentation of target objects. Experimentation was done using a publicly available image data set which contains 7931 manually labeled cells of heterogeneous histology images. The experiments have demonstrated acceptable level of detection and segmentation results for the proposed approach (precision=0.904, recall=0.870 and Zijdenbos similarity index=73%). Thus, the prototype software developed based on proposed method can be considered as a potential tool for pathologists in clinical process.

The perirhinal cortex modulates V2 activity in response to the agreement between part familiarity and configuration familiarity

Research has demonstrated that the perirhinal cortex (PRC) represents complex object-level feature configurations, and participates in familiarity versus novelty discrimination. Barense et al. [(in press) Cerebral Cortex, 22:11, doi:10.1093/cercor/bhr347] postulated that, in addition, the PRC modulates part familiarity responses in lower-level visual areas. We used fMRI to measure activation in the PRC and V2 in response to silhouettes presented peripherally while participants maintained central fixation and performed an object recognition task. There were three types of silhouettes: Familiar Configurations portrayed real-world objects; Part-Rearranged Novel Configurations created by spatially rearranging the parts of the familiar configurations; and Control Novel Configurations in which both the configuration and the ensemble of parts comprising it were novel. For right visual field (RVF) presentation, BOLD responses revealed a significant linear trend in bilateral BA 35 of the PRC (highest activation for Familiar Configurations, lowest for Part-Rearranged Novel Configurations, with Control Novel Configurations in between). For left visual field (LVF) presentation, a significant linear trend was found in a different area (bilateral BA 38, temporal pole) in the opposite direction (Part-Rearranged Novel Configurations highest, Familiar Configurations lowest). These data confirm that the PRC is sensitive to the agreement in familiarity between the configuration level and the part level. As predicted, V2 activation mimicked that of the PRC: for RVF presentation, activity in V2 was significantly higher in the left hemisphere for Familiar Configurations than for Part-Rearranged Novel Configurations, and for LVF presentation, the opposite effect was found in right hemisphere V2. We attribute these patterns in V2 to feedback from the PRC because receptive fields in V2 encompass parts but not configurations. These results reveal two new aspects of PRC function: (1) it is sensitive to the congruency between the familiarity of object configurations and the parts comprising those configurations and (2) it likely modulates familiarity responses in visual area V2.

A three-level architecture for bridging the image semantic gap

Image retrieval systems face the problem of dealing with the different ways to apprehend the content of images and in particular the difficulty to characterize the visual semantics. To address this issue, we examine the use of three abstract levels of representation, namely Signal, Object and Semantic. At the Signal Level, we propose a framework mapping the extracted low-level features to symbolic signal descriptors. The Object Level features a statistical model considering the joint distribution of object concepts (such as mountains, sky?) and the symbolic signal descriptors. At the Semantic Level, signal and object characterizations are coupled within a logic-based framework. The latter is instantiated by a knowledge representation formalism allowing to define an expressive query language consisting of several boolean and quantification operators. Our architecture therefore makes it possible to process topic-based queries. Experimentally, we evaluate our theoretical proposition on a corpus of real-world photographs and the TRECVid corpus.

Labeling of Lumbar Discs Using Both Pixel- and Object-Level Features With a Two-Level Probabilistic Model

Backbone anatomical structure detection and labeling is a necessary step for various analysis tasks of the vertebral column. Appearance, shape and geometry measurements are necessary for abnormality detection locally at each disc and vertebrae (such as herniation) as well as globally for the whole spine (such as spinal scoliosis). We propose a two-level probabilistic model for the localization of discs from clinical magnetic resonance imaging (MRI) data that captures both pixel- and object-level features. Using a Gibbs distribution, we model appearance and spatial information at the pixel level, and at the object level, we model the spatial distribution of the discs and the relative distances between them. We use generalized expectation-maximization for optimization, which achieves efficient convergence of disc labels. Our two-level model allows the assumption of conditional independence at the pixel-level to enhance efficiency while maintaining robustness. We use a dataset that contains 105 MRI clinical normal and abnormal cases for the lumbar area. We thoroughly test our model and achieve encouraging results on normal and abnormal cases.

Implicit Attentional Selection of Bound Visual Features

Traditionally, research on visual attention has been focused on the processes involved in conscious, explicit selection of task-relevant sensory input. Recently, however, it has been shown that attending to a specific feature of an object automatically increases neural sensitivity to this feature throughout the visual field. Here we show that directing attention to a specific color of an object results in attentional modulation of the processing of task-irrelevant and not consciously perceived motion signals that are spatiotemporally associated with this color throughout the visual field. Such implicit cross-feature spreading of attention takes place according to the veridical physical associations between the color and motion signals, even under special circumstances when they are perceptually misbound. These results imply that the units of implicit attentional selection are spatiotemporally colocalized feature clusters that are automatically bound throughout the visual field.