Multiple Descriptors Research Articles

Video representation by dense trajectories motion map applied to human activity recognition

ABSTRACTThis paper introduces an efficient video representation method based on the dense trajectory motion map (DTM). We utilize the salient features of dense trajectories and motion descriptor to integrate the discriminative information of a video into a map. Firstly, we extract the dense trajectories features by using dense optical flow then multiple descriptors are computed along trajectories to capture appearance and motion information. This result is then integrated into frames difference to integrate entire discriminative information and motion energy to get our first motion map. For the final DTM each generated motion map will be integrated with the absolute frame difference of next two frames till the end of entire video. Finally, we process the resultant DTM by exploring the efficient long-term recurrent convolutional network module for encoding and action label generation. The developed approach is shown better and had comparable recognition results over the existing methods when applied to the publically available human action datasets.

Ensemble of convolutional neural networks for bioimage classification

This work presents a system based on an ensemble of Convolutional Neural Networks (CNNs) and descriptors for bioimage classification that has been validated on different datasets of color images. The proposed system represents a very simple yet effective way of boosting the performance of trained CNNs by composing multiple CNNs into an ensemble and combining scores by sum rule. Several types of ensembles are considered, with different CNN topologies along with different learning parameter sets. The proposed system not only exhibits strong discriminative power but also generalizes well over multiple datasets thanks to the combination of multiple descriptors based on different feature types, both learned and handcrafted. Separate classifiers are trained for each descriptor, and the entire set of classifiers is combined by sum rule. Results show that the proposed system obtains state-of-the-art performance across four different bioimage and medical datasets. The MATLAB code of the descriptors will be available at https://github.com/LorisNanni.

An active contour model for brain magnetic resonance image segmentation based on multiple descriptors

With the increasing use of surgical robots, robust and accurate segmentation techniques for brain tissue in the brain magnetic resonance image are needed to be embedded in the robot vision module. However, the brain magnetic resonance image segmentation results are often unsatisfactory because of noise and intensity inhomogeneity. To obtain accurate segmentation of brain tissue, one new multiphase active contour model, which is based on multiple descriptors mean, variance, and the local entropy, is proposed in this study. The model can bring about a more full description of local intensity distribution. Also, the entropy is introduced to improve the performance of robustness to noise of the algorithm. The segmentation and bias correction for brain magnetic resonance image can be simultaneously incorporated by introducing the bias factor in the proposed approach. At last, three experiments are carried out to test the performance of the method. The results in the experiments show that method proposed in this study performed better than most current methods in regard to accuracy and robustness. In addition, the bias-corrected images obtained by proposed method have better visual effect.

Interactions between predation and disturbances shape prey communities

Ecological disturbances are important drivers of biodiversity patterns. Many biodiversity studies rely on endpoint measurements instead of following the dynamics that lead to those outcomes and testing ecological drivers individually, often considering only a single trophic level. Manipulating multiple factors (biotic and abiotic) in controlled settings and measuring multiple descriptors of multi-trophic communities could enlighten our understanding of the context dependency of ecological disturbances. Using model microbial communities, we experimentally tested the effects of imposed disturbances (i.e. increased dilution simulating density-independent mortality as press or pulse disturbances coupled with resource deprivation) on bacterial abundance, diversity and community structure in the absence or presence of a protist predator. We monitored the communities immediately before and after imposing the disturbance and four days after resuming the pre-disturbance dilution regime to infer resistance and recovery properties. The results highlight that bacterial abundance, diversity and community composition were more affected by predation than by disturbance type, resource loss or the interaction of these factors. Predator abundance was strongly affected by the type of disturbance imposed, causing temporary relief of predation pressure. Importantly, prey community composition differed significantly at different phases, emphasizing that endpoint measurements are insufficient for understanding the recovery of communities.

Size and shape distributions of carbon black aggregates by transmission electron microscopy

Carbon black aggregate size and shape affects its performance in many applications. In this interlaboratory comparison, an industry reference carbon black, SRB8, was analysed with a protocol based on ASTM D3849-14a, a method for morphological characterization of carbon black aggregates using electron microscopy. Multiple descriptor types (size, elongation, ruggedness, plus those of ASTM D3849-14a) were assessed for repeatability, reproducibility, and measurement uncertainties. Carbon black aggregates have been characterized using descriptor correlations: two important such correlations are affinity coefficients and fractal exponents. SRB8 aggregates appear to be self-affine, i.e., their width and length descriptors scale anisotropically. ASTM D3849-14a derived descriptors have low interlaboratory reproducibilities and high measurement uncertainties. When these descriptors are used for projected area-based fractal analysis, the estimated fractal exponents do not have realistic values. However, the use of an average nodule diameter generated self-consistent values of fractal exponents with measurement uncertainties of about 9%. Carbon black aggregates can be categorized using shape descriptors into the categories: spheroidal, ellipsoidal, branched, and linear. These shape categories contribute non-uniformly to descriptor values across their data ranges and lead to multimodal distributions. These findings illustrate the importance of assessing data quality and measurement uncertainty for particle size and shape distributions.

Combining Multiple Image Descriptions for Loop Closure Detection

The success of visual loop closure detection depends on the discrimination ability of the image descriptions. Different sources of image descriptions may carry complementary information as well as redundant information. Though integrating them properly can be beneficial, a main obstacle is the lack of analytical quality indicators to weigh different descriptions jointly. Inspired by the linear discriminant analysis, we propose an efficacy index to evaluate the weighted linear combinations of multiple image descriptions for loop closure detection. When a collection of image descriptions is given, the optimal weights maximizing the efficacy index are deduced analytically. As negative weights may negatively affect the performance of detection, a gradient descent algorithm is further proposed to jointly optimize the nonnegative weights. We use the proposed weighting strategies to combine the image descriptions extracted from multiple local image patches by multiple descriptor extractors. It is experimentally demonstrated that our weighted combinations of image descriptions can greatly improve the performance of loop closure detection by emphasizing informative components and de-emphasizing redundant components.

Contribution for Content based Image Retrieval by Multiple Descriptors

Contribution for Content based Image Retrieval by Multiple Descriptors

Predicting in vivo effect levels for repeat-dose systemic toxicity using chemical, biological, kinetic and study covariates

In an effort to address a major challenge in chemical safety assessment, alternative approaches for characterizing systemic effect levels, a predictive model was developed. Systemic effect levels were curated from ToxRefDB, HESS-DB and COSMOS-DB from numerous study types totaling 4379 in vivo studies for 1247 chemicals. Observed systemic effects in mammalian models are a complex function of chemical dynamics, kinetics, and inter- and intra-individual variability. To address this complex problem, systemic effect levels were modeled at the study-level by leveraging study covariates (e.g., study type, strain, administration route) in addition to multiple descriptor sets, including chemical (ToxPrint, PaDEL, and Physchem), biological (ToxCast), and kinetic descriptors. Using random forest modeling with cross-validation and external validation procedures, study-level covariates alone accounted for approximately 15% of the variance reducing the root mean squared error (RMSE) from 0.96 log10 to 0.85 log10 mg/kg/day, providing a baseline performance metric (lower expectation of model performance). A consensus model developed using a combination of study-level covariates, chemical, biological, and kinetic descriptors explained a total of 43% of the variance with an RMSE of 0.69 log10 mg/kg/day. A benchmark model (upper expectation of model performance) was also developed with an RMSE of 0.5 log10 mg/kg/day by incorporating study-level covariates and the mean effect level per chemical. To achieve a representative chemical-level prediction, the minimum study-level predicted and observed effect level per chemical were compared reducing the RMSE from 1.0 to 0.73 log10 mg/kg/day, equivalent to 87% of predictions falling within an order-of-magnitude of the observed value. Although biological descriptors did not improve model performance, the final model was enriched for biological descriptors that indicated xenobiotic metabolism gene expression, oxidative stress, and cytotoxicity, demonstrating the importance of accounting for kinetics and non-specific bioactivity in predicting systemic effect levels. Herein, we generated an externally predictive model of systemic effect levels for use as a safety assessment tool and have generated forward predictions for over 30,000 chemicals.

Metrics for comparing neuronal tree shapes based on persistent homology.

As more and more neuroanatomical data are made available through efforts such as NeuroMorpho.Org and FlyCircuit.org, the need to develop computational tools to facilitate automatic knowledge discovery from such large datasets becomes more urgent. One fundamental question is how best to compare neuron structures, for instance to organize and classify large collection of neurons. We aim to develop a flexible yet powerful framework to support comparison and classification of large collection of neuron structures efficiently. Specifically we propose to use a topological persistence-based feature vectorization framework. Existing methods to vectorize a neuron (i.e, convert a neuron to a feature vector so as to support efficient comparison and/or searching) typically rely on statistics or summaries of morphometric information, such as the average or maximum local torque angle or partition asymmetry. These simple summaries have limited power in encoding global tree structures. Based on the concept of topological persistence recently developed in the field of computational topology, we vectorize each neuron structure into a simple yet informative summary. In particular, each type of information of interest can be represented as a descriptor function defined on the neuron tree, which is then mapped to a simple persistence-signature. Our framework can encode both local and global tree structure, as well as other information of interest (electrophysiological or dynamical measures), by considering multiple descriptor functions on the neuron. The resulting persistence-based signature is potentially more informative than simple statistical summaries (such as average/mean/max) of morphometric quantities—Indeed, we show that using a certain descriptor function will give a persistence-based signature containing strictly more information than the classical Sholl analysis. At the same time, our framework retains the efficiency associated with treating neurons as points in a simple Euclidean feature space, which would be important for constructing efficient searching or indexing structures over them. We present preliminary experimental results to demonstrate the effectiveness of our persistence-based neuronal feature vectorization framework.

Hierarchical ELM ensembles for visual descriptor fusion

Extreme Learning Machines (ELM) have been successfully applied to variety of classification problems by utilizing a single descriptor type. However, a single descriptor may be insufficient for the visual classification task, due to the high level of intra-class variability coupled with low inter-class distance. Although several studies have investigated methods for combining multiple descriptors by ELM, they predominantly apply a simple concatenation of descriptors before classifying them. This type of descriptor fusion may impose problems of descriptor compatibility, high dimensionality and restricted accuracy. In this paper, we propose a hierarchical descriptors fusion strategy at the decision level (“late-fusion”), which relies on ELM ensembles (ELM-E). The proposed method, denoted as H-ELM-E, effectively combines multiple complementary descriptors by a two-level ELM-E based architecture, which ensures that a more informative descriptors will gain more impact on the final decision. In the first level, a separate ELM-E classifier is trained for every image descriptor. In the second level, the output scores from the previous level are aggregated into the mid-level representation which is conducted to an additional ELM-E classifier. The exhaustive experimental evaluation confirmed that the proposed hierarchical ELM-E based strategy is superior to the single-descriptor methods as well as “early fusion” of multiple descriptors, for the visual classification task. Additionally, it was shown that significant accuracy improvement is achieved by integrating ensembles of ELM as a basic classifier, instead of using a single ELM.

MultiDK: A Multiple Descriptor Multiple Kernel Approach for Molecular Discovery and Its Application to Organic Flow Battery Electrolytes.

We propose a multiple descriptor multiple kernel (MultiDK) method for efficient molecular discovery using machine learning. We show that the MultiDK method improves both the speed and accuracy of molecular property prediction. We apply the method to the discovery of electrolyte molecules for aqueous redox flow batteries. Using multiple-type-as opposed to single-type-descriptors, we obtain more relevant features for machine learning. Following the principle of "wisdom of the crowds", the combination of multiple-type descriptors significantly boosts prediction performance. Moreover, by employing multiple kernels-more than one kernel function for a set of the input descriptors-MultiDK exploits nonlinear relations between molecular structure and properties better than a linear regression approach. The multiple kernels consist of a Tanimoto similarity kernel and a linear kernel for a set of binary descriptors and a set of nonbinary descriptors, respectively. Using MultiDK, we achieve an average performance of r2 = 0.92 with a test set of molecules for solubility prediction. We also extend MultiDK to predict pH-dependent solubility and apply it to a set of quinone molecules with different ionizable functional groups to assess their performance as flow battery electrolytes.

An Implementation of Effective Logo Matching and Detection using Multiple Descriptors to Enhance the Resolution

An Implementation of Effective Logo Matching and Detection using Multiple Descriptors to Enhance the Resolution

Adsorption of CO on Low-Energy, Low-Symmetry Pt Nanoparticles: Energy Decomposition Analysis and Prediction via Machine-Learning Models

We present a systematic analysis of CO adsorption on Pt nanoclusters in the 0.2–1.5 nm size range with the aim of unraveling size-dependent trends and developing predictive models for site-specific adsorption behavior. Using an empirical-potential-based genetic algorithm and density functional theory (DFT) modeling, we show that there exists a size window (40–70 atoms) over which Pt nanoclusters bind CO weakly, the binding energies being comparable to those on (111) or (100) facets. The size-dependent adsorption energy trends are, however, distinctly nonmonotonic and are not readily captured using traditional descriptors such as d-band energies or (generalized) coordination numbers of the Pt binding sites. Instead, by applying machine-learning algorithms, we show that multiple descriptors, broadly categorized as structural and electronic descriptors, are essential for qualitatively capturing the CO adsorption trends. Nevertheless, attaining quantitative accuracy requires further refinement, and we propose...

Adaptive Local Spatiotemporal Features from RGB-D Data for One-Shot Learning Gesture Recognition

Noise and constant empirical motion constraints affect the extraction of distinctive spatiotemporal features from one or a few samples per gesture class. To tackle these problems, an adaptive local spatiotemporal feature (ALSTF) using fused RGB-D data is proposed. First, motion regions of interest (MRoIs) are adaptively extracted using grayscale and depth velocity variance information to greatly reduce the impact of noise. Then, corners are used as keypoints if their depth, and velocities of grayscale and of depth meet several adaptive local constraints in each MRoI. With further filtering of noise, an accurate and sufficient number of keypoints is obtained within the desired moving body parts (MBPs). Finally, four kinds of multiple descriptors are calculated and combined in extended gradient and motion spaces to represent the appearance and motion features of gestures. The experimental results on the ChaLearn gesture, CAD-60 and MSRDailyActivity3D datasets demonstrate that the proposed feature achieves higher performance compared with published state-of-the-art approaches under the one-shot learning setting and comparable accuracy under the leave-one-out cross validation.

Robust camera pose estimation by viewpoint classification using deep learning

Camera pose estimation with respect to target scenes is an important technology for superimposing virtual information in augmented reality (AR). However, it is difficult to estimate the camera pose for all possible view angles because feature descriptors such as SIFT are not completely invariant from every perspective. We propose a novel method of robust camera pose estimation using multiple feature descriptor databases generated for each partitioned viewpoint, in which the feature descriptor of each keypoint is almost invariant. Our method estimates the viewpoint class for each input image using deep learning based on a set of training images prepared for each viewpoint class. We give two ways to prepare these images for deep learning and generating databases. In the first method, images are generated using a projection matrix to ensure robust learning in a range of environments with changing backgrounds. The second method uses real images to learn a given environment around a planar pattern. Our evaluation results confirm that our approach increases the number of correct matches and the accuracy of camera pose estimation compared to the conventional method.

Learning universal multiview dictionary for human action recognition

Recently, many sparse coding based approaches have been proposed for human action recognition. However, most of them focus on learning a discriminative dictionary without explicitly taking into account the common patterns shared among different action classes. In this paper, we propose a novel discriminative dictionary learning framework by formulating a universal dictionary which consists of a shared sub-dictionary and a set of class-specific sub-dictionaries. As a result, inter-class differences can be better characterized with sparse codes obtained from the class-specific dictionaries. In addition, group sparsity and locality constraints are utilized to preserve the relationship and structure among features. In order to leverage the benefits of multiple descriptors, a dictionary is learned for each view, and the corresponding sparse representations of those descriptors are fused in a low dimensional feature space together with temporal information. The experimental results on three challenging datasets demonstrate that our method is able to achieve better performance than a number of state-of-the-art ones.

Performance of a simplified termination of resuscitation rule for adult traumatic cardiopulmonary arrest in the prehospital setting

ObjectiveThe prehospital termination of resuscitation (TOR) guidelines for traumatic cardiopulmonary arrest (TCPA) was proposed in 2003. Its multiple descriptors of cases where efforts can be terminated make it complex to...

Classification of uveitis

SummaryClassification and standardization of uveitis is important, as it enhances the precision and comparability of clinical research from different centers and assists in the development of a complete picture of the course of the disorders and their response to treatment. Uveitis may be classified in a number of ways, according to several systems and multiple descriptors. The uveitis classification includes anterior uveitis (iritis, iridocyclitis, and anterior cyclitis), intermediate uveitis (pars planitis, posterior cyclitis, and hyalitis), posterior uveitis (focal, multifocal, or diffuse choroiditis, chorioretinitis, retinitis, and neuroretinitis) or panuveitis (anterior chamber, vitreous, retina, and choroid).Today's uveitis nomenclature has been revised regarding the anatomical location and the grade of inflammation, and supplemented by the inclusion of definitions for onset, duration and course. While helpful for clinical practice and clinical trials, further work still needs to be carried out concentrating on validated clinical assessment tools for activity and damage for common, specific uveitic entities.

A Bayesian network model for predicting aquatic toxicity mode of action using two dimensional theoretical molecular descriptors

The mode of toxic action (MoA) has been recognized as a key determinant of chemical toxicity, but development of predictive MoA classification models in aquatic toxicology has been limited. We developed a Bayesian network model to classify aquatic toxicity MoA using a recently published dataset containing over one thousand chemicals with MoA assignments for aquatic animal toxicity. Two dimensional theoretical chemical descriptors were generated for each chemical using the Toxicity Estimation Software Tool. The model was developed through augmented Markov blanket discovery from the dataset of 1098 chemicals with the MoA broad classifications as a target node. From cross validation, the overall precision for the model was 80.2%. The best precision was for the AChEI MoA (93.5%) where 257 chemicals out of 275 were correctly classified. Model precision was poorest for the reactivity MoA (48.5%) where 48 out of 99 reactive chemicals were correctly classified. Narcosis represented the largest class within the MoA dataset and had a precision and reliability of 80.0%, reflecting the global precision across all of the MoAs. False negatives for narcosis most often fell into electron transport inhibition, neurotoxicity or reactivity MoAs. False negatives for all other MoAs were most often narcosis. A probabilistic sensitivity analysis was undertaken for each MoA to examine the sensitivity to individual and multiple descriptor findings. The results show that the Markov blanket of a structurally complex dataset can simplify analysis and interpretation by identifying a subset of the key chemical descriptors associated with broad aquatic toxicity MoAs, and by providing a computational chemistry-based network classification model with reasonable prediction accuracy.

Action recognition with spatio-temporal augmented descriptor and fusion method

Action recognition is one of the most popular fields of computer vision, and lots of efforts have been made to improve recognition accuracy. While multiple descriptors are extracted to represent action, the spatio-temporal information is lost. In order to incorporate spatio-temporal information, we propose a novel method called augmented descriptor by adding the information to the original descriptor. As descriptors represent different video features, such as static appearance and motion information, previous methods just concatenate various descriptors. However, we propose a fusion method to boost the recognition accuracy of action recognition. The Multiple Kernel Learning is utilized to fuse different descriptors to get better representation in our fusion method. We also evaluate the contribution of normalization method to recognition accuracy. Our proposed methods are tested on the benchmark datasets, Olympic Sports dataset and HMDB51 dataset. The experimental results show that our approaches outperform the baseline method of improved trajectories and are effective in recognizing various actions.