Sparse Feature Space Research Articles

SparseComm: An Efficient Sparse Communication Framework for Vehicle-Infrastructure Cooperative 3D Detection

Collaborative perception, aiming at achieving a comprehensive perception range through inter-agent communication, faces challenges such as high communication costs and domain gaps between multiple agents. This paper introduces SparseComm, an innovative sparse communication collaborative perception framework designed to mitigate these challenges. SparseComm efficiently operates in sparse feature spaces and aggregates features related to the same objects by a sparse instance communication module. Meanwhile, a sparse 3D cooperation module is incorporated to enhance 3D feature representation during communication, thus improving detection performance. Furthermore, a bounding box restoration module is designed to recover undetected bounding boxes due to feature fusion and to address the quality drop issue caused by domain gaps at minimal additional communication cost. Extensive experiments conducted on the DAIR-V2X and V2XSet demonstrate the efficacy of SparseComm, achieving 47.12% at 211.46 communication bytes on DAIR-V2X and 78.03% at 216.63 communication bytes on V2XSet. Notably, SparseComm reduces communication consumption from 10× to 1000× compared with the prior methods while maintaining the detection performance.

Sparse representation learning using [formula omitted] compressed sensing and rank-revealing QR factorization

Compressive sensing can be conceptualized for a classification problem in the context of representation learning, which is frequently applied for signal reconstruction via a few measurements. A novel compressive sensing-based approach is presented to substantially reduce the number of required samples for classification. This technique considers the pixels of image as sensors and it seeks to identify the most optimum sensors as decision variables in feature space. In this study, Spatial sensor locations are acquired through learning to identify the regions within an image where the most discriminative information for classification is embedded. ℓ1−2 minimization, as nonconvex but Lipschitz continuous, is solved to obtain the least non-zero elements of the full measurement vector to completely reconstruct the discriminative vector in feature space. Optimal sensors are localized from the training-set and subsequent test images are categorized based on learned sensors. The algorithm consists of three primary components: sparse minimization, feature space, and discrimination vector. ℓ1−2 minimization, rank-revealing QR factorization (RRQR) and SVM are considered for enhanced sparsity exploitation, feature space extraction and discrimination vector acquiring, respectively. The proposed method is evaluated on four different experiments and is compared against a state-of-art technique. The results demonstrate the superiority of proposed method to the compared method.

A probabilistic view on modelling weather regimes

AbstractThe statistical modelling of weather regimes encompasses the definition of a framework involving a dimensionality reduction step to reduce the highly sparse feature space of weather anomaly maps, and unsupervised learning techniques to correctly categorize regimes. Across the literature's methodology, the two stages stick to a recurrent scheme: Empirical Orthogonal Functions or Principal Component Analysis are used to assess the former; a standard K‐Means clustering algorithm maps each datapoint to the closest‐matching regime. However, such a combination has to cope with an overall reduction in the modelling accuracy. In our study, we re‐think both the two steps in favour of a more dynamic methodology which we apply to the last 42‐years winters' geopotential anomaly maps in the North Atlantic‐European zone. The dimensionality reduction is tackled by means of Variational Autoencoders, leading to better compressed feature spaces, enhancing the datapoints separability. Finally, we employ two probabilistic clustering methods based on Gaussian and Dirichlet Process mixture models, enabling a more faithful recognition of weather regimes, and allowing to reproduce their dynamics and transitions.

Electronic-Structure Properties from Atom-Centered Predictions of the Electron Density.

The electron density of a molecule or material has recently received major attention as a target quantity of machine-learning models. A natural choice to construct a model that yields transferable and linear-scaling predictions is to represent the scalar field using a multicentered atomic basis analogous to that routinely used in density fitting approximations. However, the nonorthogonality of the basis poses challenges for the learning exercise, as it requires accounting for all the atomic density components at once. We devise a gradient-based approach to directly minimize the loss function of the regression problem in an optimized and highly sparse feature space. In so doing, we overcome the limitations associated with adopting an atom-centered model to learn the electron density over arbitrarily complex data sets, obtaining very accurate predictions using a comparatively small training set. The enhanced framework is tested on 32-molecule periodic cells of liquid water, presenting enough complexity to require an optimal balance between accuracy and computational efficiency. We show that starting from the predicted density a single Kohn-Sham diagonalization step can be performed to access total energy components that carry an error of just 0.1 meV/atom with respect to the reference density functional calculations. Finally, we test our method on the highly heterogeneous QM9 benchmark data set, showing that a small fraction of the training data is enough to derive ground-state total energies within chemical accuracy.

Segmentation of shoulder muscle MRI using a new Region and Edge based Deep Auto-Encoder

Automatic segmentation of shoulder muscle MRI is challenging due to the high variation in muscle size, shape, texture, and spatial position of tears. Manual segmentation of tear and muscle portion is hard, time-consuming, and subjective to pathological expertise. This work proposes a new Region and Edge-based Deep Auto-Encoder (RE-DAE) for shoulder muscle MRI segmentation. The proposed RE-DAE harmoniously employs average and max-pooling operations in the Convolutional encoder and decoder blocks. The DAE’s region and edge-based segmentation encourage the network to extract homogenous and anatomical information, respectively. These two concepts, systematically combined in a DAE, generate a discriminative and sparse hybrid feature space (exploiting both region homogeneity and boundaries). Moreover, the concept of static attention is exploited in the proposed RE-DAE that helps effectively learn the tear region. The performances of the proposed RE-DAE architectures have been tested using a 3D MRI shoulder muscle dataset using the hold-out cross-validation technique. The MRI data has been collected from the Korea University Anam Hospital, Seoul, South Korea. Experimental comparisons have been conducted by employing innovative custom-made and existing pre-trained CNN architectures using transfer learning and fine-tuning. Objective evaluation on the muscle datasets using the proposed SA-RE-DAE showed a dice similarity of 85.58% and 87.07%, an accuracy of 81.57% and 95.58% for tear and muscle regions, respectively. The high visual quality and the objective result suggest that the proposed SA-RE-DAE can correctly segment tear and muscle regions in shoulder muscle MRI for better clinical decisions.

Social Spammer Detection via Convex Nonnegative Matrix Factorization

With the increasing popularity of social network platforms such as Twitter and Sina Weibo, a lot of malicious users, also known as social spammers, disseminate illegal information to normal users. Several approaches are proposed to detect spammers by training a classifier with optimization methods and mainly using content and social following information. Due to the development of spammers’ strategies and the courtesy of some legitimate users, social following information becomes vulnerable to fake by spammers. Meanwhile, the possible social activities and behaviors vary significantly among different users, which leads to a large yet sparse feature space to be modeled by existing approaches. To address issues, in this paper, we propose a new approach named CNMFSD for spammer detection in social networks, which exploits both content information and users interaction relationships in an innovative manner. We have empirically validated the proposed method on a real-world Twitter dataset, and experimental results show that the proposed CNMFSD method improves the detection performance significantly compared with baselines.

Combining Hashing and Abstraction in Sparse High Dimensional Feature Spaces

With the exponential increase in the number of documents available online, e.g., news articles, weblogs, scientific documents, the development of effective and efficient classification methods is needed. The performance of document classifiers critically depends, among other things, on the choice of the feature representation. The commonly used "bag of words" and n-gram representations can result in prohibitively high dimensional input spaces. Data mining algorithms applied to these input spaces may be intractable due to the large number of dimensions. Thus, dimensionality reduction algorithms that can process data into features fast at runtime, ideally in constant time per feature, are greatly needed in high throughput applications, where the number of features and data points can be in the order of millions. One promising line of research to dimensionality reduction is feature clustering. We propose to combine two types of feature clustering, namely hashing and abstraction based on hierarchical agglomerative clustering, in order to take advantage of the strengths of both techniques. Experimental results on two text data sets show that the combined approach uses significantly smaller number of features and gives similar performance when compared with the "bag of words" and n-gram approaches.

Similarity-based machine learning framework for predicting safety signals of adverse drug–drug interactions

Drug–drug interaction (DDI) is a major public health problem contributing to 30% of the unexpected clinical adverse drug events. Informatics-based studies for DDI signal detection have been evolving in the last decade. We aim at providing a boosted machine learning (ML) framework to predict novel DDI safety signals with high precision. We propose a similarity-based machine learning framework called “SMDIP” using DrugBank as one of the most reliable pharmaceutical knowledge bases. For this study, DrugBank provides the latest drug information in terms of DDIs, targets, enzymes, transporters, and carriers. We computed drug–drug similarities using a Russell–Rao measure for the available biological and structural information on DrugBank for representing the sparse feature space. Logistic regression is adopted to conduct DDI classification with a focus on searching for key similarity predictors. Six types of ML models are deployed on the selected DDI key features. Our study reveals that SMDIP has yielded favourable predictive performance compared to relevant studies with results as follows: AUC 76%, precision 82%, accuracy 79%, recall 62%, specificity 90%, and F-measure 78%. To further confirm the reliability and reproducibility of SMDIP, we investigate SMDIP on an unseen subset of direct-acting-antiviral (DAA) drugs for treating hepatitis C infections. Forty novel DAA DDIs are predicted that show consistency with the pharmacokinetic and pharmacodynamic profiles of these drugs. Furthermore, several reports from the pharmacovigilance literature corroborate our framework results. Those evaluations show that SMDIP is a promising framework for uncovering DDIs, which can be multifariously feasible in drug development, postmarketing surveillance, and public health fields.

The Evaluation of Accuracy Performance in an Enhanced Embedded Feature Selection for Unstructured Text Classification

Text documents are unstructured and high dimensional. Effective feature selection is required to select the most important and significant feature from the sparse feature space. Thus, this paper proposed an embedded feature selection technique based on Term Frequency-Inverse Document Frequency (TF-IDF) and Support Vector Machine-Recursive Feature Elimination (SVM-RFE) for unstructured and high dimensional text classificationhis technique has the ability to measure the feature’s importance in a high-dimensional text document. In addition, it aims to increase the efficiency of the feature selection. Hence, obtaining a promising text classification accuracy. TF-IDF act as a filter approach which measures features importance of the text documents at the first stage. SVM-RFE utilized a backward feature elimination scheme to recursively remove insignificant features from the filtered feature subsets at the second stage. This research executes sets of experiments using a text document retrieved from a benchmark repository comprising a collection of Twitter posts. Pre-processing processes are applied to extract relevant features. After that, the pre-processed features are divided into training and testing datasets. Next, feature selection is implemented on the training dataset by calculating the TF-IDF score for each feature. SVM-RFE is applied for feature ranking as the next feature selection step. Only top-rank features will be selected for text classification using the SVM classifier. Based on the experiments, it shows that the proposed technique able to achieve 98% accuracy that outperformed other existing techniques. In conclusion, the proposed technique able to select the significant features in the unstructured and high dimensional text document.

Predicting Math Student Success in the Initial Phase of College With Sparse Information Using Approaches From Statistical Learning

In math teacher education, dropout research relies mostly on frameworks which carry out extensive variable collections leading to a lack of practical applicability. We investigate the completion of a first semester course as a dropout indicator and thereby provide not only good predictions, but also generate interpretable and practicable results together with easy-to-understand recommendations. As proof-of-concept, a sparse feature space together with machine learning methods is used for prediction of dropout, wherein the most predictive features have to be identified. Interpretability can be reached by introducing risk groups for the students. Implications for interventions are discussed.

Prioritizing positive feature values: a new hierarchical feature selection method

In this work we address the problem of feature selection for the classification task in hierarchical and sparse feature spaces, which characterize many real-world applications nowadays. A binary feature space is deemed hierarchical when its binary features are related via generalization-specialization relationships, and is considered sparse when in general the instances contain much fewer “positive” than “negative” feature values. In any given instance, a feature value is deemed positive (negative) when the property associated with the feature has been (has not been) observed for that instance. Although there are many methods for the traditional feature selection problem in the literature, the proper treatment to hierarchical feature structures is still a challenge. Hence, we introduce a novel hierarchical feature selection method that follows the lazy learning paradigm—selecting a feature subset tailored for each instance in the test set. Our strategy prioritizes the selection of features with positive values, since they tend to be more informative—the presence of a relatively rare property is usually a piece of more relevant information than the absence of that property. Experiments on different application domains have shown that the proposed method outperforms previous hierarchical feature selection methods and also traditional methods in terms of predictive accuracy, selecting smaller feature subsets in general.

Enhanced Weight Based Convolutional Neural Network (EWCNN) and Fuzzy Clustering For Semantically Rich Multi-Label Social Emotion Classification

In Recent Years, Social Emotion In Recent Years Acquires Natural Language Processing Researchers’ Attention, Because Of Analyzing User-Generated Emotional Documents On The Web. But, These Emotions Has Noisy Instance Mixed And It Is Great Dispute To Acquire The Textual Meaning Of Short Messages. Definition: In General, Large-Scale Datasets Will Have Many Noisy Data, Which Can’t Be Used Readily And Also It Is Costly, Because Of Ambiguity Of Various Informal Expressions In User-Generated Comments. It Is Very Tedious One To Recognize The Similar User Documents From The Entire Social Media Text Message. Furthermore, Online Comments Are Characteristically Categorized By A Sparse Feature Space, Which Makes The Respective Emotion Classification Task A Complex One. Methodology: Three Major Contributions Were Done In This Work In Order To Rectify These Problems, They Are: Development Of A Novel Mutation Bat Optimization Based Sparse Encoding (MBO-SC) Which Transforming The Sparse Low-Level Features Into Dense High-Level Features, Was The 1st Contribution, Next Is, An Enhanced Weight Based Convolutional Neural Network (EWCNN) To Target-Specific Layer. It Influences The Semantically EWCNN Classifier To Include Semantic Domain Knowledge Into The Neural Network To Bootstrap Its Inference Power And Interpretability. Fuzzy Clustering Algorithm Is Proposed To Minimize The Similarity Among Two Documents. Uses: It Is Quite Constructive In Recommending Products, Collecting Public Opinions, And Predicting Election Results. Proposed Work Is Distinguished With The Existing Methods, With The Metrics Such As: Precision, Recall, Sensitivity, Specificity, F-Measure And Accuracy. From The Experimental Result It Is Confirmed That The Quality Of Learned Semantic Vectors And The Performance Of Social Emotion Classification Can Be Enhanced By Proposed Models.

Sparse feature map-based Markov models for nonlinear fluid flows

Data-driven Markov linear models of nonlinear fluid flows using maps of the state into a sparse feature space are explored in this article. The underlying principle of low-order models for fluid systems is identifying maps to a feature space where the system evolution (a) is simpler and efficient to model accurately and (b) the state can be recovered accurately from the features through inverse mapping. Such methods are useful when real-time models are needed for online decision making from sensor data. The Markov linear approximation is popular as it allows us to leverage the well established linear systems machinery. Examples include the Koopman operator approximation techniques and evolutionary kernel methods in machine learning. The success of these models in approximating nonlinear dynamical systems is tied to the effectiveness of the feature map in accomplishing both (a) and (b) above as long as the system provides a feasible prediction horizon using data. We assess this by performing an in-depth study of two different classes of sparse linear feature transformations of the state: (i) a pure data-driven POD-based projection that uses left singular vectors of the data snapshots – a staple of common Koopman approximation methods such as Dynamic Mode Decomposition (DMD) and its variants such as extended DMD; and (ii) a partially data-driven sparse Gaussian kernel (sGK) regression (a mean sparse Gaussian Process (sGP) predictor). The sGK/sGP regression equivalently represents a projection onto an infinite-dimensional basis characterized by a kernel in the inner product reproducing kernel Hilbert space (RKHS). We are particularly interested in the effectiveness of these linear feature maps for long-term prediction using limited data for three classes of fluid flows with escalating complexity (and decreasing prediction horizons) starting from a limit-cycle attractor in a cylinder wake followed by a transient wake evolution with a shift in the base flow and finally, a continuously evolving buoyant Boussinesq mixing flow with no well-defined base state. The results indicate that a purely data-driven POD map is good for full state reconstruction as long as the basis remains relevant to the predictions whereas the more generic sparse Gaussian Kernel (sGK) basis is less sensitive to the evolution of the dynamics but prone to reconstruction errors from lack of parsimony. Contrastingly, the sGK-maps outperform POD-based maps in learning the transient nonlinear evolution of the state for the same feature dimension in systems that contain a well-defined attractor(s). Consequently, both POD and sGK-maps require additional layer(s) to help mitigate these shortcomings. For example, POD-maps require nonlinear functional extensions for improved feature space predictions whereas sGK-maps require dimensionality reduction to balance the large feature dimension needed for accurate full state reconstruction. However, both classes of multilayer feature maps fail to predict the highly evolving buoyant mixing flow for very different reasons.

Anomaly Detection Based on PMF Encoding and Adversarially Learned Inference

In order to solve the problem of increasing the dimension and sparse feature space caused by the categorization coding method in the existing abnormal traffic detection problem, a coding method based on Probability Mass Function (PMF) is proposed. Secondly, in order to improve the ability of abnormal traffic detection algorithms to identify unknown attack type data and improve detection efficiency, we use Adversarially Learned Inference as the basic detection algorithm. The comparison experiments on the standard dataset show that the proposed method has improved the accuracy and detection efficiency greatly compared with the existing anomaly detection methods.

Sparse feature space representation: A unified framework for semi-supervised and domain adaptation learning

In a semi-supervised domain adaptation (DA) task, one has access to only few labeled target examples. In this case, the success of DA needs the effective utilization of a large number of unlabeled target data to extract more discriminative information that is useful for generalization. To this end, we exploit in this paper the feature space embeddings of the target data as well as multi-source prior models to augment the discrimination space for the target function learning. Therefore, we propose a novel multi-source adaptation learning framework based on Sparse Feature Space Representation (SFSR), or called SFSR-MSAL for short. Specifically, the SFSR algorithm is first presented for the further construction of robust graph, on which the discriminative information can be smoothly propagated into the unlabeled target data by additionally incorporating the geometric structure of the target data. Considering the robustness in the semi-supervised DA, we replace the traditional l2-norm based least squares regression with the l2, 1-norm sparse regression, and then construct the SFSR-graph based semi-supervised DA framework with multi-source adaptation constraints. Our framework is universal and can be easily degraded into semi-supervised learning by just tuning the regularization parameter. Moreover, to select the discriminative SFSR-graph Laplacians, we also introduce the ensemble SFSR-graph Laplacians regularization into SFSR-MSAL, thus further improving the performance of SFSR-MSAL. The validity of our methods including semi-supervised and DA learning are examined by several visual recognition tasks on some benchmark datasets, which demonstrate the superiority of our methods in comparison with other related state-of-the-art algorithms.

Bootstrapping Social Emotion Classification with Semantically Rich Hybrid Neural Networks

Social emotion classification aims to predict the aggregation of emotional responses embedded in online comments contributed by various users. Such a task is inherently challenging because extracting relevant semantics from free texts is a classical research problem. Moreover, online comments are typically characterized by a sparse feature space, which makes the corresponding emotion classification task very difficult. On the other hand, though deep neural networks have been shown to be effective for speech recognition and image analysis tasks because of their capabilities of transforming sparse low-level features to dense high-level features, their effectiveness on emotion classification requires further investigation. The main contribution of our work reported in this paper is the development of a novel model of semantically rich hybrid neural network (HNN) which leverages unsupervised teaching models to incorporate semantic domain knowledge into the neural network to bootstrap its inference power and interpretability. To our best knowledge, this is the first successful work of incorporating semantics into neural networks to enhance social emotion classification and network interpretability. Through empirical studies based on three real-world social media datasets, our experimental results confirm that the proposed hybrid neural networks outperform other state-of-the-art emotion classification methods.

Saliency Analysis via Hyperparameter Sparse Representation and Energy Distribution Optimization for Remote Sensing Images

In an effort to detect the region-of-interest (ROI) of remote sensing images with complex data distributions, sparse representation based on dictionary learning has been utilized, and has proved able to process high dimensional data adaptively and efficiently. In this paper, a visual attention model uniting hyperparameter sparse representation with energy distribution optimization is proposed for analyzing saliency and detecting ROIs in remote sensing images. A dictionary learning algorithm based on biological plausibility is adopted to generate the sparse feature space. This method only focuses on finite features, instead of various considerations of feature complexity and massive parameter tuning in other dictionary learning algorithms. In another portion of the model, aimed at obtaining the saliency map, the contribution of each feature is evaluated in a sparse feature space and the coding length of each feature is accumulated. Finally, we calculate the segmentation threshold using the saliency map and obtain the binary mask to separate the ROI from the original images. Experimental results show that the proposed model achieves better performance in saliency analysis and ROI detection for remote sensing images.

Use of Word Clustering to Improve Emotion Recognition from Short Text

Emotion recognition is an important component of affective computing, and is significant in the implementation of natural and friendly human-computer interaction. An effective approach to recognizing emotion from text is based on a machine learning technique, which deals with emotion recognition as a classification problem. However, in emotion recognition, the texts involved are usually very short, leaving a very large, sparse feature space, which decreases the performance of emotion classification. This paper proposes to resolve the problem of feature sparseness, and largely improve the emotion recognition performance from short texts by doing the following: representing short texts with word cluster features, offering a novel word clustering algorithm, and using a new feature weighting scheme. Emotion classification experiments were performed with different features and weighting schemes on a publicly available dataset. The experimental results suggest that the word cluster features and the proposed weighting scheme can partly resolve problems with feature sparseness and emotion recognition performance.

Short-text feature construction and selection in social media data: a survey

Social networking sites such as Facebook or Twitter attract millions of users, who everyday post an enormous amount of content in the form of tweets, comments and posts. Since social network texts are usually short, learning tasks have to deal with a very high dimensional and sparse feature space, in which most features have low frequencies. As a result, extracting useful knowledge from such noisy data is a challenging task, that converts large-scale short-text learning tasks in social environments into one of the most relevant problems in machine learning and data mining. Feature selection is one of the most known and commonly used techniques for reducing the impact of the high dimensional feature space in text learning. A wide variety of feature selection techniques can be found in the literature applied to traditional, long-texts and document collections. However, short-texts coming from the social Web pose new challenges to this well-studied problem as texts’ shortness offers a limited context to extract enough statistical evidence about words relations (e.g. correlation), and instances usually arrive in continuous streams (e.g. Twitter timeline), so that the number of features and instances is unknown, among other problems. This paper surveys feature selection techniques for dealing with short texts in both offline and online settings. Then, open issues and research opportunities for performing online feature selection over social media data are discussed.

Appearance-based gaze estimation using deep features and random forest regression

Conventional appearance-based gaze estimation methods employ local or global features as eye gaze appearance descriptor. But these methods don’t work well under natural light with free head movement. To solve this problem, we present an appearance-based gaze estimation method using deep feature representation and feature forest regression. The deep feature is learned through hierarchical extraction of deep Convolutional Neural Network (CNN). And random forest regression with cluster-to-classify node splitting rules is used to take advantage of data distribution in sparse feature space. Experimental results demonstrate that the deep feature has a better performance than local features on calibrated gaze regression. The combination of deep features and random forest regression provides an effective solution for gaze estimation in a natural environment.