Hierarchical Sparse Representation Research Articles

Toward Intelligent Cross-Domain Resource Coordinate Scheduling for Satellite Networks

The new generation satellite network is a comprehensive service system that can provide communication, observation, navigation, and other functions. Typically, a system with a specific service function is defined as a domain and the supply and demand relationship of resources, such as communication, storage, and energy resources, are unbalanced among domains. Therefore, it is nontrivial to accurately characterize the cross-domain resource state and coordinate the data transmission policy of interrelated satellites from different domains to make full use of the resources in each domain aiming at improving the resource utilization ratio (RUR) of the whole network. To this end, this paper investigates the cross-domain resource scheduling (CDRS) problem in satellite networks aiming at maximizing the total amount of downloaded transmitted data. We start with the orbit motion law of satellites and construct the satellite orbit feature matrix of each domain to design a hierarchical sparse resource representation (HSRR) scheme to accurately characterize the resource state of each domain in real-time with low complexity. Further, based on the HSRR, we develop a cross-domain dynamic multi-resource scheduling algorithm to solve the CDRS problem by introducing the advantage factor and policy-oriented hyper-parameter. The algorithm can make full use of the resources in each domain to achieve efficient CDRS by dynamically adjusting the data transmission policy of satellites among domains. Simulation results show that the greater the service demand and resource difference among domains, the more significant the performance improvement brought by CDRS, and compared with the existing algorithms, the RUR and resource utilization efficiency have been significantly improved.

Finding motifs using DNA images derived from sparse representations.

Motifs play a crucial role in computational biology, as they provide valuable information about the binding specificity of proteins. However, conventional motif discovery methods typically rely on simple combinatoric or probabilistic approaches, which can be biased by heuristics such as substring-masking for multiple motif discovery. In recent years, deep neural networks have become increasingly popular for motif discovery, as they are capable of capturing complex patterns in data. Nonetheless, inferring motifs from neural networks remains a challenging problem, both from a modeling and computational standpoint, despite the success of these networks in supervised learning tasks. We present a principled representation learning approach based on a hierarchical sparse representation for motif discovery. Our method effectively discovers gapped, long, or overlapping motifs that we show to commonly exist in next-generation sequencing datasets, in addition to the short and enriched primary binding sites. Our model is fully interpretable, fast, and capable of capturing motifs in a large number of DNA strings. A key concept emerged from our approach-enumerating at the image level-effectively overcomes the k-mers paradigm, enabling modest computational resources for capturing the long and varied but conserved patterns, in addition to capturing the primary binding sites. Our method is available as a Julia package under the MIT license at https://github.com/kchu25/MOTIFs.jl, and the results on experimental data can be found at https://zenodo.org/record/7783033.

OwlFusion: Depth-Only Onboard Real-Time 3D Reconstruction of Scalable Scenes for Fast-Moving MAV

Real-time 3D reconstruction combined with MAVs has garnered significant attention in a variety of fields, including building maintenance, geological exploration, emergency rescue, and cultural heritage protection. While MAVs possess the advantages of speed and lightness, they also exhibit strong image blur and limited computational resources. To address these limitations, this paper presents a novel approach for onboard, depth-only, real-time 3D reconstruction capable of accommodating fast-moving MAVs. Our primary contribution is a dense SLAM system that combines surface hierarchical sparse representation and particle swarm pose optimization. Our system enables the robust tracking of high-speed camera motion and facilitates scaling to large scenes without being constrained by GPU memory resources. Our robust camera tracking framework is capable of accommodating fast camera motions and varying environments solely by relying on depth images. Furthermore, by integrating path planning methods, we explore the capabilities of MAV autonomous mapping in unknown environments with restricted lighting. Our efficient reconstruction system is capable of generating highly dense point clouds with resolutions ranging from 2 mm to 8 mm on surfaces of different complexities at rates approaching 30 Hz, fully onboard a MAV. We evaluate the performance of our method on both datasets and real-world platforms and demonstrate its superior accuracy and efficiency compared to existing methods.

High-Dimensional Nonlinear Spatio-Temporal Filtering by Compressing Hierarchical Sparse Cholesky Factors

Spatio-temporal filtering is a common and challenging task in many environmental applications, where the evolution is often nonlinear and the dimension of the spatial state may be very high. We propose a scalable filtering approach based on a hierarchical sparse Cholesky representation of the filtering covariance matrix. At each time point, we compress the sparse Cholesky factor into a dense matrix with a small number of columns. After applying the evolution to each of these columns, we decompress to obtain a hierarchical sparse Cholesky factor of the forecast covariance, which can then be updated based on newly available data. We illustrate the Cholesky evolution via an equivalent representation in terms of spatial basis functions. We also demonstrate the advantage of our method in numerical comparisons, including using a high-dimensional and nonlinear Lorenz model.

Short-Term Electric Load Forecasting With Sparse Coding Methods

Short-term load forecasting is a key task for planning and stability of the current and future distribution grid, as it can significantly contribute to the management of energy market for ancillary services. In this paper we introduce the beneficial properties of applications of sparse representation and corresponding dictionary learning to the net load forecasting problem on a substation level. In this context, sparse representation theory can provide parsimonial predictive models, which become attractive mainly due to their ability to successfully model the input space in a self-learning manner, by interacting between theory, algorithms, and applications. Several techniques are implemented, incorporating numerous dictionary learning and sparse decomposition algorithms, and a hierarchical structured model is proposed. The concept of sparsity in each case is embedded throughout the utilization of different regularization forms which include the <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> , <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> , <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> and <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">tree</sup> norms. The observed superiority of the proposed theory, especially the one which embeds the atoms and corresponding coefficients in a tree structure, stems from the construction of the dictionary so as to represent efficiently the ambient electricity signal space and the consequent extraction of sparse basis-vectors. The performance of each model is evaluated using real hourly load measurements from a high voltage/medium voltage (HV/MV) substation and compared with that of widely used machine learning methods. The provided analytical results, verify the effectiveness of hierarchical sparse representation in short-term load forecasting applications, in terms of common accuracy indices.

Ontology-driven hierarchical sparse coding for large-scale image classification

An ontology-driven hierarchical sparse representation is developed in this paper, which aims to support hierarchical learning for large scale image classification. Firstly, a two-layer ontology (semantic ontology and visual ontology) is built to organize large number of image classes hierarchically, where WordNet is used to construct semantic ontology and deep features extracted by Inception V3 are used to construct visual ontology (visual tree). Secondly, a novel algorithm based on Split Bregman Iteration is developed to learn hierarchical sparse representation, i.e., learning a shared dictionary and a set of class-specified dictionaries depending on the two-layer ontology. For multi-class image classification, a tree classifier is trained according to the two-layer ontology by using the hierarchical sparse representation. Thirdly, for a given test image, multiple paths are simultaneously evaluated to achieve optimal prediction of its class label. Our proposed approach has been evaluated over three benchmark datasets: ILSVRC2010, SUN397, and Caltech256, and the experimental results have demonstrated that our approach is better than the original joint dictionary learning method and can achieve better accuracy compared with other approaches which use handcrafted features.

Gender Identification of Human Cortical 3-D Morphology Using Hierarchical Sparsity.

Difference exists widely in cognition, behavior and psychopathology between males and females, while the underlying neurobiology is still unclear. As brain structure is the fundament of its function, getting insight into structural brain may help us to better understand the functional mechanism of gender difference. Previous structural studies of gender difference in Magnetic Resonance Imaging (MRI) usually focused on gray matter (GM) concentration and structural connectivity (SC), leaving cortical morphology not characterized properly. In this study a large dataset is used to explore whether cortical three-dimensional (3-D) morphology can offer enough discriminative morphological features to effectively identify gender. Data of all available healthy controls (N = 1113) from the Human Connectome Project (HCP) were utilized. We suggested a multivariate pattern analysis method called Hierarchical Sparse Representation Classifier (HSRC) and got an accuracy of 96.77% for gender identification. Permutation tests were used to testify the reliability of gender discrimination (p < 0.001). Cortical 3-D morphological features within the frontal lobe were found the most important contributors to gender difference of human brain morphology. Moreover, we investigated gender discriminative ability of cortical 3-D morphology in predefined Anatomical Automatic Labeling (AAL) and Resting-State Networks (RSN) templates, and found the superior frontal gyrus the most discriminative in AAL and the default mode network the most discriminative in RSN. Gender difference of surface-based morphology was also discussed. The frontal lobe, as well as the default mode network, was widely reported of gender difference in previous structural and functional MRI studies, which suggested that morphology indeed affect human brain function. Our study indicates that gender can be identified on individual level by using cortical 3-D morphology and offers a new approach for structural MRI research, as well as highlights the importance of gender balance in brain imaging studies.

A method of visibility forecast based on hierarchical sparse representation

This paper proposes a visibility forecast method based on hierarchical sparse representations. Firstly, it selects meteorological factors from the data of 138 ground stations located in Beijing, Tianjin and Hebei during the months (Oct.-to-Dec. and January) of years 2002–2016. Then, it uses fuzzy C means algorithm (FCM) to construct historical databases containing 5000 samples. Finally, it takes the meteorological factors corresponding to visibility as the sample of historical databases, and uses a hierarchical sparse representation to predict the visibility of new inputs. Experiment, conducted with the data of European Centre for Medium-Range Weather Forecasts (ECMWF), indicates a better performance of the hierarchical sparse representation in contrast to a sparse representation. And the visibility forecast based on hierarchical sparse representation is better than Beijing Regional Environmental Meteorology Prediction System (BREMPS) and BP neural network. The hierarchical sparse representation is simple and easy to expand, which improves the accuracy and reduce the absolute error, which is convenience for other meteorological analysis.

Hierarchical sparse representation with deep dictionary for multi-modal classification

Sparse representation based classification (SRC) methods have achieved many successes in pattern recognition and machine learning. In such methods, the training samples of all categories are mixed and compose a dictionary to represent the test sample via sparsity constraint. Then, the class with the minimum representation error wins for labeling the test sample. In general, SRC is more flexible and effective than many supervised learning methods. However, in some cases it is unlikely to represent the test sample accurately, which tends to undermine the classification accuracy. To alleviate this issue, we propose a hierarchical sparse representation based classification method by augmenting the single-layer sparse representation into the hierarchical representation with a deep dictionary. Specifically, the features from all training samples are first divided into several groups according to their labels. Then we employ hierarchical clustering in each group and combine them to form a deep dictionary such that the root layer includes only a certain amount of the most representative exemplars while the subsequent layers focus on characterizing the remaining individual information across different groups. Furthermore, we use the layer-after-layer residuals to encode the variation patterns across individuals in different scales. Given the deep dictionary, a hierarchical sparse representation based classification method is presented to determine the label for each test sample by iteratively representing its primary part with the exemplars in different groups but the remaining parts by the variation patterns encoded in different layers. To further improve the classification accuracy and robustness, we extend our method by taking advantage of the complementary information in multi-view features. Experiments on Multiple Features Data Set show promising results compared with the state-of-the-art classification methods.

A Sparse Auto Encoder Deep Process Neural Network Model and its Application

Aiming at the problem of time-varying signal pattern classification, a sparse auto-encoder deep process neural network (SAE-DPNN) is proposed. The input of SAE-DPNN is time-varying process signal and the output is pattern category. It combines the time-varying signal classification method of process neural network (PNN) and the data feature extraction and hierarchical sparse representation mechanism of sparse automatic encoder (SAE). Based on the feedforward PNN model, SAE-DPNN is constructed by stacking the process neurons, SAE network and softmax classifier. It can maintain the time-sequence and structure of the input signal, express and synthesize the process distribution characteristics of multidimensional time-varying signals and their combinations. SAE-DPNN improves the identification of complex features and distinguishes between different types of signals, realizes the direct classification of time-varying signals. In this paper, the feature extraction and representation mechanism of time-varying signal in SAE-DPNN are analyzed, and a specific learning algorithm is given. The experimental results verify the effectiveness of the model and algorithm.

Online Hierarchical Sparse Representation of Multifeature for Robust Object Tracking.

Object tracking based on sparse representation has given promising tracking results in recent years. However, the trackers under the framework of sparse representation always overemphasize the sparse representation and ignore the correlation of visual information. In addition, the sparse coding methods only encode the local region independently and ignore the spatial neighborhood information of the image. In this paper, we propose a robust tracking algorithm. Firstly, multiple complementary features are used to describe the object appearance; the appearance model of the tracked target is modeled by instantaneous and stable appearance features simultaneously. A two-stage sparse-coded method which takes the spatial neighborhood information of the image patch and the computation burden into consideration is used to compute the reconstructed object appearance. Then, the reliability of each tracker is measured by the tracking likelihood function of transient and reconstructed appearance models. Finally, the most reliable tracker is obtained by a well established particle filter framework; the training set and the template library are incrementally updated based on the current tracking results. Experiment results on different challenging video sequences show that the proposed algorithm performs well with superior tracking accuracy and robustness.

基于分层稀疏表示特征学习的高光谱图像分类研究

基于分层稀疏表示特征学习的高光谱图像分类研究

Hierarchical Structured Sparse Representation for T–S Fuzzy Systems Identification

“The curse of dimensionality” has become a significant bottleneck for fuzzy system identification and approximation. In this paper, we cast the Takagi-Sugeno (T-S) fuzzy system identification into a hierarchical sparse representation problem, where our goal is to establish a T-S fuzzy system with a minimal number of fuzzy rules, which simultaneously have a minimal number of nonzero consequent parameters. The proposed method, which is called hierarchical sparse fuzzy inference systems ( H-sparseFIS), explicitly takes into account the block-structured information that exists in the T-S fuzzy model and works in an intuitive way: First, initial fuzzy rule antecedent part is extracted automatically by an iterative vector quantization clustering method; then, with block-structured sparse representation, the main important fuzzy rules are selected, and the redundant ones are eliminated for better model accuracy and generalization performance; moreover, we simplify the selected fuzzy rules consequent with sparse regularization such that more consequent parameters can approximate to zero. This algorithm is very efficient and shows good performance in well-known benchmark datasets and real-world problems.

Hierarchical sparse representation based Multi-Instance Semi-Supervised Learning with application to image categorization

Recent studies have shown that sparse representation (SR) can deal well with many computer vision problems. In this paper, we extend a hierarchical sparse representation algorithm into Multi-Instance Semi-Supervised Learning (MISSL) problem. Specifically, at the instance level, after investigating the properties of true positive instances in depth, we propose a novel instance disambiguation strategy based on sparse representation that can identify the instance confidence value in both positive and unlabeled bags more effectively. At the bag level, in contrast to the traditional k-NN or ε-graph construction methods used in the graph-based semi-supervised learning settings, we propose a weighted multi-instance kernel and a corresponding kernel sparse representation method for robust ℓ1-graph construction. The improved ℓ1-graph that encodes the multi-instance properties can be utilized in the manifold regularization framework for the label propagation. Experimental results on different image data sets have demonstrated that the proposed algorithm outperforms existing multi-instance learning (MIL) algorithms, as well as the MISSL algorithms with the application to image categorization task.

Unified Dictionary Learning and Region Tagging with Hierarchical Sparse Representation

Image patterns at different spatial levels are well organized, such as regions within one image and feature points within one region. These classes of spatial structures are hierarchical in nature. The appropriate integration and utilization of such relationship are important to improve the performance of region tagging. Inspired by the recent advances of sparse coding methods, we propose an approach, called Unified Dictionary Learning and Region Tagging with Hierarchical Sparse Representation. This approach consists of two steps: region representation and region reconstruction. In the first step, rather than using the ℓ1-norm as it is commonly done in sparse coding, we add a hierarchical structure to the process of sparse coding and form a framework of tree-guided dictionary learning. In this framework, the hierarchical structures among feature points, regions, and images are encoded by forming a tree-guided multi-task learning process. With the learned dictionary, we obtain a better representation of training and testing regions. In the second step, we propose to use a sub-hierarchical structure to guide the sparse reconstruction for testing regions, i.e., the structure between regions and images. Thanks to this hierarchy, the obtained reconstruction coefficients are more discriminate. Finally, tags are propagated to testing regions by the learned reconstruction coefficients. Extensive experiments on three public benchmark image data sets demonstrate that the proposed approach has better performance of region tagging than the current state of the art methods.