Subspace Partitioning Research Articles

Attribute Subspace Partitioning with Neural Regression for Contextual Outlier Detection

Outlier detection is a critical machine learning task, especially when dealing with context-driven outliers. Such outliers usually remain undetected by conventional detection methods. Detecting these specific outliers requires deep domain knowledge. This paper introduces an innovative approach to contextual outlier detection, leveraging a hybrid method that combines a neural network-based regression model with traditional techniques. The proposed method enables precisely distinguishing contextual outliers by employing attribute subspace partitioning. The proposed algorithm effectively separates anomalous data points by integrating domain knowledge on attribute-specific contextual behavioral deviations with standard outlier detection methods. This experiment exhibits the effectiveness of the proposed method across five real-world datasets, yielding a notable enhancement in the range of 22% to 45% in the AUC score.

Reconciling business analytics with graphically initialized subspace clustering for optimal nonlinear pricing

The relationship between price and quantity in nonlinear pricing transcends simple proportionality, as conditional rebates and discounts may be contingent upon the quantity of goods or services purchased by consumers. This dynamic introduces significant challenges for both consumers and business operators because incomplete information arises from the inherent uncertainty of consumer behavior. In light of this, the present research elucidates the pursuit of optimal nonlinear pricing strategies by business operators through an innovative data-driven approach. Our contributions encompass two distinctive facets: a novel unsupervised spectral clustering method, termed graphically initialized subspace clustering, and a decision optimization framework. The proposed data-driven method introduces an optimization problem aimed at minimizing subspace partitioning costs, leveraging the efficient utilization of a mixture multivariate skewed t distribution to effectively capture heavy users and to characterize their parametric behavioral patterns. In addition, the decision optimization component builds upon the aforementioned method, employing a convex optimization algorithm to enable seamless modification of attributes in nonlinear pricing, while ensuring revenue consistency during pre- and post-modification. Notably, we substantiate the interpretability and practical applicability of our proposed methodology in the realm of business analytics, through empirical analysis utilizing real-world data obtained from a cellular carrier. The findings of this study confirm the efficacy and viability of our approach in enabling business operators to navigate the complexities of nonlinear pricing optimization with confidence and informed decision-making.

Subspace partitioning in the human prefrontal cortex resolves cognitive interference

The human prefrontal cortex (PFC) constitutes the structural basis underlying flexible cognitive control, where mixed-selective neural populations encode multiple task features to guide subsequent behavior. The mechanisms by which the brain simultaneously encodes multiple task-relevant variables while minimizing interference from task-irrelevant features remain unknown. Leveraging intracranial recordings from the human PFC, we first demonstrate that competition between coexisting representations of past and present task variables incurs a behavioral switch cost. Our results reveal that this interference between past and present states in the PFC is resolved through coding partitioning into distinct low-dimensional neural states; thereby strongly attenuating behavioral switch costs. In sum, these findings uncover a fundamental coding mechanism that constitutes a central building block of flexible cognitive control.

An ensemble framework for microarray data classification based on feature subspace partitioning

Feature selection is exposed to the curse of dimensionality risk, and it is even more exacerbated with high-dimensional data such as microarrays. Moreover, the low-instance/high-feature (LIHF) property of microarray data needs considerable processing time to do some calculations and comparisons among features to choose the best subset of them, which has led to many efforts to subdue the LIHF property of such genomic medicine data. Due to the promising results of the ensemble models in machine learning problems, this paper presents a novel framework, named feature-level aggregation-based ensemble based on overlapped feature subspace partitioning (FLAE-OFSP) for microarray data classification. The proposed ensemble has three main steps: after generating several subsets by the proposed partitioning approach, a feature selection algorithm (i.e., a feature ranker) is applied on each subset, and finally, their results are combined into a single ranked list using six defined aggregation functions. Evaluation of the presented framework based on seven microarray datasets and using four measures, including stability, classification accuracy, runtime, and Modscore shows substantial runtime improvement and also quality results in other evaluated measures compared to individual methods.

Hyperspectral Band Selection via Optimal Combination Strategy

Band selection is one of the main methods of reducing the number of dimensions in a hyperspectral image. Recently, various methods have been proposed to address this issue. However, these methods usually obtain the band subset in the perspective of a locally optimal solution. To achieve an optimal solution with a global perspective, this paper developed a novel method for hyperspectral band selection via optimal combination strategy (OCS). The main contributions are as follows: (1) a subspace partitioning approach is proposed which can accurately obtain the partitioning points of the subspace. This ensures that similar bands can be divided into the same subspace; (2) two candidate representative bands with a large amount of information and high similarity are chosen from each subspace, which can fully represent all bands in the subspace; and (3) an optimal combination strategy is designed to acquire the optimal band subset, which achieves an optimal solution with a global perspective. The results on four public datasets illustrate that the proposed method achieves satisfactory performance against other methods.

DataStorm: Coupled, Continuous Simulations for Complex Urban Environments

Urban systems are characterized by complexity and dynamicity. Data-driven simulations represent a promising approach in understanding and predicting complex dynamic processes in the presence of shifting demands of urban systems. Yet, today’s silo-based, de-coupled simulation engines fail to provide an end-to-end view of the complex urban system, preventing informed decision-making. In this article, we present DataStorm to support integration of existing simulation, analysis and visualization components into integrated workflows. DataStorm provides a flow engine, DataStorm-FE , for coordinating data and decision flows among multiple actors (each representing a model, analytic operation, or a decision criterion) and enables ensemble planning and optimization across cloud resources. DataStorm provides native support for simulation ensemble creation through parameter space sampling to decide which simulations to run, as well as distributed instantiation and parallel execution of simulation instances on cluster resources. Recognizing that simulation ensembles are inherently sparse relative to the potential parameter space, we also present a density-boosting partition-stitch sampling scheme to increase the effective density of the simulation ensemble through a sub-space partitioning scheme, complemented with an efficient stitching mechanism that leverages partial and imperfect knowledge from partial dynamical systems to effectively obtain a global view of the complex urban process being simulated.

Combinational design of heterogeneous lattices with hybrid region stiffness tuning for additive manufacturing

Lattices with engineered properties are significant for engineering applications in the energy, aerospace, optics and medical sectors. Current additive manufacturing (AM) combined with computer-aided design (CAD) methods are not flexible enough to create multi-substructure lattices with locally varying geometrical features and mechanical properties. For example, if the struts of one substructure correspond spatially to the void of another substructure in the hybrid region, the resulting structure is disconnected there. When the number of substructures is large, such a problem becomes inevitable and intractable. Here we propose a new AM & CAD method integrating all the processes of designing a complex lattice structure, such as subspace partitioning, substructure rotation, filling in each subspace and connection, and hybrid region stiffness tuning. This method provides a general framework for designing heterogeneous lattice structures with disorder, gradient and hybrid features. Additionally, we numerically and experimentally investigate the effectiveness of our hybrid region stiffness tuning method. Using this new method, the problem of low volume fraction or disconnection in the hybrid region in multi-substructure lattices is overcome, and the resulting structures can be generated for analysis or fabrication by any suitable AM technique.

Affinity matrix with large eigenvalue gap for graph-based subspace clustering and semi-supervised classification

In the graph-based learning method, the data graph or similarity matrix reveals the relationship between data, and reflects similar attributes within a class and differences between classes. Inspired by Davis–Kahan Theorem that the stability of matrix eigenvector space depends on its spectral distance (i.e. its eigenvalue gap), in this paper, we propose a global local affinity matrix model with low rank subspace sparse representation (GLAM-LRSR) based on global information of eigenvalue gap and local distance between samples. This method approximate the similarity matrix with ideally diagonal block structure from the perspective of maximizing the eigenvalue gap, and the local distance between data is utilized as a regular term to prevent the eigenvalue gap from being too large to ensure the efficacy of similarity matrix. We have shown that the combination of subspace (LRSR) partitioning method such as Sparse Subspace Clustering(SSC) and the similarity matrix constructed by GLAM can improve the accuracy of subspace clustering, and that the similarity matrix constructed by GLAM-LRSR can be successfully applied to graph-based semi-supervised classification task. Our experiments on synthetic data as well as the real-world datasets for face clustering, face recovery and motion segmentation have clearly demonstrate the significant advantages of GLAM-LRSR and its effectiveness.

Automatic Parallel Detection of Neovascularization from Retinal Images Using Ensemble of Extreme Learning Machine.

Retinopathy screening is a non-invasive method to collect retinal images and neovascularization detection from retinal images plays a significant role on the identification and classification of diabetes retinopathy. In this paper, an automatic parallel detection framework for neovascularization with color retinal images using ensemble of extreme learning machine is proposed. The framework employs two Map-Reduce Jobs to extract features and trains Extreme Learning Machine models. Ensemble methods such as bagging, subspace partitioning and cross validating are used to increase the accuracy. The framework is evaluated with retinal images from MESSIDOR database. Experimental results show the framework can improve the detection accuracy, as well as speedup the processing time to 22 times on average.

The Fiedler Vector of a Laplacian Tensor for Hypergraph Partitioning

Based on recent advances in spectral hypergraph theory [L. Qi and Z. Luo, Tensor Anaysis: Spectral Theory and Special Tensors, SIAM, Philadelphia, 2017], we explore the Fiedler vector of an even-uniform hypergraph, which is the Z-eigenvector associated with the second smallest Z-eigenvalue of a normalized Laplacian tensor arising from the hypergraph. Then, we develop a novel tensor-based spectral method for partitioning vertices of the hypergraph. For this purpose, we extend the normalized Laplacian matrix of a simple graph to the normalized Laplacian tensor of an even-uniform hypergraph. The corresponding Fiedler vector is related to the Cheeger constant of the hypergraph. Then, we establish a feasible optimization algorithm to compute the Fiedler vector according to the normalized Laplacian tensor. The convergence of the proposed algorithm and the probability of obtaining the Fiedler vector of the hypergraph are analyzed theoretically. Finally, preliminary numerical experiments illustrate that the new approach based on a hypergraph-based Fiedler vector is effective and promising for some combinatorial optimization problems arising from subspace partitioning and face clustering.

A hybrid classification algorithm by subspace partitioning through semi-supervised decision tree

Among data mining techniques, the decision tree is one of the more widely used methods for building classification models in the real world because of its simplicity and ease of interpretation. However, the method has some drawbacks, including instability, the nonsmooth nature of the decision boundary, and the possibility of overfitting. To overcome these problems, several works have utilized the relative advantages of other classifiers, such as logistic regression, support vector machine, and neural networks, in combination with a decision tree, in hybrid models which avoid the drawbacks of other models. Some hybrid models have used decision trees to quickly and efficiently partition the input space, and many studies have proved the effectiveness of the hybrid methods. However, there is room for further improvement by considering the topological properties of a dataset, because typical decision trees split nodes based only on the target variable. The proposed semi-supervised decision tree splits internal nodes by utilizing both labels and the structural characteristics of data for subspace partitioning, to improve the accuracy of classifiers applied to terminal nodes in the hybrid models. Experimental results confirm the superiority of the proposed algorithm and demonstrate the detailed characteristics of the algorithm.

ASP: Abstraction Subspace Partitioning for Detection of Atomicity Violations with an Empirical Study

Dynamic concurrency bug detectors predict and then examine suspicious instances of atomicity violations from executions of multithreaded programs. Only few predicted instances are real bugs. Prioritizing such instances can make the examinations cost-effective, but is there any design factor exhibiting significant influence? This work presents the first controlled experiment that studies two design factors, abstraction level and subspace, in partitioning such instances through 35 resultant partition-based techniques on 10 benchmarks with known vulnerability-related bugs. The empirical analysis reveals significant findings. First, partition-based prioritization can significantly improve the fault detection rate. Second, coarse-grained techniques are more effective than fine-grained ones, and using some one-dimensional subspaces is more effective than using other dimensional subspaces. Third, eight previously unknown techniques can be more effective than the technique modeled after a state-of-the-art dynamic detector.

Parallel ensemble of online sequential extreme learning machine based on MapReduce

In this era of big data, analyzing large scale data efficiently and accurately has become a challenging problem. As one of the ELM variants, online sequential extreme learning machine (OS-ELM) provides a method to analyze incremental data. Ensemble methods provide a way to learn from data more accurately. MapReduce, which provides a simple, scalable and fault-tolerant framework, can be utilized for large scale learning. In this paper, we first propose an ensemble OS-ELM framework which supports any combination of bagging, subspace partitioning and cross validation. Then we design a parallel ensemble of online sequential extreme learning machine (PEOS-ELM) algorithm based on MapReduce for large scale learning. PEOS-ELM algorithm is evaluated with real and synthetic data with the maximum number of training data 5120K and the maximum number of attributes 512. The speedup of this algorithm reaches as high as 40 on a cluster with maximum 80 cores. The accuracy of PEOS-ELM algorithm is at the same level as that of ensemble OS-ELM executing on a single machine, which is higher than that of the original OS-ELM.

Subspace Partitioning for Target Detection and Identification

Detection of a given target or set of targets from observed data is a problem countered in many applications. Regardless of the algorithm selected, detection performance can be severely degraded when the subspace defined by the target data set is singular or ill conditioned. High correlations between target components and their linear combinations lead to false positives and misidentifications, especially for subspace-based detectors. In this paper, we propose a subspace partitioning scheme that allows for detection to be performed in a number of better conditioned subspaces instead of the original subspace. The proposed technique is applied to Raman spectroscopic data analysis. Through both simulation and experimental results, we demonstrate the improvement in the overall detection performance when using the proposed subspace partitioning scheme in conjunction with several subspace detection methods that are commonly used in practice.

The GC-tree: a high-dimensional index structure for similarity search in image databases

We propose a new dynamic index structure called the GC-tree (or the grid cell tree) for efficient similarity search in image databases. The GC-tree is based on a special subspace partitioning strategy which is optimized for a clustered high-dimensional image dataset. The basic ideas are threefold: 1) we adaptively partition the data space based on a density function that identifies dense and sparse regions in a data space; 2) we concentrate the partition on the dense regions, and the objects in the sparse regions of a certain partition level are treated as if they lie within a single region; and 3) we dynamically construct an index structure that corresponds to the space partition hierarchy. The resultant index structure adapts well to the strongly clustered distribution of high-dimensional image datasets. To demonstrate the practical effectiveness of the GC-tree, we experimentally compared the GC-tree with the IQ-tree, LPC-file, VA-file, and linear scan. The result of our experiments shows that the GC-tree outperforms all other methods.

A neural network approach to statistical pattern classification by 'semiparametric' estimation of probability density functions

A method for designing near-optimal nonlinear classifiers, based on a self-organizing technique for estimating probability density functions when only weak assumptions are made about the densities, is described. The method avoids disadvantages of other existing methods by parametrizing a set of component densities from which the actual densities are constructed. The parameters of the component densities are optimized by a self-organizing algorithm, reducing to a minimum the labeling of design samples. All the required computations are realized with the simple sum-of-product units commonly used in connectionist models. The density approximations produced by the method are illustrated in two dimensions for a multispectral image classification task. The practical use of the method is illustrated by a small speech recognition problem. Related issues of invariant projections, cross-class pooling of data, and subspace partitioning are discussed.