Different Subsets Of Dimensions Research Articles

A meta-heuristic density-based subspace clustering algorithm for high-dimensional data

Subspace clustering is one of the efficient techniques for determining the clusters in different subsets of dimensions. Ideally, these techniques should find all possible non-redundant clusters in which the data point participates. Unfortunately, existing hard subspace clustering algorithms fail to satisfy this property. Additionally, with the increase in dimensions of data, classical subspace algorithms become inefficient. This work presents a new density-based subspace clustering algorithm (S_FAD) to overcome the drawbacks of classical algorithms. S_FAD is based on a bottom-up approach and finds subspace clusters of varied density using different parameters of the DBSCAN algorithm. The algorithm optimizes parameters of the DBCAN algorithm through a hybrid meta-heuristic algorithm and uses hashing concepts to discover all non-redundant subspace clusters. The efficacy of S_FAD is evaluated against various existing subspace clustering algorithms on artificial and real datasets in terms of F_Score and rand_index. Performance is assessed based on three parameters: average ranking, SRR ranking, and scalability on varied dimensions. Statistical analysis is performed through the Wilcoxon signed-rank test. Results reveal that S_FAD performs considerably better on the majority of the datasets and scales well up to 6400 dimensions on the actual dataset.

Segmentation-based clustering of hyperspectral images using local band selection

This study addresses the problems associated with high dimensionality of hyperspectral images in reference to clustering. A new local band selection approach that takes both relevancy and redundancy among the bands into account while obtaining the multiple relevant set of bands is developed. The local band selection approach is then incorporated within a multistage clustering framework that includes three stages: segmentation, region merging, and projected clustering. At first, k-means is used to produce initial segments/regions. Then, in the region merging stage, the modified local mutually best region merging strategy is applied on the initial segments to produce the refined segmentation map. Finally, an improved projected clustering technique is used to group these segments into a fixed number of clusters. Further, the main principle of projected clustering, that different sets of point may cluster better for different subsets of dimensions, is extended to region merging by incorporating the suggested local band selection approach. The framework requires input for two major parameters, which are number of clusters (k) and number of relevant bands (l). The framework is tested over two hyperspectral images and compared with other clustering frameworks. The experimental results confirm the effectiveness of the proposed framework.

A novel algorithm for fast and scalable subspace clustering of high-dimensional data

Rapid growth of high dimensional datasets in recent years has created an emergent need to extract the knowledge underlying them. Clustering is the process of automatically finding groups of similar data points in the space of the dimensions or attributes of a dataset. Finding clusters in the high dimensional datasets is an important and challenging data mining problem. Data group together differently under different subsets of dimensions, called subspaces. Quite often a dataset can be better understood by clustering it in its subspaces, a process called subspace clustering. But the exponential growth in the number of these subspaces with the dimensionality of data makes the whole process of subspace clustering computationally very expensive. There is a growing demand for efficient and scalable subspace clustering solutions in many Big data application domains like biology, computer vision, astronomy and social networking. Apriori based hierarchical clustering is a promising approach to find all possible higher dimensional subspace clusters from the lower dimensional clusters using a bottom-up process. However, the performance of the existing algorithms based on this approach deteriorates drastically with the increase in the number of dimensions. Most of these algorithms require multiple database scans and generate a large number of redundant subspace clusters, either implicitly or explicitly, during the clustering process. In this paper, we present SUBSCALE, a novel clustering algorithm to find non-trivial subspace clusters with minimal cost and it requires only k database scans for a k-dimensional data set. Our algorithm scales very well with the dimensionality of the dataset and is highly parallelizable. We present the details of the SUBSCALE algorithm and its evaluation in this paper.

Ranking the sky: Discovering the importance of skyline points through subspace dominance relationships

Skyline queries aim to help users make intelligent decisions over complex data by discovering a set of interesting points, when different and often conflicting criteria are considered. Unfortunately, as the dimensionality of the dataset grows, the skyline operator loses its discriminating power and returns a large fraction of the data. The huge size of the result set hinders decision-making and motivates the ranking of skyline points. Therefore, users prefer to retrieve the top- k skyline points instead of the whole skyline set. In this paper, we propose SKYRANK, a framework for ranking the skyline points in the absence of a user-defined preference function, thereby discovering a limited subset of the most interesting points of the skyline set. For this purpose, we define the skyline graph, which relies on the dominance relationships between the skyline points for different subsets of dimensions (subspaces). SKYRANK applies well-known authority-based ranking algorithms on the skyline graph and, as described in this paper, discovers the importance of a skyline point exploiting the subspace dominance relationships. Furthermore, we extend SKYRANK to handle top- k preference skyline queries, when the user's preferences are available. Our experimental evaluation illustrates the complexity of the dominance relationships and the ranking ability of our framework.

Efficiently Mining Time-Delayed Gene Expression Patterns

Unlike pattern-based biclustering methods that focus on grouping objects in the same subset of dimensions, in this paper, we propose a novel model of coherent clustering for time-series gene expression data, i.e., time-delayed cluster (td-cluster). Under this model, objects can be coherent in different subsets of dimensions if these objects follow a certain time-delayed relationship. Such a cluster can discover the cycle time of gene expression, which is essential in revealing gene regulatory networks. This paper is the first attempt to mine time-delayed gene expression patterns from microarray data. A novel algorithm is also presented and implemented to mine all significant td-clusters. Our experimental results show following two results: 1) the td-cluster algorithm can detect a significant amount of clusters that were missed by previous models, and these clusters are potentially of high biological significance and 2) the td-cluster model and algorithm can easily be extended to 3-D gene x sample x time data sets to identify 3-D td-clusters.

Fast algorithms for projected clustering

The clustering problem is well known in the database literature for its numerous applications in problems such as customer segmentation, classification and trend analysis. Unfortunately, all known algorithms tend to break down in high dimensional spaces because of the inherent sparsity of the points. In such high dimensional spaces not all dimensions may be relevant to a given cluster. One way of handling this is to pick the closely correlated dimensions and find clusters in the corresponding subspace. Traditional feature selection algorithms attempt to achieve this. The weakness of this approach is that in typical high dimensional data mining applications different sets of points may cluster better for different subsets of dimensions. The number of dimensions in each such cluster-specific subspace may also vary. Hence, it may be impossible to find a single small subset of dimensions for all the clusters. We therefore discuss a generalization of the clustering problem, referred to as the projected clustering problem , in which the subsets of dimensions selected are specific to the clusters themselves. We develop an algorithmic framework for solving the projected clustering problem, and test its performance on synthetic data.

Alternative Schemes for the Analysis off Basic Versus Applied Science: Beauty is Still in the Eye of the Beholder

Hoffman and Deffenbacher (1993) presented a multidimensional scheme for analyzing the relations of basic and applied science, reliant on concepts from ecological psychology. In his commentary, Vicente (1994) argued, among other things, that the scheme can profitably be simplified. Although we are sympathetic to Vicente's viewpoint and some of his goals, differences between Vicente's prescriptive purposes (the transformation of traditional human factors psychology) and the descriptive purposes of Hoffman and Deffenbacher (the empirical analysis of sciences in general) suggest that different subsets of dimensions may be useful in the analyses of different fields of science. In addition, if one desires to be prescriptive, one can perhaps successfully focus on particular dimensions.