High-dimensional Data Vectors Research Articles

Assessing the effectiveness of feature groups in author recognition tasks with the SOM model

The present paper focuses on studying the effectiveness of the self-organizing map (SOM) when applied to the task of categorizing a corpus of texts according to the style of their authors. This task is of particular importance for information retrieval applications using very large databases of documents. The emphasis of this article is to determine the extent to which the SOM possesses the ability to analyze such data, successfully uncovering the stylistic differences among authors in an unsupervised manner. To that end, a variety of feature vectors are studied, each of which either 1) comprises a single category of linguistic features or 2) spans several different categories of linguistic features, in order to determine the effectiveness of each feature category. It is shown that the highest accuracy is achieved when using a vector covering multiple linguistic categories. A comparison of the results obtained to the results of statistical methods indicates the ability of the SOM network to reveal the clustering potential of isolated parameter groups and its effectiveness in handling efficiently high-dimensional data vectors. Potential extensions to related text-organization techniques, such as the WEBSOM, thus become evident.

Supervised model-based visualization of high-dimensional data

When high-dimensional data vectors are visualized on a two- or three-dimensional display, the goal is that two vectors close to each other in the multi-dimensional space should also be close to each other in the low-dimensional space. Traditionally, closeness is defined in terms of some standard geometric distance measure, such as the Euclidean distance, based on a more or less straightforward comparison between the contents of the data vectors. However, such distances do not generally reflect properly the properties of complex problem domains, where changing one bit in a vector may completely change the relevance of the vector. What is more, in real-world situations the similarity of two vectors is not a universal property: even if two vectors can be regarded as similar from one point of view, from another point of view they may appear quite dissimilar. In order to capture these requirements for building a pragmatic and flexible similarity measure, we propose a data visualization scheme where the similarity of two vectors is determined indirectly by using a formal model of the problem domain; in our case, a Bayesian network model. In this scheme, two vectors are considered similar if they lead to similar predictions, when given as input to a Bayesian network model. The scheme is supervised in the sense that different perspectives can be taken into account by using different predictive distributions, i.e., by changing what is to be predicted. In addition, the modeling framework can also be used for validating the rationality of the resulting visualization. This model-based visualization scheme has been implemented and tested on real-world domains with encouraging results.

Dimension Reduction by Local Principal Component Analysis

Reducing or eliminating statistical redundancy between the components of high-dimensional vector data enables a lower-dimensional representation without significant loss of information. Recognizing the limitations of principal component analysis (PCA), researchers in the statistics and neural network communities have developed nonlinear extensions of PCA. This article develops a local linear approach to dimension reduction that provides accurate representations and is fast to compute. We exercise the algorithms on speech and image data, and compare performance with PCA and with neural network implementations of nonlinear PCA. We find that both nonlinear techniques can provide more accurate representations than PCA and show that the local linear techniques outperform neural network implementations.