Multidimensional Scaling Problem Research Articles

A Globally Convergent Inertial First-Order Optimization Method for Multidimensional Scaling

Multidimensional scaling (MDS) is a popular tool for dimensionality reduction and data visualization. Given distances between data points and a target low-dimension, the MDS problem seeks to find a configuration of these points in the low-dimensional space, such that the inter-point distances are preserved as well as possible. We focus on the most common approach to formulate the MDS problem, known as stress minimization, which results in a challenging non-smooth and non-convex optimization problem. In this paper, we propose an inertial version of the well-known SMACOF Algorithm, which we call AI-SMACOF. This algorithm is proven to be globally convergent, and to the best of our knowledge this is the first result of this kind for algorithms aiming at solving the stress MDS minimization. In addition to the theoretical findings, numerical experiments provide another evidence for the superiority of the proposed algorithm.

Metric multidimensional scaling for large single-cell datasets using neural networks

Metric multidimensional scaling is one of the classical methods for embedding data into low-dimensional Euclidean space. It creates the low-dimensional embedding by approximately preserving the pairwise distances between the input points. However, current state-of-the-art approaches only scale to a few thousand data points. For larger data sets such as those occurring in single-cell RNA sequencing experiments, the running time becomes prohibitively large and thus alternative methods such as PCA are widely used instead. Here, we propose a simple neural network-based approach for solving the metric multidimensional scaling problem that is orders of magnitude faster than previous state-of-the-art approaches, and hence scales to data sets with up to a few million cells. At the same time, it provides a non-linear mapping between high- and low-dimensional space that can place previously unseen cells in the same embedding.

On complexity and convergence of high-order coordinate descent algorithms for smooth nonconvex box-constrained minimization

Coordinate descent methods have considerable impact in global optimization because global (or, at least, almost global) minimization is affordable for low-dimensional problems. Coordinate descent methods with high-order regularized models for smooth nonconvex box-constrained minimization are introduced in this work. High-order stationarity asymptotic convergence and first-order stationarity worst-case evaluation complexity bounds are established. The computer work that is necessary for obtaining first-order $\varepsilon$-stationarity with respect to the variables of each coordinate-descent block is $O(\varepsilon^{-(p+1)/p})$ whereas the computer work for getting first-order $\varepsilon$-stationarity with respect to all the variables simultaneously is $O(\varepsilon^{-(p+1)})$. Numerical examples involving multidimensional scaling problems are presented. The numerical performance of the methods is enhanced by means of coordinate-descent strategies for choosing initial points.

Optimizing a polynomial function on a quantum processor

The gradient descent method is central to numerical optimization and is the key ingredient in many machine learning algorithms. It promises to find a local minimum of a function by iteratively moving along the direction of the steepest descent. Since for high-dimensional problems the required computational resources can be prohibitive, it is desirable to investigate quantum versions of the gradient descent, such as the recently proposed (Rebentrost et al.1). Here, we develop this protocol and implement it on a quantum processor with limited resources. A prototypical experiment is shown with a four-qubit nuclear magnetic resonance quantum processor, which demonstrates the iterative optimization process. Experimentally, the final point converged to the local minimum with a fidelity >94%, quantified via full-state tomography. Moreover, our method can be employed to a multidimensional scaling problem, showing the potential to outperform its classical counterparts. Considering the ongoing efforts in quantum information and data science, our work may provide a faster approach to solving high-dimensional optimization problems and a subroutine for future practical quantum computers.

Optimizing the mutual arrangement of pilot indicators on an aircraft dashboard and analysis of this procedure from the viewpoint of quantum representations

The purpose of this work is to present the first attempt to provide quantitative analysis and objective justification for designers' decisions that relate to the arrangement of pilot indicators on an aircraft dashboard with the use of video oculography measurements. To date, such decisions have been made only based on the practical experience accumulated by designers and subjective expert assessments. A new method for optimizing the mutual arrangement of the dashboard indicators is under consideration. This is based on iterative correction of the gaze transition probability matrix between the selected zones of attention, to minimize the difference between the stationary distribution of relative frequencies of gaze that are staying in these zones and the corresponding desirable target eye movements that are given for distribution for qualified pilots. When solving the subsequent multidimensional scaling problem, the gaze transition probability matrix that is obtained is considered to be the similarity matrix, the elements of which quantitatively characterize the proximity between the zones of attention. The main findings of this novel work are as follows: the use of oculography data to justify dashboard design decisions, the optimizing method itself, and its mathematical components, as well as analysis of the optimization in question from the viewpoint of quantum representations, all revealed design mistakes. The results that were obtained can be applied for prototyping variants of aircraft dashboards by rearranging the display areas associated with the corresponding zones of attention.

Local commute-time guided MDS for 3D non-rigid object retrieval

In this paper, we propose a 3D non-rigid shape retrieval method based on canonical shape analysis. Our main idea is to transform the problem of non-rigid shape retrieval into a rigid shape retrieval problem via the well-known multidimensional scaling (MDS) approach and random walk on graphs. We first segment the non-rigid shape into local partitions based on its salient features. Then, we calculate a local MDS problem for each partition, where the local commute time distance is used as weighting function in order to preserve local shape details. Finally, we aggregate the set of local MDS problems as a global constrained problem. The constraint is formulated using the biharmonic function between local salient features. In contrast to MDS method, the proposed local MDS is computationally efficient, parameters free and gives isometry-invariant forms with minimum features distortion. Due to these advantageous properties, the proposed method achieved good retrieval accuracy on non-rigid shape benchmark datasets.

M-estimators for robust multidimensional scaling employing ℓ2,1 norm regularization

Multidimensional Scaling (MDS) has been exploited to visualise the hidden structures among a set of entities in a reduced dimensional metric space. Here, we are interested in cases whenever the initial dissimilarity matrix is contaminated by outliers. It is well-known that the state-of-the-art algorithms for solving the MDS problem generate erroneous embeddings due to the distortion introduced by such outliers. To remedy this vulnerability, a unified framework for the solution of MDS problem is proposed, which resorts to half-quadratic optimization and employs potential functions of M-estimators in combination with ℓ2,1 norm regularization. Two novel algorithms are derived. Their performance is assessed for various M-estimators against state-of-the-art MDS algorithms on four benchmark data sets. The numerical tests demonstrate that the proposed algorithms perform better than the competing alternatives.

A Convex Matrix Optimization for the Additive Constant Problem in Multidimensional Scaling with Application to Locally Linear Embedding

The additive constant problem has a long history in multidimensional scaling and it has recently been used to resolve the issue of indefiniteness of the geodesic distance matrix in ISOMAP. But it would lead to a large positive constant being added to all eigenvalues of the centered geodesic distance matrix, often causing significant distortion of the original distances. In this paper, we reformulate the problem as a convex optimization of almost negative semidefinite matrix so as to achieve minimal variation of the original distances. We then develop a Newton-CG method and further prove its quadratic convergence. Finally, we include a novel application to the famous LLE (locally linear embedding in nonlinear dimensionality reduction), addressing the issue when the input of LLE has missing values. We justify the use of the developed method to tackle this issue by establishing that the local Gram matrix used in LLE can be obtained through a local Euclidean distance matrix. The effectiveness of our method is...

Shape‐from‐Operator: Recovering Shapes from Intrinsic Operators

AbstractWe formulate the problem of shape‐from‐operator (SfO), recovering an embedding of a mesh from intrinsic operators defined through the discrete metric (edge lengths). Particularly interesting instances of our SfO problem include: shape‐from‐Laplacian, allowing to transfer style between shapes; shape‐from‐difference operator, used to synthesize shape analogies; and shape‐from‐eigenvectors, allowing to generate ‘intrinsic averages’ of shape collections. Numerically, we approach the SfO problem by splitting it into two optimization sub‐problems: metric‐from‐operator (reconstruction of the discrete metric from the intrinsic operator) and embedding‐from‐metric (finding a shape embedding that would realize a given metric, a setting of the multidimensional scaling problem). We study numerical properties of our problem, exemplify it on several applications, and discuss its imitations.

Spectral multidimensional scaling

Significance Scientific theories describe observations by equations with a small number of parameters or dimensions. Memory and computational efficiency of dimension reduction procedures is a function of the size of the observed data. Sparse local operators that involve almost linear complexity, and faithful multiscale models with quadratic cost, make the design of dimension reduction tools a delicate balance between modeling accuracy and efficiency. Here, we propose multiscale modeling of the data at a modest computational cost. We project the classical multidimensional scaling problem into the data spectral domain. Then, embedding into a low-dimensional space is efficiently accomplished. Theoretical support and empirical evidence demonstrate that working in the natural eigenspace of the data, one could reduce the complexity while maintaining model fidelity.

H‐plots for displaying nonmetric dissimilarity matrices

AbstractNonmetric pairwise data with violations of symmetry, reflexivity, or triangle inequality appear in fields such as image matching, web mining, or cognitive psychology. When data are inherently nonmetric, we should not enforce metricity as real information could be lost. The multidimensional scaling problem is addressed from a new perspective. I propose a method based on the h‐plot, which naturally handles asymmetric proximity data. Pairwise proximities between the objects are defined, though I do not embed these objects, but rather the variables that give the proximity to or from each object. The method is very simple to implement. The representation goodness can be easily assessed. The methodology is illustrated through several small examples and applied to the analysis of digital images of human corneal endothelia. Comparisons with well‐known methods show its good behavior, especially with nonmetric pairwise data, which motivate my methodology. Other databases and methods are analyzed in the supporting information. © 2013 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 6: 136–143, 2013

A convex quadratic semi-definite programming approach to the partial additive constant problem in multidimensional scaling

Bénasséni [Partial additive constant, J. Statist. Comput. Simul. 49 (1994), pp. 179–193] studied the partial additive constant problem in multidimensional scaling. This problem is quite challenging to solve, and Bénasséni proposed a numerical procedure for two special cases: the cross-set partial perturbation and the within-set partial perturbation. This paper casts the problem as a modern quadratic semi-definite programming (QSDP) problem, which is not only capable of dealing with general cases, but also enjoys a number of good properties. One of the good properties is that the proposed approach can find the minimal constant under very weak conditions. Another is that there exists a ready-to-use numerical package such as the QSDP solver in Toh [An inexact path-following algorithm for convex quadratic SDP, Math. Program. 112 (2008), pp. 221–254], allowing a great deal of flexibility in choosing the index set to which the partial constant should be added. Our numerical results show a significant improvement over that reported in Bénasséni (1994).

The K-INDSCAL Model for Heterogeneous Three-Way Dissimilarity Data

A weighted Euclidean distance model for analyzing three-way dissimilarity data (stimuli by stimuli by subjects) for heterogeneous subjects is proposed. First, it is shown that INDSCAL may fail to identify a common space representative of the observed data structure in presence of heterogeneity. A new model that removes the rotational invariance of the classical multidimensional scaling problem and specifies K common homogeneous spaces is proposed. The model, called mixture INDSCAL in K classes, or briefly K-INDSCAL, still includes individual saliencies. However, the large number of parameters in K-INDSCAL may produce instability of the estimates and therefore a parsimonious model will also be discussed. The parameters of the model are estimated in a least-squares fitting context and an efficient coordinate descent algorithm is given. The usefulness of K-INDSCAL is demonstrated by both artificial and real data analyses.

An ordinal metrical scale built on a fuzzy nominal scale

The Measurement theory defines a measurement as a mapping from a set of empirical property manifestations to a set of abstract property values called symbols. The ordinal metrical scales were introduced within the context of Psychophysics as a way to solve the problem of multidimensional scaling. Usually the distances used to define such scales are based on the hypothesis that symbols are vectors of numbers and that each component is expressed on an interval scale or a ratio scale. In a recent paper was introduced a distance-based scale that represents manifestations from an empirical world with fuzzy subsets of lexical terms. This approach supposes only the existence of a fuzzy nominal scale and allows a choice into a wider set of distances to build the ordinal metrical scales. This paper focuses on the knowledge source used to choose a scale definition and takes metrical scales built on fuzzy nominal scale as example. Then it opens a discussion on the reality of some distances in the empirical world.

Molecular dynamics multidimensional scaling

We discuss a molecular dynamics approach to the problem of multidimensional scaling, which is a statistical method used for the visualization of dissimilarities in multidimensional data. Numerical estimates of dissimilarities between pairs of objects from a high dimensional data collection are approximated by distances between corresponding pairs of objects in a low dimensional visual representation. The quality of this approximation is expressed as an energy function, which yields the optimal arrangement at its minimum. We show how this optimal arrangement can be efficiently computed using the molecular dynamics approach. The numerical estimates show that the proposed molecular dynamics method is more accurate and about an order of magnitude faster than the simulated annealing method.

Multidimensional Scaling Using Majorization: SMACOF inR

In this paper we present the methodology of multidimensional scaling problems (MDS) solved by means of the majorization algorithm. The objective function to be minimized is known as stress and functions which majorize stress are elaborated. This strategy to solve MDS problems is called SMACOF and it is implemented in an <b>R</b> package of the same name which is presented in this article. We extend the basic SMACOF theory in terms of configuration constraints, three-way data, unfolding models, and projection of the resulting configurations onto spheres and other quadratic surfaces. Various examples are presented to show the possibilities of the SMACOF approach offered by the corresponding package.

A hybrid method for multidimensional scaling using city-block distances

The problem of multidimensional scaling with city-block distances in the embedding space is reduced to a two level optimization problem consisting of a combinatorial problem at the upper level and a quadratic programming problem at the lower level. A hybrid method is proposed combining randomized search for the upper level problem with a standard quadratic programming algorithm for the lower level problem. Several algorithms for the combinatorial problem have been tested and an evolutionary global search algorithm has been proved most suitable. An experimental code of the proposed hybrid multidimensional scaling algorithm is developed and tested using several test problems of two- and three-dimensional scaling.

THE SOLUTION OF THE DISTANCE GEOMETRY PROBLEM IN PROTEIN MODELING VIA GEOMETRIC BUILDUP

A well-known problem in protein modeling is the determination of the structure of a protein with a given set of inter-atomic or inter-residue distances obtained from either physical experiments or theoretical estimates. A general form of the problem is known as the distance geometry problem in mathematics, the graph embedding problem in computer science, and the multidimensional scaling problem in statistics. The problem has applications in many other scientific and engineering fields as well such as sensor network localization, image recognition, and protein classification. We describe the formulations and complexities of the problem in its various forms, and introduce a geometric buildup approach to the problem. Central to this approach is the idea that the coordinates of the atoms in a protein can be determined one atom at a time, with the distances from the determined atoms to the undetermined ones. It can determine a structure more efficiently than other conventional approaches, yet without requiring more distance constraints than necessary.

Global Optimization in Any Minkowski Metric: A Permutation-Translation Simulated Annealing Algorithm for Multidimensional Scaling

It is well known that considering a non-Euclidean Minkowski metric in Multidimensional Scaling, either for the distance model or for the loss function, increases the computational problem of local minima considerably. In this paper, we propose an algorithm in which both the loss function and the composition rule can be considered in any Minkowski metric, using a multivariate randomly alternating Simulated Annealing procedure with permutation and translation phases. The algorithm has been implemented in Fortran and tested over classical and simulated data matrices with sizes up to 200 objects. A study has been carried out with some of the common loss functions to determine the most suitable values for the main parameters. The experimental results confirm the theoretical expectation that Simulated Annealing is a suitable strategy to deal by itself with the optimization problems in Multidimensional Scaling, in particular for City-Block, Euclidean and Infinity metrics.

Degree Frequencies in the Minimal Spanning Tree and Dimension Identification

We discuss the application of linear combinations of the degree frequencies in the minimal spanning tree to the problem of identifying the appropriate dimension for a data set from its interpoint distance matrix. This graph-theoretical methodology, of very low computational cost, can be of aid in the problem of Multidimensional Scaling and in dimensionality reduction. Results of Lee [Lee, S. (1999). The central limit theorem for euclidean minimal spanning trees II. Adv. Appl. Probability 31(4): 969–984] imply that the procedure proposed here is asymptotically consistent.