Deterministic Annealing Algorithm Research Articles

Data visualization by multidimensional scaling: a deterministic annealing approach

Multidimensional scaling addresses the problem how proximity data can be faithfully visualized as points in a low-dimensional Euclidean space. The quality of a data embedding is measured by a stress function which compares proximity values with Euclidean distances of the respective points. The corresponding minimization problem is non-convex and sensitive to local minima. We present a novel deterministic annealing algorithm for the frequently used objective SSTRESS and for Sammon mapping, derived in the framework of maximum entropy estimation. Experimental results demonstrate the superiority of our optimization technique compared to conventional gradient descent methods.

Multiscale Annealing for Grouping and Unsupervised Texture Segmentation

We derive real-time global optimization methods for several clustering optimization problems commonly used in unsupervised texture segmentation. Speed is achieved by exploiting the image neighborhood relation of features to design a multiscale optimization technique, while accuracy and global optimization properties are gained using annealing techniques. Coarse grained cost functions are derived for central and histogram-based clustering as well as several sparse proximity-based clustering methods. For optimization deterministic annealing algorithms are applied. Annealing schedule, coarse-to-fine optimization and the estimated number of segments are tightly coupled by a statistical convergence criterion derived from computational learning theory. The notion of optimization scale parametrized by a computational temperature is thus unified with the scales defined by the image resolution and the model or segment complexity. The algorithms are benchmarked on Brodatz-like microtexture mixtures. Results are presented for an autonomous robotics application. Extensions are discussed in the context of prestructuring large image databases valuable for fast and reliable image retrieval.

Data clustering using hierarchical deterministic annealing and higher order statistics [image processing

In this brief, we propose an extension to the hierarchical deterministic annealing (HDA) algorithm for clustering by incorporating additional features into the algorithm. To decide a split in a cluster, the interdependency among all the clusters is taken into account by using the entire data distribution. A general distortion measure derived from the higher order statistics (HOS) of the data is used to analyze the phase transitions. Experimental results clearly demonstrate the improvement in the performance of the HDA algorithm when the interdependency among the clusters and the HOS of the data points are also utilized for the purpose of clustering.

A deterministic annealing approach for parsimonious design of piecewise regression models

A new learning algorithm is proposed for piecewise regression modeling. It employs the technique of deterministic annealing to design space partition regression functions. While the performance of traditional space partition regression functions such as CART and MARS is limited by a simple tree-structured partition and by a hierarchical approach for design, the deterministic annealing algorithm enables the joint optimization of a more powerful piecewise structure based on a Voronoi partition. The new method is demonstrated to achieve consistent performance improvements over regular CART as well as over its extension to allow arbitrary hyperplane boundaries. Comparison tests, on several benchmark data sets from the regression literature, are provided.

Image Restoration Using Generalized Deterministic Annealing

Regularization provides an effective method by which to generate high-quality solutions to the image restoration problem. Typically, an energy functional that balances deconvolution (sharpening) and smoothing is used to evaluate the quality of an iteratively generated solution. The success of the regularization approach is strongly tied to the method used to generate solutions of lower energy and, thus, higher quality. The generalized deterministic annealing (GDA) algorithm is developed here specifically for image restoration. Compared to other regularization techniques, GDA yields improvements to the quality of the restored image and in the computational expense of the restoration process. This paper specifies the numerical method, the parameters, and the energy functional used to generate image restoration solutions via GDA. Results are given that contrast the superior performance of GDA in image restoration against the traditional constrained least squares method, the steepest descent method, and the simulated annealing method.

Hierarchical, Unsupervised Learning with Growing via Phase Transitions

We address unsupervised learning subject to structural constraints, with particular emphasis placed on clustering with an imposed decision tree structure. Most known methods are greedy, optimizing one node of the tree at a time to minimize a local cost. By constrast, we develop a joint optimization method, derived based on information-theoretic principles and closely related to known methods in statistical physics. The approach is inspired by the deterministic annealing algorithm for unstructured data clustering, which was based on maximum entropy inference. The new approach is founded on the principle of minimum cross-entropy, using informative priors to approximate the unstructured clustering solution while imposing the structural constraint. The resulting method incorporates supervised learning principles applied in an unsupervised problem setting. In our approach, the tree “grows” by a sequence of bifurcations that occur while optimizing an effective free energy cost at decreasing temperature scales. Thus, estimates of the tree size and structure are naturally obtained at each temperature in the process. Examples demonstrate considerable improvement over known methods.

Improved clustering using deterministic annealing with a gradient descent technique

Various techniques exist to solve the non-convex optimization problem of clustering. Recent developments have employed a deterministic annealing approach to solving this problem. In this letter a new approximation clustering algorithm, incorporating a gradient descent technique with deterministic annealing, is described. Results are presented for this new method, and its performance is compared with the K-means algorithm and a previously used deterministic annealing clustering algorithm. The new method is shown to produce more effective and robust clustering.

Track finding with deformable templates — the elastic arms approach

A novel algorithm for particle tracking is presented and evaluated. It is based on deformable templates that converge using a deterministic annealing algorithm. These deformable templates are initialized by Hough transforms. The algorithm, which effectively represents a merger between neuronic decision making and parameter fitting, naturally lends itself to parallel execution. Very good performance is obtained for both non-magnetic and magnetic tracks. For the latter simulated TPC tracks from the CERN DELPHI detector are used.