Color Image Segmentation Based on Hue-Saturation Similarity

In this paper we analyze a family of general random block coordinate descent methods for the minimization of ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> regularized optimization problems, i.e. The objective function is composed of a smooth convex function and the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> regularization. Our family of methods covers particular cases such as random block coordinate gradient descent and random proximal coordinate descent methods. We analyze necessary optimality conditions for this nonconvex ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> regularized problem and devise a separation of the set of local minima into restricted classes based on approximation versions of the objective function. We provide a unified analysis of the almost sure convergence for this family of block coordinate descent algorithms and prove that, for each approximation version, the limit points are local minima from the corresponding restricted class of local minimizers.

Eigenvector computation such as Singular Value Decomposition (SVD) is one of the most fundamental problems in machine learning, optimization and numerical linear algebra. In recent years, many stochastic variance reduction algorithms and randomized coordinate descent algorithms have been developed to efficiently solve the leading eigenvalue problem. By taking full advantage of both variance reduction and randomized coordinate descent techniques, this paper proposes a novel Semi-stochastic Block Coordinate Descent algorithm (SBCD-SVD), which is more suitable than existing algorithms for large-scale leading eigenvalue problems of SVD, and can obtain linear convergence. Unlike existing stochastic variance reduction and randomized coordinate descent methods, our algorithm inherits their advantages. Moreover, we propose a new Asynchronous parallel Semi-stochastic Block Coordinate Descent algorithm (ASBCD-SVD) and one new Asynchronous parallel Sparse approximated Variance Reduction algorithm (ASVR-SVD) for large-scale dense and sparse datasets, respectively. Finally, we prove that both dense and sparse asynchronous parallel variants can converge linearly. Extensive experimental results show that our algorithms attain high parallel speedup and achieve almost the same performance with significantly shorter time, and thus they can be widely used in various practice applications.

Support vector machines (SVMs) are famous and important classifiers in classification researches. SVM and its application have been used in an enormous amount of research in various scientific domains in recent years. There are different methods to solve SVM problems. Coordinate descent methods are one of the main categories of methods for solving these problems. In this paper, using the generalization of Aitken’s $\Delta 2$ process to the vector case, a new accelerated coordinate descent algorithm is proposed. The new algorithm is tested with three different datasets. The numerical results indicate the efficiency of the new algorithm in the concept of increasing the speed of convergence.

Parameterized quantum circuits (PQCs) are fundamental to many hybrid quantum-classical algorithms. However, existing structure-based optimizers for the training of PQCs, such as Rotosolve and sequential minimal optimization, rely on heuristic node selection or ignore statistical noise, which limits their robustness and accuracy. To address this issue, we propose an interpolation-based coordinate descent (ICD) method as a unified framework for all structure-based optimizers. ICD approximates the cost function through interpolation, recovers its trigonometric structure, and performs global one-dimensional updates on individual parameters. Unlike previous methods, ICD derives optimal interpolation nodes that minimize statistical errors from measurements. For the common case of r equidistant frequencies, we prove that equidistant nodes with spacing 2π/(2r + 1) jointly minimize the mean squared error of Fourier coefficient estimates, the condition number of the interpolation matrix, and the average variance of the approximated cost function. Numerical experiments confirm the superior robustness and efficiency of ICD over gradient-based methods. Parameterized quantum circuits are a common tool in variational quantum algorithms and quantum machine learning. The authors design an interpolation-based coordinate descent method that reconstructs the cost landscape from a few circuit runs and achieves more efficient training than standard gradient and coordinate descent methods in our numerical tests.

In this paper we develop parallel random coordinate gradient descent methods for minimizing huge linearly constrained separable convex problems over networks. Since we have coupled constraints in the problem, we devise a family of algorithms that updates in parallel τ ≥ 2 (block) components per iteration. Moreover, the algorithms are adequate for distributed and parallel computations and their complexity per iteration is cheaper than of the full gradient method when the number of nodes N in the network is huge. We prove that for these methods we obtain in expectation an o-accurate solution in at most O(N over τe) iterations and thus the convergence rate depends linearly on the number of (block) components to be updated. We also describe several applications that fit in our framework, in particular the convex feasibility problem. Numerically, we show that the parallel coordinate descent method with τ > 2 accelerates on its basic counterpart corresponding to τ = 2.

Coordinate descent methods have considerable impact in global optimization because global (or, at least, almost global) minimization is affordable for low-dimensional problems. Coordinate descent methods with high-order regularized models for smooth nonconvex box-constrained minimization are introduced in this work. High-order stationarity asymptotic convergence and first-order stationarity worst-case evaluation complexity bounds are established. The computer work that is necessary for obtaining first-order $\varepsilon$-stationarity with respect to the variables of each coordinate-descent block is $O(\varepsilon^{-(p+1)/p})$ whereas the computer work for getting first-order $\varepsilon$-stationarity with respect to all the variables simultaneously is $O(\varepsilon^{-(p+1)})$. Numerical examples involving multidimensional scaling problems are presented. The numerical performance of the methods is enhanced by means of coordinate-descent strategies for choosing initial points.

A modified road perception algorithm is presented based on the color image clustering segmentation. According to the comparison of color spaces' uniformity and integrity, an improved K-means clustering algorithm is proposed to segment color images in the space LAB. Firstly, the target area contains road which is gained in images class using the connected domain labeling algorithm. Then, credible road edge points can be obtained in response to alternate-line sampling labeled region of images and assuming the constant of road width consequently. By establishing the B-splines curve model to fit road shape, the algorithm adopts the least square method used to search the optimal control points of splines curve to identify the road boundaries.

We present a novel column generation based boosting method for multi-class classification. Our multi-class boosting is formulated in a single optimization problem as in [1]. Different from most existing multi-class boosting methods, which use the same set of weak learners for all the classes, we train class specified weak learners (i.e., each class has a different set of weak learners). We show that using separate weak learner sets for each class leads to fast convergence, without introducing additional computational overhead in the training procedure. To further make the training more efficient and scalable, we also propose a fast coordinate descent method for solving the optimization problem at each boosting iteration. The proposed coordinate descent method is conceptually simple and easy to implement in that it is a closed-form solution for each coordinate update. Experimental results on a variety of datasets show that, compared to a range of existing multi-class boosting methods, the proposed method has much faster convergence rate and better generalization performance in most cases. We also empirically show that the proposed fast coordinate descent algorithm needs less training time than the MultiBoost algorithm in [1].KeywordsTraining TimeColumn GenerationMaster ProblemWeak LearnerCoordinate DescentThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

We consider coordinate descent (CD) methods with exact line search on convex quadratic problems. Our main focus is to study the performance of the CD method that use random permutations in each epoch and compare it to the performance of the CD methods that use deterministic orders and random sampling with replacement. We focus on a class of convex quadratic problems with a diagonally dominant Hessian matrix, for which we show that using random permutations instead of random with-replacement sampling improves the performance of the CD method in the worst-case. Furthermore, we prove that as the Hessian matrix becomes more diagonally dominant, the performance improvement attained by using random permutations increases. We also show that for this problem class, using any fixed deterministic order yields a superior performance than using random permutations. We present detailed theoretical analyses with respect to three different convergence criteria that are used in the literature and support our theoretical results with numerical experiments.

Recently, more and more real-world datasets have been composed of heterogeneous but related features from diverse views. Multiview clustering provides a promising attempt at a solution for partitioning such data according to heterogeneous information. However, most existing methods suffer from hyper-parameter tuning trouble and high computational cost. Besides, there is still an opportunity for improvement in clustering performance. To this end, a novel multiview framework, called parameter-free multiview k -means clustering with coordinate descent method (PFMVKM), is presented to address the above problems. Specifically, PFMVKM is completely parameter-free and learns the weights via a self-weighted scheme, which can avoid the intractable process of hyper-parameters tuning. Moreover, our model is capable of directly calculating the cluster indicator matrix, with no need to learn the cluster centroid matrix and the indicator matrix simultaneously as previous multiview methods have to do. What's more, we propose an efficient optimization algorithm utilizing the idea of coordinate descent, which can not only reduce the computational complexity but also improve the clustering performance. Extensive experiments on various types of real datasets illustrate that the proposed method outperforms existing state-of-the-art competitors and conforms well with the actual situation.

Nonnegative Matrix Factorization (NMF) is an effective dimension reduction method for non-negative dyadic data, and has proven to be useful in many areas, such as text mining, bioinformatics and image processing. NMF is usually formulated as a constrained non-convex optimization problem, and many algorithms have been developed for solving it. Recently, a coordinate descent method, called FastHals, has been proposed to solve least squares NMF and is regarded as one of the state-of-the-art techniques for the problem. In this paper, we first show that FastHals has an inefficiency in that it uses a cyclic coordinate descent scheme and thus, performs unneeded descent steps on unimportant variables. We then present a variable selection scheme that uses the gradient of the objective function to arrive at a new coordinate descent method. Our new method is considerably faster in practice and we show that it has theoretical convergence guarantees. Moreover when the solution is sparse, as is often the case in real applications, our new method benefits by selecting important variables to update more often, thus resulting in higher speed. As an example, on a text dataset RCV1, our method is 7 times faster than FastHals, and more than 15 times faster when the sparsity is increased by adding an L1 penalty. We also develop new coordinate descent methods when error in NMF is measured by KL-divergence by applying the Newton method to solve the one-variable sub-problems. Experiments indicate that our algorithm for minimizing the KL-divergence is faster than the Lee & Seung multiplicative rule by a factor of 10 on the CBCL image dataset.

The problem of optimal placement of elements of electrical and electronic circuits is considered. The minimum weighted connection length is selected as the criterion. A computational method is proposed that is a modification of the coordinate descent method and one of the variants of the General approach based on pair permutations. The scheme is defined by the connection matrix. We consider a fixed set of element positions and a distance matrix based on an orthogonal metric. This problem is a variant of the General mathematical model, called the quadratic assignment problem. Geometric restriction of the problem – no more than one element can be placed in one cell. It is stated that approaches based on paired and similar permutations are economical, and the method of the penalty function leads to” ditching ” and is ineffective. A modified coordinate descent method is described, which is a variant of the pair permutation method, in which pairs are selected based on the coordinate descent method. In the proposed version of the coordinate descent method, two coordinates are changed simultaneously at one stage of calculations (and not one, as in the usual optimization method). one of the coordinates is used for the usual trial step, and the other is used for correction, returning to the acceptable area. Next, the value of the target function is calculated at the found point and compared with the previously reached value. If the value has improved, the found point becomes the new starting point. Otherwise, a step is made on a different coordinate with simultaneous correction of the vector of item position numbers (return to the allowed area). The experience of using the modified method in solving the problem of placing EVA elements has shown its significant advantages in comparison with other known methods, for example, the genetic algorithm, as well as the method of penalty functions. An example of calculations using the proposed method is considered. The connection matrix was set analytically. First, the initial approximation was searched by the Monte Carlo method (10,000 iterations), after which the local optimum was calculated using a modified method of coordinate descent in the permutation space without repetitions (a limit of 100 iterations was set). The initial value of the coordinate step is equal to the size of the permutation, then at each iteration it was reduced by 1 to the minimum possible value of 1. The advantage of this method is that there is no penalty function. The search is performed automatically in the permutation space without repetitions. Computational experiments have shown high computational qualities of the proposed method.

The method of coordinate descent in conditional minimization problems

We present a new method for segmenting color images into their composite surfaces by combining color segmentation with model-based fitting utilizing sparse depth data, acquired using time-of-flight (Swissranger, PMD CamCube) and stereo techniques. The main target of our work is the segmentation of plant structures, i.e., leaves, from color-depth images, and the extraction of color and 3D shape information for automating manipulation tasks. Since segmentation is performed in the dense color space, even sparse, incomplete, or noisy depth information can be used. This kind of data often represents a major challenge for methods operating in the 3D data space directly. To achieve our goal, we construct a three-stage segmentation hierarchy by segmenting the color image with different resolutions-assuming that “true” surface boundaries must appear at some point along the segmentation hierarchy. 3D surfaces are then fitted to the color-segment areas using depth data. Those segments which minimize the fitting error are selected and used to construct a new segmentation. Then, an additional region merging and a growing stage are applied to avoid over-segmentation and label previously unclustered points. Experimental results demonstrate that the method is successful in segmenting a variety of domestic objects and plants into quadratic surfaces. At the end of the procedure, the sparse depth data is completed using the extracted surface models, resulting in dense depth maps. For stereo, the resulting disparity maps are compared with ground truth and the average error is computed.

Generalized fused Lasso (GFL) is a powerful method based on adjacent relationships or the network structure of data. It is used in a number of research areas, including clustering, discrete smoothing, and spatio-temporal analysis. When applying GFL, the specific optimization method used is an important issue. In generalized linear models, efficient algorithms based on the coordinate descent method have been developed for trend filtering under the binomial and Poisson distributions. However, to apply GFL to other distributions, such as the negative binomial distribution, which is used to deal with overdispersion in the Poisson distribution, or the gamma and inverse Gaussian distributions, which are used for positive continuous data, an algorithm for each individual distribution must be developed. To unify GFL for distributions in the exponential family, this paper proposes a coordinate descent algorithm for generalized linear models. To illustrate the method, a real data example of spatio-temporal analysis is provided.