Noisy Constraints Research Articles

MSC-CSMC: A multi-objective semi-supervised clustering algorithm based on constraints selection and multi-source constraints for gene expression data.

Many clustering techniques have been proposed to group genes based on gene expression data. Among these methods, semi-supervised clustering techniques aim to improve clustering performance by incorporating supervisory information in the form of pairwise constraints. However, noisy constraints inevitably exist in the constraint set obtained on the practical unlabeled dataset, which degenerates the performance of semi-supervised clustering. Moreover, multiple information sources are not integrated into multi-source constraints to improve clustering quality. To this end, the research proposes a new multi-objective semi-supervised clustering algorithm based on constraints selection and multi-source constraints (MSC-CSMC) for unlabeled gene expression data. The proposed method first uses the gene expression data and the gene ontology (GO) that describes gene annotation information to form multi-source constraints. Then, the multi-source constraints are applied to the clustering by improving the constraint violation penalty weight in the semi-supervised clustering objective function. Furthermore, the constraints selection and cluster prototypes are put into the multi-objective evolutionary framework by adopting a mixed chromosome encoding strategy, which can select pairwise constraints suitable for clustering tasks through synergistic optimization to reduce the negative influence of noisy constraints. The proposed MSC-CSMC algorithm is testified using five benchmark gene expression datasets, and the results show that the proposed algorithm achieves superior performance.

Simulated Annealing-Based Optimization (SA) Approach for Solving Economic Load Dispatch Problems

The economic dispatch problem with non smooth cost function has become a major issue with depletion of coal and increasing fuel prices. A proper schedule of available generating units may save millions of dollars per year in production cost. In this paper, Simulated Annealing Optimization (SA) algorithm is proposed for solving economic load dispatch (ELD) problems. Simulated annealing is a random-search technique which exploits an analogy between the way in which a metal cools and freezes into a minimum energy crystalline structure (the annealing process) and the search for a minimum in a more general system; it forms the basis of an optimization technique for combinatorial and other problems Simulated annealing can deal with highly nonlinear models, chaotic and noisy data and many constraints. It is a robust and general technique. Its main advantages over other local search methods are its flexibility and its ability to approach global optimality. In all cases, the proposed algorithm either matches or outperforms the solution reported for the existing algorithms. The proposed technique is also easy to implement and capable of finding feasible near global optimal solution.

A framework for deep constrained clustering

The area of constrained clustering has been extensively explored by researchers and used by practitioners. Constrained clustering formulations exist for popular algorithms such as k-means, mixture models, and spectral clustering but have several limitations. A fundamental strength of deep learning is its flexibility, and here we explore a deep learning framework for constrained clustering and in particular explore how it can extend the field of constrained clustering. We show that our framework can not only handle standard together/apart constraints (without the well documented negative effects reported earlier) generated from labeled side information but more complex constraints generated from new types of side information such as continuous values and high-level domain knowledge. Furthermore, we propose an efficient training paradigm that is generally applicable to these four types of constraints. We validate the effectiveness of our approach by empirical results on both image and text datasets. We also study the robustness of our framework when learning with noisy constraints and show how different components of our framework contribute to the final performance. Our source code is available at: http://github.com/blueocean92 .

Overlapping community finding with noisy pairwise constraints

In many real applications of semi-supervised learning, the guidance provided by a human oracle might be “noisy” or inaccurate. Human annotators will often be imperfect, in the sense that they can make subjective decisions, they might only have partial knowledge of the task at hand, or they may simply complete a labeling task incorrectly due to the burden of annotation. Similarly, in the context of semi-supervised community finding in complex networks, information encoded as pairwise constraints may be unreliable or conflicting due to the human element in the annotation process. This study aims to address the challenge of handling noisy pairwise constraints in overlapping semi-supervised community detection, by framing the task as an outlier detection problem. We propose a general architecture which includes a process to “clean” or filter noisy constraints. Furthermore, we introduce multiple designs for the cleaning process which use different type of outlier detection models, including autoencoders. A comprehensive evaluation is conducted for each proposed methodology, which demonstrates the potential of the proposed architecture for reducing the impact of noisy supervision in the context of overlapping community detection.

Neural Machine Translation With Noisy Lexical Constraints

Lexically constrained decoding for machine translation has shown to be beneficial in previous studies. Unfortunately, constraints provided by users may contain mistakes in real-world situations. It is still an open question that how to manipulate these noisy constraints in such practical scenarios. We present a novel framework that treats constraints as external memories. In this soft manner, a mistaken constraint can be corrected. Experiments demonstrate that our approach can achieve substantial BLEU gains in handling noisy constraints. These results motivate us to apply the proposed approach on a new scenario where constraints are generated without the help of users. Experiments show that our approach can indeed improve the translation quality with the automatically generated constraints.

An Adaptive Robust Semi-supervised Clustering Framework Using Weighted Consensus of Random k-Means Ensemble

Semi-supervised cluster ensemble usually introduces a small amount of supervision in the first stage of cluster ensemble, i.e., ensemble generation, by performing many runs of semi-supervised clustering algorithms. However, it is neither efficient in terms of computational complexity, nor flexible in a dynamic learning environment where limited supervision changes over time. In this article we propose a new framework which generates base partitions in an unsupervised manner and attributes different weights to each cluster of the base partitions. The weighting scheme considers both the internal validation measures of clustering and the degrees of satisfaction of pairwise constraints. A weighted co-association matrix based consensus approach is then applied to achieve a final partition. To handle high-dimensional data, we generate base partitions using k-means with both random sampling and random subspace techniques. The new framework retains a high accuracy, and is efficient since it avoids performing semi-supervised clustering in ensemble generation and the complexity of the weighting scheme is independent of the number of instances in a dynamic environment. It is more adaptive than the traditional approach because it does not require rerunning semi-supervised clustering algorithms when the limited supervision changes. Empirical results on 12 datasets demonstrate that it is also more robust to noisy constraints.

Constrained Bayesian Optimization with Noisy Experiments

Randomized experiments are the gold standard for evaluating the effects of changes to real-world systems. Data in these tests may be difficult to collect and outcomes may have high variance, resulting in potentially large measurement error. Bayesian optimization is a promising technique for efficiently optimizing multiple continuous parameters, but existing approaches degrade in performance when the noise level is high, limiting its applicability to many randomized experiments. We derive an expression for expected improvement under greedy batch optimization with noisy observations and noisy constraints, and develop a quasi-Monte Carlo approximation that allows it to be efficiently optimized. Simulations with synthetic functions show that optimization performance on noisy, constrained problems outperforms existing methods. We further demonstrate the effectiveness of the method with two real-world experiments conducted at Facebook: optimizing a ranking system, and optimizing server compiler flags.

Semi-Supervised Affinity Propagation with Soft Instance-Level Constraints.

Soft-constraint semi-supervised affinity propagation (SCSSAP) adds supervision to the affinity propagation (AP) clustering algorithm without strictly enforcing instance-level constraints. Constraint violations lead to an adjustment of the AP similarity matrix at every iteration of the proposed algorithm and to addition of a penalty to the objective function. This formulation is particularly advantageous in the presence of noisy labels or noisy constraints since the penalty parameter of SCSSAP can be tuned to express our confidence in instance-level constraints. When the constraints are noiseless, SCSSAP outperforms unsupervised AP and performs at least as well as the previously proposed semi-supervised AP and constrained expectation maximization. In the presence of label and constraint noise, SCSSAP results in a more accurate clustering than either of the aforementioned established algorithms. Finally, we present an extension of SCSSAP which incorporates metric learning in the optimization objective and can further improve the performance of clustering.

Constrained Clustering With Imperfect Oracles.

While clustering is usually an unsupervised operation, there are circumstances where we have access to prior belief that pairs of samples should (or should not) be assigned with the same cluster. Constrained clustering aims to exploit this prior belief as constraint (or weak supervision) to influence the cluster formation so as to obtain a data structure more closely resembling human perception. Two important issues remain open: 1) how to exploit sparse constraints effectively and 2) how to handle ill-conditioned/noisy constraints generated by imperfect oracles. In this paper, we present a novel pairwise similarity measure framework to address the above issues. Specifically, in contrast to existing constrained clustering approaches that blindly rely on all features for constraint propagation, our approach searches for neighborhoods driven by discriminative feature selection for more effective constraint diffusion. Crucially, we formulate a novel approach to handling the noisy constraint problem, which has been unrealistically ignored in the constrained clustering literature. Extensive comparative results show that our method is superior to the state-of-the-art constrained clustering approaches and can generally benefit existing pairwise similarity-based data clustering algorithms, such as spectral clustering and affinity propagation.

Localization in Wireless Sensor Networks: Solvability Improvement Technique Using Priori Information from Sensing Data and Network Properties in Unit Disk Graph Model

From the theoretical point of view, network localization can be viewed as finding a unique solution from distances constraint among points. The one of the difficulties is that even if the network is uniquely localizable, it is proven to be an NP-Hard [1]. It is also true that the network graph has to be sufficiently dense [2]. This poses even more challenges to the original problem as we often work on sparse networks. To cope with this, in [3], we introduce priori knowledge to assist the process of finding the unique localization solution. It helps to speed up the searching algorithm; however, the ambiguity still exists among sparse networks. In this paper we try to bring as much priori knowledge as possible to assist or to be used as constraints. Hopefully this will reduce search space and reach the unique solution quickly. In clean environment, this extra info will, by some magnitude, bring the graph closer to the unique answer. We start from integer-coordinate noise-free position and then add sources of priori knowledge. Then we examine the case where assisted data can be noisy. A search is used within the noisy but useful constraint. The justification of using the assisted knowledge is from the practical uses of some networks, e.g. sensor network, where other measurements are available and they are often correlated and can be helpful in determining the positions.

Simulation-based optimization over discrete sets with noisy constraints

This article considers a constrained optimization problem over a discrete set where noise-corrupted observations of the objective and constraints are available. The problem is challenging because the feasibility of a solution cannot be known for certain, due to the noisy measurements of the constraints. To tackle this issue, a method is proposed that converts constrained optimization into the unconstrained optimization problem of finding a saddle point of the Lagrangian. The method applies stochastic approximation to the Lagrangian in search of the saddle point. The proposed method is shown to converge, under suitable conditions, to the optimal solution almost surely as the number of iterations grows. The effectiveness of the proposed method is demonstrated numerically in three settings: (i) inventory control in a periodic review system; (ii) staffing in a call center; and (iii) staffing in an emergency room.

A study of K-Means-based algorithms for constrained clustering

The problem of clustering with constraints has received considerable attention in the last decade. Indeed, several algorithms have been proposed, but only a few studies have partially compared their performances. In this work, three well-known algorithms for k-means-based clustering with soft constraints --Constrained Vector Quantization Error CVQE, its variant named LCVQE, and the Metric Pairwise Constrained K-Means MPCK-Means --are systematically compared according to three criteria: Adjusted Rand Index, Normalized Mutual Information, and the number of violated constraints. Experiments were performed on 20 datasets, and for each of them 800 sets of constraints were generated. In order to provide some reassurance about the non-randomness of the obtained results, outcomes of statistical tests of significance are presented. In terms of accuracy, LCVQE has shown to be competitive with CVQE, while violating less constraints. In most of the datasets, both CVQE and LCVQE presented better accuracy compared to MPCK-Means, which is capable of learning distance metrics. In this sense, it was also observed that learning a particular distance metric for each cluster does not necessarily lead to better results than learning a single metric for all clusters. The robustness of the algorithms with respect to noisy constraints was also analyzed. From this perspective, the most interesting conclusion is that CVQE has shown better performance than LCVQE in most of the experiments. The computational complexities of the algorithms are also presented. Finally, a variety of more specific new experimental findings are discussed in the paper --e.g., deduced constraints usually do not help finding better data partitions.

Navigating a robotic swarm in an uncharted 2D landscape

Navigation is a major issue in robotics due to the necessity for the robots’ course of movement. Navigation consists of two essential components known as localization and planning. Localization in robotics refers to one’s location with reference to a well known position inside the map. Planning is considered as the computation of a path through a map which represents the environment. This given path would be chosen based on the potential of the problem so that the expected destination would be achieved. As such, a reliable map is essential for navigation without which robots would not be able to accomplish the goals. In navigational approaches, reliability of the map would be challenged due to the dynamic and unpredictable nature of real-world applications. It is, consequently, crucial to implement solutions for searching such environments—those affected by dynamic and noisy constraints. In the present study, two enhanced versions of particle swarm optimization (PSO) called area extension PSO (AEPSO) and cooperative AEPSO (CAEPSO) are employed. During the study, AEPSO and CAEPSO are employed as decision-makers and movement controllers of simulated robots (hereafter referred to as agents). The agents’ task is to seek for survivors in realistic simulations based on real-world hostile situations. This study examines the feasibility of AEPSO and CAEPSO on uncertain and time-dependent simulated environments. The simulations follow two phases of training and testing model. Agents use past knowledge gathered during the training phase in their testing phase. The study addresses the impacts of past knowledge, homogeneity and heterogeneity in robotic swarm search. The results demonstrate the feasibility of CAEPSO as robot controller and decision-maker.

Reconstruction of pedestrian–vehicle accident using sequential linear programming optimizer

The purpose of this paper is to present an optimization-based method for reconstructing pedestrian–vehicle accident and testing its performance. Using this method, the optimizer should automatically vary boundary conditions of multi-body dynamic simulation to generate the best match between simulation and real accident data. Considering the characteristic of pedestrian–vehicle collision, a corresponding optimization model is designed and sequential linear programming is used to deal with the non-linear objective function and the noisy constraints. By the reconstruction of two real-world pedestrian collisions, the performance of the method is tested. We compare the results with the global response surface of the objective function, which is constructed by a full-factorial experimental design. And the results are also evaluated in terms of injuries from the impact with the ground. The results show that this method is very effective in finding an optimum solution, which can not only reduce the number of cycles but also save manual operation.

Simulation Optimization: Integrating Research and Practice

The feature article by Fu (2002) identifies several areas within the field of simulation optimization that need further attention, including the development of good stopping criteria, providing improved measures of goodness for the estimated optimal solution, and addressing optimization problems with noisy constraints. These are all important areas for future research. Fu (2002) also describes the differences between research and software developments in the area of simulation optimization. In particular, the research literature is focused on rigorous simulation optimization algorithms (with provable performance guarantees), while algorithms implemented in commercial simulation software are generally heuristic in nature. This presumably reflects a belief that the simulation optimization approaches developed by researchers either require too much special structure to be implemented in a general purpose simulation language, or they are too slow to be used to solve large and complex simulation optimization problems. The main thesis of this article is that it is possible to develop simulation optimization algorithms that are efficient, rigorous, and suitable for implementation in simulation languages and that the development and implementation of such algorithms should be a common goal for both researchers and software developers. The desirability of having efficient optimization approaches that are suitable for implementation in commercial simulation software packages seems obvious given the extensive use of simulation software for design and analysis of complex systems. Similarly, among algorithms that are comparable in terms of applicability and efficiency, it seems evident that the choices that have provable performance guarantees are preferable over choices that are purely heuristic. However, many simulationists appear to have the perception that one can either use a heuristic algorithm with a fast empirical convergence rate, or use a rigorous method with a slow empirical convergence rate, but that it is not possible to use a simulation optimization approach that is both efficient and rigorous. When evaluating this perceived tradeoff between a fast empirical convergence rate and provable performance guarantees, it is not surprising that software developers have chosen to implement heuristic approaches that appear to work well in practice on a wide range of problems. The perceived tradeoff described in the previous paragraph may be due to the frequent use of provable performance guarantees in the research literature that severely restrict the flexibility of algorithm developers (in that these guarantees only hold under highly restrictive conditions). However, there are several different types of provable performance guarantees that a simulation optimization algorithm may have. Fu (2002) discusses a few of these (see also Banks et al. 2000), and additional discussion is provided below. An ideal performance guarantee would be to know that an algorithm stops and returns an estimated optimal solution whose corresponding objective function value differs by less than a prespecified amount from the optimal objective function value. In other words, if it is of interest to solve the optimization problem min ∈ J , if ∗ ∈ is a (global) optimal solution, and if
∈ is the estimated solution returned by the optimization algorithm, then ideally, we would want to know either that J
≤ J ∗ + or that J
≤

Solving Nonlinear Programming Problems with Noisy Function Values and Noisy Gradients

An efficient algorithm for solving nonlinear programs with noisy equality constraints is introduced and analyzed. The unknown exact constraints are replaced by surrogates based on the bundle idea, a well-known strategy from nonsmooth optimization. This concept allows us to perform a fast computation of the surrogates by solving simple quadratic optimization problems, control the memory needed by the algorithm, and prove the differentiability properties of the surrogate functions. The latter aspect allows us to invoke a sequential quadratic programming method. The overall algorithm is of the quasi-Newton type. Besides convergence theorems, qualification results are given and numerical test runs are discussed.