Noisy Problem Research Articles

Localization for Solving Noisy Multi-Objective Optimization Problems

This paper investigates the use of a framework of local models in the context of noisy evolutionary multi-objective optimization. Within this framework, the search space is explicitly divided into several nonoverlapping hyperspheres. A direction of improvement, which is related to the average performance of the spheres, is used for moving solutions within each sphere. This helps the local models to filter noise and increase the robustness of the evolutionary algorithm in the presence of noise. A wide range of noisy problems we used for testing and the experimental results demonstrate the ability of local models to better filter noise in comparison with that of global models.

Incorporating minimum Frobenius norm models in direct search

The goal of this paper is to show that the use of minimum Frobenius norm quadratic models can improve the performance of direct-search methods. The approach taken here is to maintain the structure of directional direct-search methods, organized around a search and a poll step, and to use the set of previously evaluated points generated during a direct-search run to build the models. The minimization of the models within a trust region provides an enhanced search step. Our numerical results show that such a procedure can lead to a significant improvement of direct search for smooth, piecewise smooth, and noisy problems.

Monkey with a Tool Belt and the Noisy Problem (review)

Monkey with a Tool Belt and the Noisy Problem (review)

Supervised projection approach for boosting classifiers

In this paper we present a new approach for boosting methods for the construction of ensembles of classifiers. The approach is based on using the distribution given by the weighting scheme of boosting to construct a non-linear supervised projection of the original variables, instead of using the weights of the instances to train the next classifier. With this method we construct ensembles that are able to achieve a better generalization error and are more robust to noise presence. It has been proved that A daB oost method is able to improve the margin of the instances achieved by the ensemble. Moreover, its practical success has been partially explained by this margin maximization property. However, in noisy problems, likely to occur in real-world applications, the maximization of the margin of wrong instances or outliers can lead to poor generalization. We propose an alternative approach, where the distribution of the weights given by the boosting algorithm is used to get a supervised projection. Then, the supervised projection is used to train the next classifier using a uniform distribution of the training instances. The proposed approach is compared with three boosting techniques, namely A daB oost, G entleB oost and M adaB oost, showing an improved performance on a large set of 55 problems from the UCI Machine Learning Repository, and less sensitiveness to noise in the class labels. The behavior of the proposed algorithm in terms of margin distribution and bias–variance decomposition is also studied.

Evolutionary multiobjective optimization in noisy problem environments

This paper presents a multiobjective evolutionary algorithm (MOEA) capable of handling stochastic objective functions. We extend a previously developed approach to solve multiple objective optimiza...

Toward routine billion‐variable optimization using genetic algorithms

AbstractThe push for better understanding and design of complex systems requires the solution of challenging optimization problems with large numbers of decision variables. This note presents principled results demonstrating the scalable solution of a difficult test function on instances over a billion variables using a parallel implementation of a genetic algorithm (GA). The problem addressed is a noisy, blind problem over a vector of binary decision variables. Noise is added equaling a tenth of the deterministic objective function variance of the problem, thereby making it difficult for simple hillclimbers to find the optimal solution. The genetic algorithm used—the compact GA—is able to find the optimum in the presence of noise quickly, reliably, and accurately, and the solution scalability follows known convergence theories. These results on noisy problem together with other results on problems involving varying modularity, hierarchy, and overlap foreshadow routine solution of billion‐variable problems across the landscape of complexity science. © 2007 Wiley Periodicals, Inc. Complexity 12: 27–29, 2007

Novel Algorithms for Noisy Minimization Problems with Applications to Neural Networks Training

The supervisor and searcher cooperation framework (SSC), introduced in Refs. 1 and 2, provides an effective way to design efficient optimization algorithms combining the desirable features of the two existing ones. This work aims to develop efficient algorithms for a wide range of noisy optimization problems including those posed by feedforward neural networks training. It introduces two basic SSC algorithms. The first seems suited for generic problems. The second is motivated by neural networks training problems. It introduces also inexact variants of the two algorithms, which seem to possess desirable properties. It establishes general theoretical results about the convergence and speed of SSC algorithms and illustrates their appealing attributes through numerical tests on deterministic, stochastic, and neural networks training problems.

Regularized spectral matching for blind source separation. Application to fMRI imaging

The main contribution of this paper is to present a Bayesian approach for solving the noisy instantaneous blind source separation problem based on second-order statistics of the time-varying spectrum. The success of the blind estimation relies on the nonstationarity of the second-order statistics and their intersource diversity. Choosing the time-frequency domain as the signal representation space and transforming the data by a short-time Fourier transform (STFT), our method presents a simple EM algorithm that can efficiently deal with the time-varying spectrum diversity of the sources. The estimation variance of the STFT is reduced by averaging across time-frequency subdomains. The algorithm is demonstrated on a standard functional resonance imaging (fMRI) experiment involving visual stimuli in a block design. Explicitly taking into account the noise in the model, the proposed algorithm has the advantage of extracting only relevant task-related components and considers the remaining components (artifacts) to be noise.

Approximate Implementations of Pure Random Search in the Presence of Noise

We discuss the noisy optimisation problem, in which function evaluations are subject to random noise. Adaptation of pure random search to noisy optimisation by repeated sampling is considered. We introduce and exploit an improving bias condition on noise-affected pure random search algorithms. Two such algorithms are considered; we show that one requires infinite expected work to proceed, while the other is practical.

Optimizing Statistical Character Recognition Using Evolutionary Strategies to Recognize Aircraft Tail Numbers

The design of statistical classification systems for optical character recognition (OCR) is a cumbersome task. This paper proposes a method using evolutionary strategies (ES) to evolve and upgrade the set of parameters in an OCR system. This OCR is applied to identify the tail number of aircrafts moving on the airport. The proposed approach is discussed and some results are obtained using a benchmark data set. This research demonstrates the successful application of ES to a difficult, noisy, and real-world problem.

Efficiency of self-generating neural networks applied to pattern recognition

Self-generating neural networks (SGNNs) have been in the spotlight of the fields of neural networks algorithm research for the sake of their efficiency. In practice, this neural network is implemented as a self-generating neural tree (SGNT) which is based on a hierarchical clustering algorithm. In this paper, we present the superior performance of the SGNT when it is applied to character recognition problems. Basically, the SGNT algorithm is generated as a kind of competitive learning algorithm. Therefore, it is natural to have a competent performance at the area of clustering or classification. However, our experimental results show that the SGNN method is very efficient to solve even pattern recognition problems, especially when they include a noisy signal problem.

Multiobjective evolutionary algorithm for the optimization of noisy combustion processes

This work introduces a multiobjective evolutionary algorithm capable of handling noisy problems with a particular emphasis on robustness against unexpected measurements (outliers). The algorithm is based on the Strength Pareto evolutionary algorithm of Zitzler and Thiele and includes the new concepts of domination dependent lifetime, re-evaluation of solutions and modifications in the update of the archive population. Several tests on prototypical functions underline the improvements in convergence speed and robustness of the extended algorithm. The proposed algorithm is implemented to the Pareto optimization of the combustion process of a stationary gas turbine in an industrial setup. The Pareto front is constructed for the objectives of minimization of NO/sub x/ emissions and reduction of the pressure fluctuations (pulsation) of the flame. Both objectives are conflicting affecting the environment and the lifetime of the turbine, respectively. The optimization leads a Pareto front corresponding to reduced emissions and pulsation of the burner. The physical implications of the solutions are discussed and the algorithm is evaluated.

Algorithms for Noisy Problems in Gas Transmission Pipeline Optimization

In this paper we describe some algorithms for noisy optimization in the context of problems from the gas transmission industry. The algorithms are implicit filtering, DIRECT, and a new hybrid of these methods, which uses DIRECT to find an intitial iterate for implicit filtering. We report on numerical results that illustrate the performance of the methods.

Unbiased blind adaptive channel identification and equalization

The blind adaptive equalization and identification of communication channels is a problem of important current theoretical and practical concerns. Previously proposed solutions for this problem exploit the diversity induced by sensor arrays or time oversampling, leading to the so-called second-order algebraic/statistical techniques. The prediction error method is one of them, perhaps the most appealing in practice, due to its inherent robustness to ill-defined channel lengths as well as for its simple adaptive implementation. Unfortunately, the performance of prediction error methods is known to be severely limited in noisy environments, which calls for the development of noise (bias) removal techniques. We present a low-cost algorithm that solves this problem and allows the adaptive estimation of unbiased linear predictors in additive noise with arbitrary autocorrelation. This algorithm does not require the knowledge of the noise variance and relies on a new constrained prediction cost function. The technique can be applied in other noisy prediction problems. Global convergence is established analytically. The performance of the denoising technique is evaluated over GSM test channels.

Cost function adaptation: a stochastic gradient algorithm for data echo cancellation

A family of stochastic gradient algorithms and their behaviour in the data echo cancellation work platform are presented. The cost function adaptation algorithms use an error exponent update strategy based on an absolute error mapping, which is updated at every iteration. The quadratic and nonquadratic cost functions are special cases of the new family. Several possible realisations are introduced using these approaches. The noisy error problem is discussed and the digital recursive filter estimator is proposed. The simulation outcomes confirm the effectiveness of the proposed family of algorithms.

Gaussian Mixture Models and Probabilistic Decision-Based Neural Networks for Pattern Classification: A Comparative Study

Probabilistic Decision-Based Neural Networks (PDBNNs) can be considered as a special form of Gaussian Mixture Models (GMMs) with trainable decision thresholds. This paper provides detailed illustrations to compare the recognition accuracy and decision boundaries of PDBNNs with that of GMMs through two pattern recognition tasks, namely the noisy XOR problem and the classification of two-dimensional vowel data. The paper highlights the strengths of PDBNNs by demonstrating that their thresholding mechanism is very effective in detecting data not belonging to any known classes. The original PDBNNs use elliptical basis functions with diagonal covariance matrices, which may be inappropriate for modelling feature vectors with correlated components. This paper overcomes this limitation by using full covariance matrices, and showing that the matrices are effective in characterising non-spherical clusters.

Application of Neural Networks to Cache Replacement

Probabilistic Decision-Based Neural Networks (PDBNNs) can be considered as a special form of Gaussian Mixture Models (GMMs) with trainable decision thresholds. This paper provides detailed illustrations to compare the recognition accuracy and decision boundaries of PDBNNs with that of GMMs through two pattern recognition tasks, namely the noisy XOR problem and the classification of two-dimensional vowel data. The paper highlights the strengths of PDBNNs by demonstrating that their thresholding mechanism is very effective in detecting data not belonging to any known classes. The original PDBNNs use elliptical basis functions with diagonal covariance matrices, which may be inappropriate for modelling feature vectors with correlated components. This paper overcomes this limitation by using full covariance matrices, and showing that the matrices are effective in characterising non-spherical clusters.

Best Fourier Approximation and Application in Efficient Blurred Signal Reconstruction

We expand upon the known results on sharp linear Fourier methods of approximation where the approximation is the best in terms of both rate and constant among all polynomial procedures of approximation. So far these results have been studied due to their mathematical beauty rather than their practical importance. In this paper we show that they are the core mathematics underlying best statistical methods of solving noisy ill-posed problems. In particular, we suggest a procedure for recovery of noisy blurred signals based on samples of small sizes where a traditional statistics concludes that the complexity of such a setting makes the problem not worthy of a further study. Thus, we present a problem where a combination of the classical approximation theory and statistics leads to interesting practical results.

Speed of convergence in the hausdorff metric for estimators of irregular mixing densities

In this paper we consider a noisy deconvolution problem where the signal to be recovered is irregular. Like in the ordinary, direct, estimation models also in the present indirect set-up the approximation or estimate is corrupted by the Gibbs phenomenon. But this effect can also be remedied using the Cesàro averaging technique known from the direct case. Although the supremum norm itself is unsuitable it seems adequate to asses the quality of the estimator in a metric related to it. Here we propose the metric defined by the Hausdorff distance between the extended, closed graphs of two functions. Convergence in this Hausdorff metric entails convergence in the supremum metric if the functions involved are continuous. We obtain a speed of almost sure convergence in the Hausdorff metric for the proposed estimators. This method provides an alternative to an approach from the wavelet or change-point perspective.

Application of the Padé Method to Solving the Noisy Trigonometric Moment Problem: Some Initial Results

The noisy trigonometric moment problem for a finite linear combination of box functions is considered, and a research program, possibly leading to a superresolving method, is outlined and some initial steps are performed. The method is based on the remark that the poles of the Pade approximant to the Z-transform of the noiseless moments show, asymptotically, a regular pattern in the complex plane. The pattern can be described by a set of arcs, connecting points on the unit circle, and a pole density function defined on the arcs. When a moderate noise affects the moments, more arcs are needed to describe the pole pattern, but the noiseless pattern is slightly deformed, still allowing its identification. When this identification is possible, a very effective noise filter and moment extrapolator should be easily constructed. In this paper only some preliminary steps of the above research program are performed. Specifically, the case of one box function is considered. A method for computing the pole patterns,...