Noisy Problem Research Articles

CMA-ES with Learning Rate Adaptation

The covariance matrix adaptation evolution strategy (CMA-ES) is one of the most successful methods for solving continuous black-box optimization problems. A practically useful aspect of CMA-ES is that it can be used without hyperparameter tuning. However, the hyperparameter settings still have a considerable impact on performance, especially for difficult tasks, such as solving multimodal or noisy problems. This study comprehensively explores the impact of learning rate on CMA-ES performance and demonstrates the necessity of a small learning rate by considering ordinary differential equations. Thereafter, it discusses the setting of an ideal learning rate. Based on these discussions, we develop a novel learning rate adaptation mechanism for CMA-ES that maintains a constant signal-to-noise ratio. Additionally, we investigate the behavior of CMA-ES with the proposed learning rate adaptation mechanism through numerical experiments and compare the results with those obtained for CMA-ES with a fixed learning rate and with population size adaptation. The results show that CMA-ES with the proposed learning rate adaptation works well for multimodal and/or noisy problems without extremely expensive learning rate tuning.

CATNet: Cross-modal fusion for audio–visual speech recognition

Automatic speech recognition (ASR) is a typical pattern recognition technology that converts human speeches into texts. With the aid of advanced deep learning models, the performance of speech recognition is significantly improved. Especially, the emerging Audio–Visual Speech Recognition (AVSR) methods achieve satisfactory performance by combining audio-modal and visual-modal information. However, various complex environments, especially noises, limit the effectiveness of existing methods. In response to the noisy problem, in this paper, we propose a novel cross-modal audio–visual speech recognition model, named CATNet. First, we devise a cross-modal bidirectional fusion model to analyze the close relationship between audio and visual modalities. Second, we propose an audio–visual dual-modal network to preprocess audio and visual information, extract significant features and filter redundant noises. The experimental results demonstrate the effectiveness of CATNet, which achieves excellent WER, CER and converges speeds, outperforms other benchmark models and overcomes the challenge posed by noisy environments.

The Influence of Noise on Multi-Parent Crossover for an Island Model Genetic Algorithm

Many optimization problems tackled by evolutionary algorithms are not only computationally expensive, but also complicated with one or more sources of noise. One technique to deal with high computational overhead is parallelization. However, though the existing literature gives good insights about the expected behavior of parallelized evolutionary algorithms, we still lack an understanding of their performance in the presence of noise. This paper considers how parallelization might be leveraged together with multi-parent crossover in order to handle noisy problems. We present a rigorous running time analysis of an island model with weakly connected topology tasked with hill climbing in the presence of general additive noise (i.e., noisy OneMax ). Our proofs yield insights into the relationship between the noise intensity and number of required parents. We translate this into positive and negative results for two kinds of multi-parent crossover operators. We then empirically analyze and extend this framework to investigate the trade-offs between noise impact, optimization time, and limits of computation power to deal with noise.

Evolutionary and Estimation of Distribution Algorithms for Unconstrained, Constrained, and Multiobjective Noisy Combinatorial Optimisation Problems.

We present an empirical study of a range of evolutionary algorithms applied to various noisy combinatorial optimisation problems. There are three sets of experiments. The first looks at several toy problems, such as OneMax and other linear problems. We find that UMDA and the Paired-Crossover Evolutionary Algorithm (PCEA) are the only ones able to cope robustly with noise, within a reasonable fixed time budget. In the second stage, UMDA and PCEA are then tested on more complex noisy problems: SubsetSum, Knapsack, and SetCover. Both perform well under increasing levels of noise, with UMDA being the better of the two. In the third stage, we consider two noisy multiobjective problems (CountingOnesCountingZeros and a multiobjective formulation of SetCover). We compare several adaptations of UMDA for multiobjective problems with the Simple Evolutionary Multiobjective Optimiser (SEMO) and NSGA-II. We conclude that UMDA, and its variants, can be highly effective on a variety of noisy combinatorial optimisation, outperforming many other evolutionary algorithms.

A Multi-Scale Hybrid Attention Network for Sentence Segmentation Line Detection in Dongba Scripture

Dongba scripture sentence segmentation is an important and basic work in the digitization and machine translation of Dongba scripture. Dongba scripture sentence segmentation line detection (DS-SSLD) as a core technology of Dongba scripture sentence segmentation is a challenging task due to its own distinctiveness, such as high inherent noise interference and nonstandard sentence segmentation lines. Recently, projection-based methods have been adopted. However, these methods are difficult when dealing with the following two problems. The first is the noisy problem, where a large number of noise in the Dongba scripture image interference detection results. The second is the Dongba scripture inherent characteristics, where many vertical lines in Dongba hieroglyphs are easily confused with the vertical sentence segmentation lines. Therefore, this paper aims to propose a module based on the convolutional neural network (CNN) to improve the accuracy of DS-SSLD. To achieve this, we first construct a tagged dataset for training and testing DS-SSLD, including 2504 real images collected from Dongba scripture books and sentence segmentation targets. Then, we propose a multi-scale hybrid attention network (Multi-HAN) based on YOLOv5s, where a multiple hybrid attention unit (MHAU) is used to enhance the distinction between important features and redundant features, and the multi-scale cross-stage partial unit (Multi-CSPU) is used to realize multi-scale and richer feature representation. The experiment is carried out on the Dongba scripture sentence segmentation dataset we built. The experimental results show that the proposed method exhibits excellent detection performance and outperforms several state-of-the-art methods.

Least Squares Polynomial Chaos Regression for Stochastic Analysis of Transmission Lines Without and With Noise

Stochastic transmission line (TL) analysis is often challenging due to the difficulty of fully identifying the probability distributions of all randomly varying parameters, especially in the presence of noise. These problems cannot be directly addressed by the existing polynomial chaos expansion (PCE) framework. To overcome the limitations, this paper proposes an improved PCE method for analyzing the stochastic TL model with noise. A regression-based non-intrusive approach termed least squares polynomial chaos regression (LSPCR) is employed to determine the PCE coefficients. Six algorithms for LSPCR are developed by respectively combining three sampling strategies, i.e., standard sampling (SS), asymptotic sampling (AS), and coherence-optimal sampling (COS), with two types of norm problems: the least-squares optimization problem (LSO) and the l1-minimum problem (l1-M). Numerical experiments are conducted on noiseless and noisy problems. The results reveal that the LSO-based algorithms (LSOs) are much faster than the l1-M-based algorithms (l1-Ms), regardless of the sampling strategy. In the absence of noise, these six algorithms require only a relatively small number of samples to achieve higher accuracy in solution moments compared to the Stochastic Galerkin (SG) method. When the sample size is large, the AS-based algorithms (ASs) and the COS-based algorithms (COSs) yield more accurate results than the SS-based algorithms (SSs), regardless of the norm problems. In the presence of noise, the LSOs outperform the l1-Ms for small sample sizes. However, no evident differences are observed among the six algorithms with large sample sizes.

Personalized Bayesian optimization for noisy problems

In many real-world applications of interest, several related optimization tasks can be encountered, where each task is associated with a specific context or personalized information. Moreover, the amount of available data for each task may be highly limited due to the expensive cost involved. Although Bayesian optimization (BO) has emerged as a promising paradigm for handling black-box optimization problems, addressing such a sequence of optimization tasks can be intractable due to the cold start issues in BO. The key challenge is to speed up the optimization by leveraging the transferable information, while taking the personalization into consideration. In this paper, optimization problems with personalized variables are formally defined at first. Subsequently, a personalized evolutionary Bayesian algorithm is proposed to consider the personalized information and the measurement noise. Specifically, a contextual Gaussian process is used to jointly learn a surrogate model in different contexts with regard to the varying personalized parameter, and an evolutionary algorithm is tailored for optimizing an acquisition function for handling the presence of personalized information. Finally, we demonstrate the effectiveness of the proposed algorithm by testing it on widely used single- and multi-objective benchmark problems with personalized variables.

ZOOpt: a toolbox for derivative-free optimization

Recent advances in derivative-free optimization allow efficient approximation of the global-optimal solutions of sophisticated functions, such as functions with many local optima, non-differentiable and non-continuous functions. This article describes the ZOOpt (Zeroth Order Optimization) toolbox that provides efficient derivative-free solvers and is designed easy to use. ZOOpt provides single-machine parallel optimization on the basis of python core and multi-machine distributed optimization for time-consuming tasks by incorporating with the Ray framework -- a famous platform for building distributed applications. ZOOpt particularly focuses on optimization problems in machine learning, addressing high-dimensional and noisy problems such as hyper-parameter tuning and direct policy search. The toolbox is maintained toward a ready-to-use tool in real-world machine learning tasks.

Adaptive Weighted Strategy Based Integrated Surrogate Models for Multiobjective Evolutionary Algorithm

Although the integrated model has good convergence ability, it is difficult to solve the multimodal problem and noisy problem due to the lack of uncertainty evaluation. Radial basis function model performs best for different degrees of nonlinear problems with small-scale and noisy training datasets but is insensitive to the increase of decision-space dimension, while Gaussian process regression model can provide prediction fitness and uncertainty evaluation. Therefore, an adaptive weighted strategy based integrated surrogate models is proposed to solve noisy multiobjective evolutionary problems. Based on the indicator-based multiobjective evolutionary framework, our proposed algorithm introduces the weighted combination of radial basis function and Gaussian process regression, and U-learning sampling scheme is adopted to improve the performance of population in convergence and diversity and judge the improvement of convergence and diversity. Finally, the effectiveness of the proposed algorithm is verified by 12 benchmark test problems, which are applied to the hybrid optimization problem on the construction of samples and the determination of parameters. The experimental results show that our proposed method is feasible and effective.

ANALYSING THE OPERATIONAL RISK INVOLVED IN THE MANUFACTURING OF AUTOMOBILES

Risk management is an important issue in manufacturing companies in to‐ day’s competitive market. Failure modes and effects analysis (FMEA) method is a risk management tool to stabilize production and enhance market com‐ punitiveness by using risk priority numbers (RPN). Although the traditional FMEA approach is a successfully and commonly used method, it has some shortcomings such as notion of equal importance of the factors, severity, incident and delectability, and not following the ordered weighted rule. Thus, in order to improve RPN, an added method blending grey personal analysis (GRA) with FMEA is used in this study. The purpose of this paper is to contribute to risk management activities by proposing solutions to assembly line challenges in an automotive manufacturing company by using combined GRA and FMEA method. In the proposed method, the priorities of production failures were determined by GRA method, and these failures were minimized by using FMEA technique. The study results indicated the actions that lead to enhancement in the product. The accomplishment of cor‐ restive/preventive activities resulted in 96 % development in door seal cuts problem caused by the door step building. Door seal cuts problem caused by instrument panel assembly and the noisy door window problem are solved totally

Resilient Biomedical Systems Design Under Noise Using Logic-Based Machine Learning

Increased reliance on electronic health records and plethora of new sensor technologies has enabled the use of machine learning (ML) in medical diagnosis. This has opened up promising opportunities for faster and automated decision making, particularly in early and repetitive diagnostic routines. Nevertheless, there are also increased possibilities of data aberrance arising from environmentally induced noise. It is vital to create ML models that are resilient in the presence of data noise to minimize erroneous classifications that could be crucial. This study uses a recently proposed ML algorithm called the Tsetlin machine (TM) to study the robustness against noise-injected medical data. We test two different feature extraction methods, in conjunction with the TM, to explore how feature engineering can mitigate the impact of noise corruption. Our results show the TM is capable of effective classification even with a signal-to-noise ratio (SNR) of −15dB as its training parameters remain resilient to noise injection. We show that high testing data sensitivity can still be possible at very low SNRs through a balance of feature distribution–based discretization and a rule mining algorithm used as a noise filtering encoding method. Through this method we show how a smaller number of core features can be extracted from a noisy problem space resulting in reduced ML model complexity and memory footprint—in some cases up to 6x fewer training parameters while retaining equal or better performance. In addition, we investigate the cost of noise resilience in terms of energy when compared with recently proposed binarized neural networks.

Inter-nucleosomal potentials from nucleosomal positioning data

No systematic method exists to derive inter-nucleosomal potentials between nucleosomes along a chromosome consistently across a given genome. Such potentials can yield information on nucleosomal ordering, thermal as well as mechanical properties of chromosomes. Thus, indirectly, they shed light on a possible mechanical genomic code along a chromosome. To develop a method yielding effective inter-nucleosomal potentials between nucleosomes, a generalized Lennard-Jones potential for the parameterization is developed based on nucleosomal positioning data. This approach eliminates some of the problems that the underlying nucleosomal positioning data have, rendering the extraction difficult on the individual nucleosomal level. Furthermore, patterns on which to base a classification along a chromosome appear on larger domains, such as hetero- and euchromatin. An intuitive selection strategy for the noisy optimization problem is employed to derive effective exponents for the generalized potential. The method is tested on the Candida albicans genome. Applying k-means clustering based on potential parameters and thermodynamic compressibilities, a genome-wide clustering of nucleosome sequences is obtained for C. albicans. This clustering shows that a chromosome beyond the classical dichotomic categories of hetero- and euchromatin is more feature-rich.

A New General Type-2 Fuzzy Predictive Scheme for PID Tuning

The proportional-integral-derivative controller is widely used in various industrial applications. But, in many noisy problems the strong methods are needed to optimize the proportional-integral-derivative parameters. In this paper, a novel method is introduced for adjusting the proportional-integral-derivative parameters through the model predictive control and generalized type-2 fuzzy-logic systems. The rules of suggested fuzzy system are online adjusted and the parameters of proportional-integral-derivative are tuned based on the fuzzy model such that a cost function to be minimized. The designed controller is applied on continuous stirred tank reactor and the performance is compared with other traditional approaches. The main advantages are that the accuracy is improved by online modeling and optimization and a predictive scheme is added to the conventional proportional-integral-derivative controller.

Semrank

Popular real-world events often create huge traffic on Twitter including real-time updates of important moments, personal comments, and so on while the event is happening. Most of the users are interested to read the important tweets that possibly include important moments of that event. However, extracting the relevant tweets of any event is a challenging task due to the endless stream of noisy tweets and vocabulary variation problem of social media content. To handle these challenges, the authors introduce a new approach for computing the relative tweet importance based on the concept of the Pagerank algorithm where adjacency matrix of the graph representation of tweets contains semantic similarity matrix based on the word mover's distance measure utilizing Word2Vec word embedding model. The results show that top-ranked tweets generated by the proposed approach are more concise and news-worthy than baseline approaches.

A variational toolbox for quantum multi-parameter estimation

With an ever-expanding ecosystem of noisy and intermediate-scale quantum devices, exploring their possible applications is a rapidly growing field of quantum information science. In this work, we demonstrate that variational quantum algorithms feasible on such devices address a challenge central to the field of quantum metrology: The identification of near-optimal probes and measurement operators for noisy multi-parameter estimation problems. We first introduce a general framework that allows for sequential updates of variational parameters to improve probe states and measurements and is widely applicable to both discrete and continuous-variable settings. We then demonstrate the practical functioning of the approach through numerical simulations, showcasing how tailored probes and measurements improve over standard methods in the noisy regime. Along the way, we prove the validity of a general parameter-shift rule for noisy evolutions, expected to be of general interest in variational quantum algorithms. In our approach, we advocate the mindset of quantum-aided design, exploiting quantum technology to learn close to optimal, experimentally feasible quantum metrology protocols.

Discriminative feature learning for underwater fish recognition

Underwater fish recognition is an important task in fish stock assessment and marine ecosystem studies. Machine learning techniques have been applied to train high-performance fish recognition models from underwater images. However, underwater images often contain extremely noisy backgrounds, hindering the training of accurate recognition models. Traditional methods exploit handcrafted features to train traditional classifiers. These methods often suffer from low recognition accuracy and limited scalability to large-scale datasets. While deep learning approaches have been proposed, the challenge of learning with noisy underwater images has not yet been fully addressed. We propose a discriminative feature learning (DFL) framework to train accurate fish recognition models on noisy underwater images. By leveraging the idea of contrastive learning, DFL encourages the model to learn more discriminative features for images in different classes and similar features for images in the same class. To better address the noisy background problem, DFL also utilizes a regularization technique called attention suppression to prevent the model from paying too much attention to the noisy background. Experimental results on three benchmark datasets validate the superior performance of DFL over the current state-of-the-art deep learning approaches.

Improving Distant Supervised Relation Extraction with Noise Detection Strategy

Distant supervised relation extraction (DSRE) is widely used to extract novel relational facts from plain text, so as to improve the knowledge graph. However, distant supervision inevitably suffers from the noisy labeling problem that will severely damage the performance of relation extraction. Currently, most DSRE methods are mainly focused on reducing the weights of noisy sentences, ignoring the bag-level noise where all sentences in a bag are wrongly labeled. In this paper, we present a novel noise detection-based relation extraction approach (NDRE) to automatically detect noisy labels with entity information and dynamically correct them, which can alleviate both instance-level and bag-level noisy problems. By this means, we can extend the dataset from the Web tables without introducing more noise. In this approach, to embed the semantics of sentences from corpus and web tables, we firstly propose a powerful sentence coder that employs an internal multi-head self-attention mechanism between the piecewise max-pooling convolutional neural network. Second, we adopt a noise detection strategy, which is expected to dynamically detect and correct the original noisy label according to the similarity between sentence representation and entity-aware embeddings. Then, we aggregate the information from corpus and web tables to make the final relation prediction. Experimental results on a public benchmark dataset demonstrate that our proposed approach achieves significant improvements over the state-of-the-art baselines and can effectively reduce the noisy labeling problem.

On the effect of linear feedback and parametric pumping on a resonator’s frequency stability

Resonant sensors based on micro- and nano-electro mechanical systems (M/NEMS) are ubiquitous in many sensing applications due to their outstanding performance capabilities, which are directly proportional to the quality factor (Q) of the devices. We address here a recurrent question in the field: do dynamical techniques that modify the effective Q (namely parametric pumping and direct drive velocity feedback) affect the performance of said sensors? We develop analytical models of both cases, while remaining in the linear regime, and introduce noise in the system from two separate sources: thermomechanical and amplifier (read-out) noise. We observe that parametric pumping enhances the quality factor in the amplitude response, but worsens it in the phase response on the resonator. In the case of feedback, we find that Q is enhanced in both cases. Then, we establish a solution for the noisy problem with direct drive and parametric pumping simultaneously. We also find that, in the case when thermomechanical noise dominates, no benefit can be obtained from either artificial Q-enhancement technique. However, in the case when amplifier noise dominates, we surprisingly observe that a significant advantage can only be achieved using parametric pumping in the squeezing region.

Applying the pattern search implicit filtering algorithm for solving a noisy problem of parameter identification

Our contribution in this paper is twofold. First, the global convergence analysis of the recently proposed pattern search implicit filtering algorithm (PSIFA), aimed at linearly constrained noisy minimization problems, is revisited to address more general locally Lipschitz objective functions corrupted by noise. Second, PSIFA is applied for solving the damped harmonic oscillator parameter identification problem. This problem can be formulated as a linearly constrained optimization problem, for which the constraints are related to the features of the damping. Such a formulation rests upon a very expensive objective function whose evaluation comprises the numerical solution of an ordinary differential equation (ODE), with intrinsic numerical noise. Computational experimentation encompasses distinct choices for the ODE solvers, and a comparative analysis of the most effective options against the pattern search and the implicit filtering algorithms.

Minimization of the difference of Nuclear and Frobenius norms for noisy low rank matrix recovery

This paper considers recovery of matrices that are low rank or approximately low rank from linear measurements corrupted with additive noise. We study minimization of the difference of Nuclear and Frobenius norms (abbreviated as [Formula: see text] norm) as a nonconvex and Lipschitz continuous metric for solving this noisy low rank matrix recovery problem. We mainly study two types of bounded observation noisy low rank matrix recovery problems, including the [Formula: see text]-norm bounded noise and the Dantizg Selector noise. Based on the matrix restricted isometry property (abbreviated as M-RIP), we prove that this [Formula: see text] norm-based minimization method can stably recover a (approximately) low rank matrix in the two types bounded noisy low rank matrix recovery problems. In addition, we use the truncated difference of Nuclear and Frobenius norms (denoted as the truncated [Formula: see text] norm) to recover a low rank matrix when the observation noise is the Dantizg Selector noise. We give the stable recovery result for this truncated [Formula: see text] norm minimization in Dantizg Selector noise case when the linear measurement map satisfies the M-RIP condition.