Regularization Constant Research Articles

Nonlocal implicit gradient enhancements for strain localization informed by controllability criteria for plastic solids

Nonlocal implicit gradient enhancements are widely used to suppress mesh dependency in simulations involving strain localization. For example, nonlocality is often introduced through internal variables that account for possible material softening. This work, however, shows that this approach may become ineffective if the solid displays plastic non-normality (i.e., non-associated plastic flow). For this purpose, we consider an over-nonlocal formulation and mathematically inspect the conditions at which regularization is lost in the presence of plastic non-normality. Specifically, such loss of regularization is linked to the loss of uniqueness and/or existence of the incremental plastic response that is kinematically compatible with the development of a deformation band. By doing so, we find a lower limit for the admissibility of the parameters controlling the effectiveness of nonlocal implicit gradient regularization. Furthermore, we show that such a lower limit is regulated by a plastic modulus reflecting the loss of controllability of the constitutive response, and, hence, depends on the degree of plastic non-normality. We also derive a closed-form expression relating the thickness of the deformation band to both the controllability modulus and gradient regularization constants, which suggests that the thickness of the process zone may change in response to the prevailing plastic flow characteristics and evolve during active plastic deformation. The proposed nonlocal enhancement is applied to a non-associated elasto-plastic model for porous sedimentary rocks, which is capable of displaying both shear-dominated and compaction-dominated bands. Numerical simulations reveal that effective regularization can be enforced only when the over-nonlocal weighting coefficient is larger than the above-mentioned lower limit.

Population-based change-point detection for the identification of homozygosity islands.

This work is motivated by the problem of identifying homozygosity islands on the genome of individuals in a population. Our method directly tackles the issue of identification of the homozygosity islands at the population level, without the need of analysing single individuals and then combine the results, as is made nowadays in state-of-the-art approaches. We propose regularized offline change-point methods to detect changes in the parameters of a multidimensional distribution when we have several aligned, independent samples of fixed resolution. We present a penalized maximum likelihood approach that can be efficiently computed by a dynamic programming algorithm or approximated by a fast binary segmentation algorithm. Both estimators are shown to converge almost surely to the set of change-points without the need of specifying a priori the number of change-points. In simulation, we observed similar performances from the exact and greedy estimators. Moreover, we provide a new methodology for the selection of the regularization constant which has the advantage of being automatic, consistent, and less prone to subjective analysis. The data used in the application are from the Human Genome Diversity Project (HGDP) and is publicly available. Algorithms were implemented using the R software R Core Team (R: A Language and Environment for Statistical Computing. Vienna (Austria): R Foundation for Statistical Computing, 2020.) in the R package blockcpd, found at https://github.com/Lucas-Prates/blockcpd.

Regularization-free multicriteria optimization of polymer viscoelasticity model

This paper introduces a multiobjective optimization (MOP) method for nonlinear regression analysis which is capable of simultaneously minimizing the model order and estimating parameter values without the need of exogenous regularization constraints. The method is introduced through a case study in polymer rheology modeling. Prevailing approaches in this field tackle conflicting optimization goals as a monobjective problem by aggregating individual regression errors on each dependent variable into a single weighted scalarization function. In addition, their supporting deterministic numerical methods often rely on assumptions which are extrinsic to the problem, such as regularization constants and restrictions on parameter distribution, thereby introducing methodology inherent biases into the model. Our proposed non-deterministic MOP strategy, on the other hand, aims at finding the Pareto-front of all optimal solutions with respect not only to individual regression errors, but also to the number of parameters needed to fit the data, automatically reducing the model order. The evolutionary computation approach does not require arbitrary constraints on objective weights, regularization parameters or other exogenous assumptions to handle the ill-posed inverse problem. The article discusses the method rationales, implementation, simulation experiments, and comparison with other methods, with experimental evidences that it can outperform state-of-art techniques. While the discussion focuses on the study case, the introduced method is general and immediately applicable to other problem domains.

Ovarian cancer detection using optimized machine learning models with adaptive differential evolution

Ovarian cancer is one of the leading causes of mortality among women. The most common detection approach is by tracking related biomarkers to see whether patient has ovarian cancer. Despite emphasis on biomarkers, feature subsets may include physiological features. In this current research, we propose an optimal simultaneous feature weighting/parameter optimization for detecting ovarian cancer occurrence. The weights are optimized using adaptive differential evolution (ADE) with cross validation error and least absolute shrinkage and selection operator (LASSO) regularization as the fitness function. Furthermore, to be in line with the Occam’s razor principle, we apply LASSO regularization in the form of weighted ℓ1 norm to reduce features being selected. The regularization constant, λ, is integral to balance the feature weights and the cross validation error. Using ADE method and, in particular LASSO regularization, we obtain the results showing clear reduction and elimination of features from feature subset. Using K-nearest neighbors (KNN) with λ=0.01, 97.24% accuracy (mean) is obtained. Meanwhile, when using support vector machines (SVM) with λ=0.01, an accuracy of 96.48% (mean) is achieved. From the viewpoint of machine learning, this, at least, suggests that the proposed approach may be a good solution for optimizing feature weights/parameters at the same time. From the perspective of cancer research, it demonstrates that a plethora of features may be valuable in building a model and should not be discarded for having weak discriminative capability. As a result, the combination of feature subset and non linear mapping plays a significant role.

Adaptive Tikhonov strategies for stochastic ensemble Kalman inversion

Ensemble Kalman inversion (EKI) is a derivative-free optimizer aimed at solving inverse problems, taking motivation from the celebrated ensemble Kalman filter. The purpose of this article is to consider the introduction of adaptive Tikhonov strategies for EKI. This work builds upon Tikhonov EKI (TEKI) which was proposed for a fixed regularization constant. By adaptively learning the regularization parameter, this procedure is known to improve the recovery of the underlying unknown. For the analysis, we consider a continuous-time setting where we extend known results such as well-posedness and convergence of various loss functions, but with the addition of noisy observations for the limiting stochastic differential equations (i.e. stochastic TEKI). Furthermore, we allow a time-varying noise and regularization covariance in our presented convergence result which mimic adaptive regularization schemes. In turn we present three adaptive regularization schemes, which are highlighted from both the deterministic and Bayesian approaches for inverse problems, which include bilevel optimization, the maximum a posteriori formulation and covariance learning. We numerically test these schemes and the theory on linear and nonlinear partial differential equations, where they outperform the non-adaptive TEKI and EKI.

NBI Suppression Method for SAR Based on Sparse Segmentation Search

In remote sensing research, narrowband interference (NBI) suppression in synthetic aperture radar (SAR) is an urgent problem. Recently, many methods on NBI suppression are proposed via sparse recovery. In these methods, the optimal regularization constants are always hard to choose. Moreover, the number of NBI signals, equaling to the sparsity of the sparse vector, may change at different pulses, and the suppression performance might be reduced due to the regularization constants which control the sparsity of the sparse vector, are fixed. In this letter, aiming at these problems, a NBI suppression method for SAR based on sparse segmentation search (SSS) is proposed. Firstly, we build a non-convex optimization model without the regularization constant. Then the adaptive linear enhancer (ALE) is used to convert the non-convex model to a convex one. Finally, we solve this convex model and suppress NBI signals. The real-world SAR data experiments illustrate the effectiveness of the proposed method.

The Equation of the Set of Natural Numbers Just to Sum

Systems of equations of the form X = Y + Z and X = C, in which the unknowns are sets of integers,”+” denotes pairwise sum of sets S + T = m + n m S, n T , and C is an ultimately periodic constant. When restricted to sets of natural numbers, such equations can be equally seen as language equations over a one-letter alphabet with concatenation and regular constants, and it is shown that such systems are computationally universal, in the sense that for every recursive set S N there exists a system with a unique solution containing T with S = n 16n + 13 T. For systems over sets of all integers, both positive and negative, there is a similar construction of a system with a unique solution S = {n|16n ∈ T} representing any hyper-arithmetical set S ⊆ N. Keywords: Language equations, Natural numbers, Equations of natural number.

Practical Treatment of the Multicollinearity: The Optimal Ridge Method and the Modified OLS

The paper discusses the applicability of the two main methods for solving the linear regression (LR) problem in the presence of multicollinearity – the OLS and the ridge methods. We compare the solutions obtained by these methods with the solution calculated by the Modified OLS (MOLS) [1; 2]. Like the ridge, the MOLS provides a stable solution for any level of data collinearity. We compare three approaches by using the Monte Carlo simulations, and the data used is generated by the Artificial Data Generator (ADG) [1; 2]. The ADG produces linear and nonlinear data samples of arbitrary size, which allows the investigation of the OLS equation's regularization problem. Two possible regularization versions are the COV version considered in [1; 2] and the ST version commonly used in the literature and practice. The performed investigations reveal that the ridge method in the COV version has an approximately constant optimal regularizer (?_ridge^((opt))?0.1) for any sample size and collinearity level. The MOLS method in this version also has an approximately constant optimal regularizer, but its value is significantly smaller (?_MOLS^((opt))?0.001). On the contrary, the ridge method in the ST version has the optimal regularizer, which is not a constant but depends on the sample size. In this case, its value needs to be set to ?_ridge^((opt))?0.1(n-1). With such a value of the ridge parameter, the obtained solution is strictly the same as one obtained with the COV version but with the optimal regularizer ?_ridge^((opt))?0.1 [1; 2]. With such a choice of the regularizer, one can use any implementation of the ridge method in all known statistical software by setting the regularization parameter ?_ridge^((opt))?0.1(n-1) without extra tuning process regardless of the sample size and the collinearity level. Also, it is shown that such an optimal ridge(0.1) solution is close to the population solution for a sample size large enough, but, at the same time, it has some limitations. It is well known that the ridge(0.1) solution is biased. However, as it has been shown in the paper, the bias is economically insignificant. The more critical drawback, which is revealed, is the smoothing of the population solution – the ridge method significantly reduces the difference between the population regression coefficients. The ridge(0.1) method can result in a solution, which is economically correct, i.e., the regression coefficients have correct signs, but this solution might be inadequate to a certain extent. The more significant the difference between the regression coefficients in the population, the more inadequate is the ridge(0.1) method. As for the MOLS, it does not possess this disadvantage. Since its regularization constant is much smaller than the corresponding ridge regularizer (0.001 versus 0.1), the MOLS method suffers little from both the bias and smoothing of its solutions. From a practical point of view, both the ridge(0.1) and the MOLS methods result in close stable solutions to the LR problem for any sample size and collinearity level. With the sample size increasing, both solutions approach the population solution. We also demonstrate that for a small sample size of less than 40, the ridge(0.1) method is preferable, as it is more stable. When the sample size is medium or large, it is preferable to use the MOLS as it is more accurate yet has approximately the same stability.

Self-Optimizing Support Vector Elastic Net.

Chemometrics is widely used to solve various quantitative and qualitative problems in analytical chemistry. A self-optimizing chemometrics method facilitates scientists to exploit the advantages of chemometrics. In this report, a parameter-free support vector elastic net that self-optimizes two key regularization constants, i.e., λ for L2 regularization and t for L1 regularization, is developed and referred to as self-optimizing support vector elastic net (SOSVEN). Response surface modeling (RSM) and bootstrapped Latin partitions (BLPs) are incorporated for the optimization. Responses at a set of design points over the ranges of the two factors are evaluated with an internal BLP validation using a calibration set. A 2-dimensional interpolation with a cubic spline fits a response surface to determine the best condition that gives the best-estimated response. The SOSVEN with RSM had comparable performances with the one tuned by grid search, while the RSM is more efficient. The developed SOSVEN was compared with two parameter-free chemometrics methods, super partial least-squares regression (sPLSR) and super support vector regression (sSVR) for calibration, and sPLS-discriminant analysis (sPLS-DA) and support vector classification (SVC) for classification. For calibration, the SOSVEN with RSM worked equivalently well or better than the other two self-optimizing methods for the evaluations using meat and hemp oil data sets. For classification, a reference wine data set and mass spectra of different marijuana extracts were used. The three classifiers had similar performances to identify the cultivars of wines with nearly 98% of accuracy. The SOSVEN significantly outperformed sPLS-DA and SVC to classify the mass spectra of marijuana extracts with an overall accuracy of 97%. These results demonstrated excellent abilities of SOSVEN for classification and calibration.

An enhanced extrapolation method based on particle swarm optimization-support vector regression to determine the friction coefficient between aircraft tire and runway surface.

This paper proposes a hybrid pretreating method in the support vector regression (SVR) machines to enhance the extrapolated performance of the traditional method based on particle swarm optimization (PSO)-SVR. This method is introduced into a novel domain wherein the friction coefficient (COF) is extrapolated between the aircraft tire and the runway surface. It includes two parts: the normalized data and the extrapolated method for COF. The first part develops a novel data normalized method, which allows the experiment data to follow sine distribution (SN) instead of traditional linear distribution. For the second part, the SVR with the training set initially extrapolates a COF, and the COF is subsequently added into the original training set to build a new training sample. This process is repeated until the extrapolation is finished. This method is named step-wise (SW) extrapolation. PSO is used to optimize the regularization constant C, the parameter gamma γ, and epsilon parameter ε in SVR. Finally, this study indicates that the proposed method (SN-SW-PSO-SVR) is more suitable for the extrapolation of the COF between the airplane tire and the runway surface than other extrapolated methods.

Stochastic gravitational wave background mapmaking using regularized deconvolution

Obtaining a faithful source intensity distribution map of the sky from noisy data demands incorporating known information of the expected signal, especially when the signal is weak compared to the noise. We introduce a widely used procedure to incorporate these priors through a Bayesian regularisation scheme in the context of map-making of the anisotropic stochastic GW background (SGWB). Specifically, we implement the quadratic form of regularizing function with varying strength of regularization and study its effect on image restoration for different types of the injected source intensity distribution in simulated LIGO data. We find that regularization significantly enhances the quality of reconstruction, especially when the intensity of the source is weak, and dramatically improves the stability of deconvolution. We further study the quality of reconstruction as a function of regularization constant. While in principle this constant is dependent on the data set, we show that the deconvolution process is robust against the choice of the constant, as long as it is chosen from a broad range of values obtained by the method presented here.

On Some Formulas for Kaprekar Constants

Let b ≥ 2 and n ≥ 2 be integers. For a b-adic n-digit integer x, let A (resp. B) be the b-adic n-digit integer obtained by rearranging the numbers of all digits of x in descending (resp. ascending) order. Then, we define the Kaprekar transformation T ( b , n ) ( x ) : = A - B . If T ( b , n ) ( x ) = x , then x is called a b-adicn-digit Kaprekar constant. Moreover, we say that a b-adic n-digit Kaprekar constant x is regular when the numbers of all digits of x are distinct. In this article, we obtain some formulas for regular and non-regular Kaprekar constants, respectively. As an application of these formulas, we then see that for any integer b ≥ 2 , the number of b-adic odd-digit regular Kaprekar constants is greater than or equal to the number of all non-trivial divisors of b. Kaprekar constants have the symmetric property that they are fixed points for recursive number theoretical functions T ( b , n ) .

Site-directed insertion: Language equations and decision problems

Site-directed insertion is an overlapping insertion operation that can be viewed as analogous to the overlap assembly or chop operations that concatenate strings by overlapping a suffix and a prefix of the argument strings. We consider decision problems and language equations involving site-directed insertion. By relying on the tools provided by semantic shuffle on trajectories (M. Domaratzki, Developments in Language Theory 2004) we show that one variable equations involving site-directed insertion and regular constants can be solved algorithmically. We consider also maximal and minimal variants of the site-directed insertion operation and the nondeterministic state complexity of site-directed insertion.

Multilabel Feature Extraction Algorithm via Maximizing Approximated and Symmetrized Normalized Cross-Covariance Operator.

Multilabel feature extraction (FE) is an effective preprocessing step to cope with some possible irrelevant, redundant, and noisy features, to reduce computational costs and even improve classification performance. Original normalized cross-covariance operator represents a kernel-based nonlinear dependence measure between features and labels, whose empirical estimator is formulated as a trace operation including two inverse matrices of feature and label kernels with a regularization constant. Due to such a complicated expression, it is impossible to derive an eigenvalue problem for linear FE directly. In this paper, we approximate this measure using Moore-Penrose inverse matrix, linear kernel for feature space, and delta kernel for label space, and then symmetrize the entire matrix in the trace operation, resulting in an effective approximated and symmetrized representation. According to orthonormal projection direction constraints, maximizing such a modified form induces a novel eigenvalue problem for multilabel linear FE. Experiments on 12 data sets illustrate that our proposed method works the best, compared with seven existing FE techniques, according to eight multilabel classification performance metrics and three statistical tests.

Sentiment analysis using convolutional neural network via word embeddings

Convolutional neural networks are known for their excellent performance in computer vision, achieving results in the state of the art. Moreover, recent research has shown that these networks can also provide promising results for natural language processing. In this case, the basic idea is to concatenate the vector representations of words into a single block and use it as an image. However, despite the good results, the problem of using convolution networks is the large numbers of design decisions that need to be made a priori. These models require the definition of many hyper-parameters, including the type of word embeddings, which consists of the data vectorized representation, the activation function that prints the non-linearity characteristics to the model, the size of the filter that applies data convolution, the number of feature maps, which are responsible for identifying the attributes and the pooling method used for data reduction. In addition, one must also predefine the regularization constant and the dropout rate, which are responsible for avoiding any network over-fitting. In existing research works, convolutional neural network architectures capable of overcoming the performance of traditional machine learning models are presented. Even though these can compete with more complex models, the problem of how the different setting of the hyper-parameters may affect the performance of this type of network has not yet been explored. In this paper, we propose an efficient sentiment analysis classifier using convolutional neural networks by analyzing the impact of the hyper-parameters on the model performance. The main interest in analyzing sentiment comes from the advent of social media and the technological advances that flood the Internet with opinions. Nonetheless, mining the Internet for opinion and sentiment analysis is not an easy task and thus needs outstanding models with the best hyper-parameters setting to be able to get pertinent answers. The results achieved are obtained with the use of GPU and show that the different configurations exceed the reference models in the most of the cases with gains of up to 18% and have similar performance to the models of the state of the art with gains of up to 2% in some cases.

Robust fault detection of piecewise affine ARX systems

ABSTRACT In this paper, regularisation-based robust fault detection technique for PWARX (PieceWise affine Auto Regressive eXogenous) systems is discussed. The problem is formulated in convex optimisation form using regularisation hence, global solution can be found. The regularisation constant plays a key role for finding the number of submodels. The MSE (mean square error) value between actual system and its model under no fault is used as a threshold for tracking dynamics change and detecting fault. The simplicity of turning only one parameter (regularisation constant) outstands this approach from others. The proposed algorithm also shows the capability of separating the dynamics change from additive fault under noise. One illustration concludes this proposal by demonstrating its effectiveness.

On Regularization of Second Kind Integrals

We obtain expressions for second kind integrals on non-hyperelliptic $(n,s)$-curves. Such a curve possesses a Weierstrass point at infinity which is a branch point where all sheets of the curve come together. The infinity serves as the basepoint for Abel's map, and the basepoint in the definition of the second kind integrals. We define second kind differentials as having a pole at the infinity, therefore the second kind integrals need to be regularized. We propose the regularization consistent with the structure of the field of Abelian functions on Jacobian of the curve. In this connection we introduce the notion of regularization constant, a uniquely defined free term in the expansion of the second kind integral over a local parameter in the vicinity of the infinity. This is a vector with components depending on parameters of the curve, the number of components is equal to genus of the curve. Presence of the term guarantees consistency of all relations between Abelian functions constructed with the help of the second kind integrals. We propose two methods of calculating the regularization constant, and obtain these constants for $(3,4)$, $(3,5)$, $(3,7)$, and $(4,5)$-curves. By the example of $(3,4)$-curve, we extend the proposed regularization to the case of second kind integrals with the pole at an arbitrary fixed point. Finally, we propose a scheme of obtaining addition formulas, where the second kind integrals, including the proper regularization constants, are used.

Energy of periodic discrete dislocation networks

We present an approach to compute the stored energy associated with arbitrary discrete dislocation networks subjected to periodic boundary conditions. To circumvent the issue of conditional convergence while keeping the computational cost tractable, we develop a regularization procedure that involves two equivalent measures of the dislocation network energy. Taking advantage of the non-singular formulation, the energy is first evaluated by explicitly calculating the conditionally convergent sum of all interactions between dislocation segments. Regularization constants are then determined by volume integral of the smooth elastic stress fields produced for large values of the dislocation core radius. The approach is employed to investigate the stored energy of a series of idealized dislocation configurations and large-scale networks generated by discrete dislocation dynamics (DDD) simulations. It is found that (1) the stored energies predicted by DDD simulations are in good agreement with experimental measurements of about 5% of the work done, and (2) Taylor lattice configurations provide surprisingly good energetics models for complex DDD networks, thereby confirming the screening of long-range stresses in work-hardened dislocation structures.

Statistics on noise covariance matrix for covariance fitting-based compressive sensing direction-of-arrival estimation algorithm: For use with optimization via regularization.

A covariance fitting algorithm for the estimation of direction-of-arrivals of multiple incident signals is addressed in this paper. The scheme takes advantage of the fact that the incident signals are spatially sparse. A previous study has presented the regularization parameters of the covariance fitting for a very large number of snapshots. In this paper, a strategy on how to determine the regularization constant of the covariance fitting for a general number of snapshots is presented. The strategy essentially exploits the norm of the noise covariance matrix. The proposed algorithm has been validated via numerical simulations.

Compressive Sensing with Cross-Validation and Stop-Sampling for Sparse Polynomial Chaos Expansions

Compressive sensing is a powerful technique for recovering sparse solutions of underdetermined linear systems, which is often encountered in uncertainty quantification analysis of expensive and high-dimensional physical models. We perform numerical investigations employing several compressive sensing solvers that target the unconstrained LASSO formulation, with a focus on linear systems that arise in the construction of polynomial chaos expansions. With core solvers of l1_ls, SpaRSA, CGIST, FPC_AS, and ADMM, we develop techniques to mitigate overfitting through an automated selection of regularization constant based on cross-validation, and a heuristic strategy to guide the stop-sampling decision. Practical recommendations on parameter settings for these techniques are provided and discussed. The overall method is applied to a series of numerical examples of increasing complexity, including large eddy simulations of supersonic turbulent jet-in-crossflow involving a 24-dimensional input. Through empirical phase-transition diagrams and convergence plots, we illustrate sparse recovery performance under structures induced by polynomial chaos, accuracy and computational tradeoffs between polynomial bases of different degrees, and practicability of conducting compressive sensing for a realistic, high-dimensional physical application. Across test cases studied in this paper, we find ADMM to have demonstrated empirical advantages through consistent lower errors and faster computational times.