Regularization Coefficient Research Articles

Adjoint based state estimation of compressible flow in porous media

Abstract In this paper, we study the state estimation of compressible single phase flow in compressible porous media. The initial pressure distribution is estimated according to discrete adjoint approach based on the collected well pressure data. The first-order Tykhonov regularization method is used to obtain reasonable estimation. By analyzing the optimality condition of estimation problem, the discrete adjoint state equation and discrete adjoint gradient are derived based on the numerical scheme of the continuous equations. A quasi-Newton numerical optimization method related to adjoint gradient is proposed to solve the estimation problem. The estimation results with different regularization coefficients are compared and analyzed by numerical experiments. The deviation between the estimated pressure obtained without regularization and the real pressure is large. Estimation result with smaller deviation and higher smoothness can be obtained through appropriate regularization coefficient. When the observation error is large, the observed values generated by the estimated pressure fit well with the real pressure.

Analyzing the Influence of Hyper-parameters and Regularizers of Topic Modeling in Terms of Renyi Entropy.

Topic modeling is a popular technique for clustering large collections of text documents. A variety of different types of regularization is implemented in topic modeling. In this paper, we propose a novel approach for analyzing the influence of different regularization types on results of topic modeling. Based on Renyi entropy, this approach is inspired by the concepts from statistical physics, where an inferred topical structure of a collection can be considered an information statistical system residing in a non-equilibrium state. By testing our approach on four models—Probabilistic Latent Semantic Analysis (pLSA), Additive Regularization of Topic Models (BigARTM), Latent Dirichlet Allocation (LDA) with Gibbs sampling, LDA with variational inference (VLDA)—we, first of all, show that the minimum of Renyi entropy coincides with the “true” number of topics, as determined in two labelled collections. Simultaneously, we find that Hierarchical Dirichlet Process (HDP) model as a well-known approach for topic number optimization fails to detect such optimum. Next, we demonstrate that large values of the regularization coefficient in BigARTM significantly shift the minimum of entropy from the topic number optimum, which effect is not observed for hyper-parameters in LDA with Gibbs sampling. We conclude that regularization may introduce unpredictable distortions into topic models that need further research.

Measurement of Thermal expansion coefficient of solid Materials by double slit Interferometric method

Thermal expansion coefficient is a regular coefficient to describe the variation of geometric characteristics of matter under the action of thermal expansion and cold shrinkage effect, and it is an important index to measure the thermal stability of materials. A device for measuring the thermal expansion coefficient of solid materials by Yang’s double slit interferometric method with variable double slit spacing is mainly composed of laser, sliding Young double slit structure, temperature measuring and controlling instrument, shading tube, receiving device and solid material to be measured. The solid material is heated by ring heater, the temperature of solid material is displayed and controlled by temperature measuring and controlling instrument, and the change of stripe spacing is recorded by linear CCD image sensor. Compared with the traditional optical leverage method, this measuring device and method can significantly improve the accuracy of the measurement results and reduce the uncertainty of the measurement results.

Nonuniform mean-square exponential dichotomies and mean-square exponential stability

In this paper, the existence conditions of nonuniform mean-square exponential dichotomy (NMS-ED) for a linear stochastic differential equation (SDE) are established. The difference of the conditions for the existence of a nonuniform dichotomy between an SDE and an ordinary differential equation (ODE) is that the first one needs an additional assumption, nonuniform Lyapunov matrix, to guarantee that the linear SDE can be transformed into a decoupled one, while the second does not. Therefore, the first main novelty of our work is that we establish some preliminary results to tackle the stochasticity. This paper is also concerned with the mean-square exponential stability of nonlinear perturbation of a linear SDE under the condition of nonuniform mean-square exponential contraction (NMS-EC). For this purpose, the concept of second-moment regularity coefficient is introduced. This concept is essential in determining the stability of the perturbed equation, and hence we deduce the lower and upper bounds of this coefficient. Our results imply that the lower and upper bounds of the second-moment regularity coefficient can be expressed solely by the drift term of the linear SDE.

Maximum Correntropy Criterion Based l1-Iterative Wiener Filter for Sparse Channel Estimation Robust to Impulsive Noise

In this paper, we propose a new sparse channel estimator robust to impulsive noise environments. For this kind of estimator, the convex regularized recursive maximum correntropy (CR-RMC) algorithm has been proposed. However, this method requires information about the true sparse channel to find the regularization coefficient for the convex regularization penalty term. In addition, the CR-RMC has a numerical instability in the finite-precision cases that is linked to the inversion of the auto-covariance matrix. We propose a new method for sparse channel estimation robust to impulsive noise environments using an iterative Wiener filter. The proposed algorithm does not need information about the true sparse channel to obtain the regularization coefficient for the convex regularization penalty term. It is also numerically more robust, because it does not require the inverse of the auto-covariance matrix.

Wheel/Rail Adhesion State Identification of Heavy-Haul Locomotive Based on Particle Swarm Optimization and Kernel Extreme Learning Machine

The traction performance of heavy-haul locomotive is subject to the wheel/rail adhesion states. However, it is difficult to obtain these states due to complex adhesion mechanism and changeable operation environment. According to the influence of wheel/rail adhesion utilization on locomotive control action, the wheel/rail adhesion states are divided into four types, namely normal adhesion, fault indication, minor fault, and serious fault in this work. A wheel/rail adhesion state identification method based on particle swarm optimization (PSO) and kernel extreme learning machine (KELM) is proposed. To this end, a wheel/rail state identification model is constructed using KELM, and then the regularization coefficient and kernel parameter of KELM are optimized by using PSO to improve its accuracy. Finally, based on the actual data, the proposed method is compared with PSO support vector machines (PSO-SVM) and basic KELM, respectively, and the results are given to verify the effectiveness and feasibility of the proposed method.

Error Estimates of Penalty Schemes for Quasi-Variational Inequalities Arising from Impulse Control Problems

This paper proposes penalty schemes for a class of weakly coupled systems of Hamilton--Jacobi--Bellman quasi-variational inequalities (HJBQVIs) arising from stochastic hybrid control problems of regime-switching models with both continuous and impulse controls. We show that the solutions of the penalized equations converge monotonically to those of the HJBQVIs. We further establish that the scheme is half-order accurate for HJBQVIs with Lipschitz coefficients and first-order accurate for equations with more regular coefficients. Moreover, we construct the action regions and optimal impulse controls based on the error estimates and the penalized solutions. The penalty schemes and convergence results are then extended to HJBQVIs with possibly negative impulse costs. We also demonstrate the convergence of monotone discretizations of the penalized equations and establish that policy iteration applied to the discrete equation is monotonically convergent with an arbitrary initial guess in an infinite-dimensional setting. Numerical examples for infinite-horizon optimal switching problems are presented to illustrate the effectiveness of the penalty schemes over the conventional direct control scheme.

A Sparse Conjugate Gradient Adaptive Filter.

In this letter, we propose a novel conjugate gradient (CG) adaptive filtering algorithm for online estimation of system responses that admit sparsity. Specifically, the Sparsity-promoting Conjugate Gradient (SCG) algorithm is developed based on iterative reweighting methods popular in the sparse signal recovery area. We propose an affine scaling transformation strategy within the reweighting framework, leading to an algorithm that allows the usage of a zero sparsity regularization coefficient. This enables SCG to leverage the sparsity of the system response if it already exists, while not compromising the optimization process. Simulation results show that SCG demonstrates improved convergence and steady-state properties over existing methods.

Learning Sparse Low-Precision Neural Networks With Learnable Regularization

We consider learning deep neural networks (DNNs) that consist of low-precision weights and activations for efficient inference of fixed-point operations. In training low-precision networks, gradient descent in the backward pass is performed with high-precision weights while quantized low-precision weights and activations are used in the forward pass to calculate the loss function for training. Thus, the gradient descent becomes suboptimal, and accuracy loss follows. In order to reduce the mismatch in the forward and backward passes, we utilize mean squared quantization error (MSQE) regularization. In particular, we propose using a learnable regularization coefficient with the MSQE regularizer to reinforce the convergence of high-precision weights to their quantized values. We also investigate how partial L2 regularization can be employed for weight pruning in a similar manner. Finally, combining weight pruning, quantization, and entropy coding, we establish a low-precision DNN compression pipeline. In our experiments, the proposed method yields low-precision MobileNet and ShuffleNet models on ImageNet classification with the state-of-the-art compression ratios of 7.13 and 6.79, respectively. Moreover, we examine our method for image super resolution networks to produce 8-bit low-precision models at negligible performance loss.

Wind Power Prediction of Kernel Extreme Learning Machine Based on Differential Evolution Algorithm and Cross Validation Algorithm

As fossil fuel is being depleted, the percentage of wind power capacity in total electricity generation is increasing. In order to improve the absorption capacity of wind power, wind power prediction has been introduced. Aiming at the disadvantage of low prediction accuracy and unstable model of traditional extreme learning machine (ELM), a kernel extreme learning machine based on differential evolution (DE) and cross validation optimization method is proposed to predict short-term wind power generation. Firstly, the average mean square error (MSE) verified by k folding and cross validation is adopted as the error function of the model to improve the stability and generalization performance of the model. Secondly, differential evolution algorithm is used to optimize the regularization coefficient and kernel width of the kernel extreme learning machine with cross validation and improve the precision of model is 8.34%. Finally, compared with the application of extreme learning machine with genetic algorithm and cross validation to a wind farm prediction case in northwest China, the experimental results show that the convergence rate of this method is twice that of genetic algorithm (GA) optimization algorithm, and the accuracy is higher.

On the intersection motive of certain Shimura varieties: the case of Siegel threefolds

In this article, we construct a Hecke-equivariant Chow motive whose realizations equal intersection cohomology of Siegel threefolds with regular algebraic coefficients. As a consequence, we are able to define Grothendieck motives for Siegel modular forms.

Aim4res, an open-source 2.5D finite differences MATLAB library for anisotropic electrical resistivity modeling

Electrical Resistivity Tomography (ERT) is one of the oldest geophysical techniques, and due to the advances of numerical techniques along with computational resources, it is widely used for geophysical prospecting. It has found various domains of application as it is easy to implement and fast to image the ground resistivity heterogeneity. However, anisotropy, which is another key resistivity parameter, is seldom considered. Although being a well-known phenomenon, its consideration in the characterization process is only recent. Amongst the reasons behind this is the absence of available anisotropic resistivity modeling tools. We present aim4res (Anisotropic Inverse Modeling for RESisitivity) to that end. This open-source MATLAB library allows for 2.5D forward and inverse anisotropic resistivity modeling based on a finite differences scheme. The inverse problem is solved with a Gauss–Newton algorithm. The regularization coefficient, initial model and constraints can be adjusted from prior knowledge in order to avoid local minima during optimization. Analytical and synthetic studies have been carried out to prove the reliability of aim4res. The results demonstrate its ability to identify anisotropy, along with the correct geometry and resistivity amplitude. It is also able to correctly detect isotropy, as the inversion comparison with a previous toolbox already proven working showed. A real case study inversion is carried out to demonstrate that aim4res is a relevant tool to use on the field, able to reveal strong anisotropy fields even at short scales.

SSN: Learning Sparse Switchable Normalization via SparsestMax

Normalization method deals with parameters training of convolution neural networks (CNNs) in which there are often multiple convolution layers. Despite the fact that layers in CNN are not homogeneous in the role they play at representing a prediction function, existing works often employ identical normalizer in different layers, making performance away from idealism. To tackle this problem and further boost performance, a recently-proposed switchable normalization (SN) provides a new perspective for deep learning: it learns to select different normalizers for different convolution layers of a ConvNet. However, SN uses softmax function to learn importance ratios to combine normalizers, not only leading to redundant computations compared to a single normalizer but also making model less interpretable. This work addresses this issue by presenting sparse switchable normalization (SSN) where the importance ratios are constrained to be sparse. Unlike $$\ell _1$$ and $$\ell _0$$ regularizations that impose difficulties in tuning layer-wise regularization coefficients, we turn this sparse-constrained optimization problem into feed-forward computation by proposing SparsestMax, which is a sparse version of softmax. SSN has several appealing properties. (1) It inherits all benefits from SN such as applicability in various tasks and robustness to a wide range of batch sizes. (2) It is guaranteed to select only one normalizer for each normalization layer, avoiding redundant computations and improving interpretability of normalizer selection. (3) SSN can be transferred to various tasks in an end-to-end manner. Extensive experiments show that SSN outperforms its counterparts on various challenging benchmarks such as ImageNet, COCO, Cityscapes, ADE20K, Kinetics and MegaFace. Models and code are available at https://github.com/switchablenorms/Sparse_SwitchNorm .

Analysis of Speaker Diarization Based on Bayesian HMM With Eigenvoice Priors

In our previous work, we introduced our Bayesian Hidden Markov Model with eigenvoice priors, which has been recently recognized as the state-of-the-art model for Speaker Diarization. In this article we present a more complete analysis of the Diarization system. The inference of the model is fully described and derivations of all update formulas are provided for a complete understanding of the algorithm. An extensive analysis on the effect, sensitivity and interactions of all model parameters is provided, which might be used as a guide for their optimal setting. The newly introduced speaker regularization coefficient allows us to control the number of speakers inferred in an utterance. A naive speaker model merging strategy is also presented, which allows to drive the variational inference out of local optima. Experiments for the different diarization scenarios are presented on CALLHOME and DIHARD datasets.

Weak solutions to Vlasov–McKean equations under Lyapunov-type conditions

We present a Lyapunov-type approach to the problem of existence and uniqueness of general law-dependent stochastic differential equations. In the existing literature, most results concerning existence and uniqueness are obtained under regularity assumptions of the coefficients with respect to the Wasserstein distance. Some existence and uniqueness results for irregular coefficients have been obtained by considering the total variation distance. Here, we extend this approach to the control of the solution in some weighted total variation distance, that allows us now to derive a rather general weak uniqueness result, merely assuming measurability and certain integrability on the drift coefficient and some non-degeneracy on the dispersion coefficient. We also present an abstract weak existence result for the solution of law-dependent stochastic differential equations with merely measurable coefficients, based on an approximation with law-dependent stochastic differential equations with regular coefficients under Lyapunov-type assumptions.

An Approach to Hyperparameter Optimization for the Objective Function in Machine Learning

In machine learning, performance is of great value. However, each learning process requires much time and effort in setting each parameter. The critical problem in machine learning is determining the hyperparameters, such as the learning rate, mini-batch size, and regularization coefficient. In particular, we focus on the learning rate, which is directly related to learning efficiency and performance. Bayesian optimization using a Gaussian Process is common for this purpose. In this paper, based on Bayesian optimization, we attempt to optimize the hyperparameters automatically by utilizing a Gamma distribution, instead of a Gaussian distribution, to improve the training performance of predicting image discrimination. As a result, our proposed method proves to be more reasonable and efficient in the estimation of learning rate when training the data, and can be useful in machine learning.

Proportionate Adaptive Filters Based on Minimizing Diversity Measures for Promoting Sparsity.

In this paper, a novel way of deriving proportionate adaptive filters is proposed based on diversity measure minimization using the iterative reweighting techniques well-known in the sparse signal recovery (SSR) area. The resulting least mean square (LMS)-type and normalized LMS (NLMS)-type sparse adaptive filtering algorithms can incorporate various diversity measures that have proved effective in SSR. Furthermore, by setting the regularization coefficient of the diversity measure term to zero in the resulting algorithms, Sparsity promoting LMS (SLMS) and Sparsity promoting NLMS (SNLMS) are introduced, which exploit but do not strictly enforce the sparsity of the system response if it already exists. Moreover, unlike most existing proportionate algorithms that design the step-size control factors based on heuristics, our SSR-based framework leads to designing the factors in a more systematic way. Simulation results are presented to demonstrate the convergence behavior of the derived algorithms for systems with different sparsity levels.

Fr\xe9chet differentiability of mild solutions to SPDEs with respect to the initial datum

We establish n-th-order Fréchet differentiability with respect to the initial datum of mild solutions to a class of jump diffusions in Hilbert spaces. In particular, the coefficients are Lipschitz-continuous, but their derivatives of order higher than one can grow polynomially, and the (multiplicative) noise sources are a cylindrical Wiener process and a quasi-left-continuous integer-valued random measure. As preliminary steps, we prove well-posedness in the mild sense for this class of equations, as well as first-order Gâteaux differentiability of their solutions with respect to the initial datum, extending previous results by Marinelli, Prévôt, and Röckner in several ways. The differentiability results obtained here are a fundamental step to construct classical solutions to non-local Kolmogorov equations with sufficiently regular coefficients by probabilistic means.

Variable coefficient depletion equations – Efficient calculation method

Neutron flux field changes in terms of intensity and spectrum from time to time in nuclear reactors. Considering this effect more faithfully, high order neutronic-depletion coupling strategies were shown to be advantageous over traditional ones. While the variable coefficient depletion equations rather than the constant ones are meant to be solved. Through exploiting characteristics of depletion problems, a two-stage method that combines Chebyshev Rational Approximation Method (CRAM) and Gauss-Legendre Method is proposed, and its implementation details are tuned to overcome the method discontinuity problem. At the same time, the regular coefficient averaging sub-step division scheme is optimized by taking advantage of the fading behavior of sub-step errors. Test problems with varying properties are formulated and calculated. Numerical results showed that the sub-step division optimization could improve calculation efficiency for about one order of magnitude, meanwhile the two-stage method offers additionally 2–10 times of efficiency gain.

Local privacy protection classification based on human-centric computing

Human-centric computing is becoming an important part of data-driven artificial intelligence (AI) and the importance of data mining under Human-centric computing is getting more and more attention. The rapid development of machine learning has gradually increased its ability to mine data. In this paper, privacy protection is combined with machine learning, in which a logistic regression is adopted for local differential privacy protection to achieves classification task utilizing noise addition and feature selection. The design idea is mainly divided into three parts: noise addition, feature selection and logistic regression. In the part of noise addition, the way of adding noise using Laplace mechanism to original data achieves the purpose of disturbing data. The part of feature selection is to highlight the impact of noised data on the classifier. The part of logistic regression is to use logistic regression to implement classification task. The experimental results show that an accuracy of 85.7% can be achieved for the privacy data by choosing appropriate regularization coefficients.