Non-regular Model Research Articles

On the Existence of Global Weak Solutions to the 3D Electrically Conductive Rosensweig System and Their Convergence Towards Quasi-Equilibrium

In this article, we study an electrically conductive Rosensweig model for ferrofluids, whose Bloch–Torrey regularization was studied by Hamdache and Hamroun (Appl Math Optim 81(2):479–509, 2020). We mainly prove the global existence of weak solutions to the non-regularized model under a certain smallness condition on the electric conductivity. Hence, our result not only solves a problem that was left open by Hamdache and Hamroun, but it can also serve as a confirmation that ferrofluids are naturally poor conductors of electric current. The proof, which is interesting in itself, is quite involved and relies on the Helmohltz–Leray decomposition of the magnetic fields and the use of renormalized solutions for the magnetization. We also give a rigorous and detailed description of the convergence of the global weak solutions towards the quasi-equilibrium in the relaxation time limit regime τ→0\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ au \\rightarrow 0$$\\end{document}.

The Choice of Noninformative Priors for Thompson Sampling in Multiparameter Bandit Models

Thompson sampling (TS) has been known for its outstanding empirical performance supported by theoretical guarantees across various reward models in the classical stochastic multi-armed bandit problems. Nonetheless, its optimality is often restricted to specific priors due to the common observation that TS is fairly insensitive to the choice of the prior when it comes to asymptotic regret bounds. However, when the model contains multiple parameters, the optimality of TS highly depends on the choice of priors, which casts doubt on the generalizability of previous findings to other models. To address this gap, this study explores the impact of selecting noninformative priors, offering insights into the performance of TS when dealing with new models that lack theoretical understanding. We first extend the regret analysis of TS to the model of uniform distributions with unknown supports, which would be the simplest non-regular model. Our findings reveal that changing noninformative priors can significantly affect the expected regret, aligning with previously known results in other multiparameter bandit models. Although the uniform prior is shown to be optimal, we highlight the inherent limitation of its optimality, which is limited to specific parameterizations and emphasizes the significance of the invariance property of priors. In light of this limitation, we propose a slightly modified TS-based policy, called TS with Truncation (TS-T), which can achieve the asymptotic optimality for the Gaussian models and the uniform models by using the reference prior and the Jeffreys prior that are invariant under one-to-one reparameterizations. This policy provides an alternative approach to achieving optimality by employing fine-tuned truncation, which would be much easier than hunting for optimal priors in practice.

Deciding Asynchronous Hyperproperties for Recursive Programs

We introduce a novel logic for asynchronous hyperproperties with a new mechanism to identify relevant positions on traces. While the new logic is more expressive than a related logic presented recently by Bozzelli et al., we obtain the same complexity of the model checking problem for finite state models. Beyond this, we study the model checking problem of our logic for pushdown models. We argue that the combination of asynchronicity and a non-regular model class studied in this paper constitutes the first suitable approach for hyperproperty model checking against recursive programs.

A Machine Learning Approach to Assess Differential Item Functioning in Psychometric Questionnaires Using the Elastic Net Regularized Ordinal Logistic Regression in Small Sample Size Groups.

Assessing differential item functioning (DIF) using the ordinal logistic regression (OLR) model highly depends on the asymptotic sampling distribution of the maximum likelihood (ML) estimators. The ML estimation method, which is often used to estimate the parameters of the OLR model for DIF detection, may be substantially biased with small samples. This study is aimed at proposing a new application of the elastic net regularized OLR model, as a special type of machine learning method, for assessing DIF between two groups with small samples. Accordingly, a simulation study was conducted to compare the powers and type I error rates of the regularized and nonregularized OLR models in detecting DIF under various conditions including moderate and severe magnitudes of DIF (DIF = 0.4 and 0.8), sample size (N), sample size ratio (R), scale length (I), and weighting parameter (w). The simulation results revealed that for I = 5 and regardless of R, the elastic net regularized OLR model with w = 0.1, as compared with the nonregularized OLR model, increased the power of detecting moderate uniform DIF (DIF = 0.4) approximately 35% and 21% for N = 100 and 150, respectively. Moreover, for I = 10 and severe uniform DIF (DIF = 0.8), the average power of the elastic net regularized OLR model with 0.03 ≤ w ≤ 0.06, as compared with the nonregularized OLR model, increased approximately 29.3% and 11.2% for N = 100 and 150, respectively. In these cases, the type I error rates of the regularized and nonregularized OLR models were below or close to the nominal level of 0.05. In general, this simulation study showed that the elastic net regularized OLR model outperformed the nonregularized OLR model especially in extremely small sample size groups. Furthermore, the present research provided a guideline and some recommendations for researchers who conduct DIF studies with small sample sizes.

Global Structure-Aware Drum Transcription Based on Self-Attention Mechanisms

This paper describes an automatic drum transcription (ADT) method that directly estimates a tatum-level drum score from a music signal in contrast to most conventional ADT methods that estimate the frame-level onset probabilities of drums. To estimate a tatum-level score, we propose a deep transcription model that consists of a frame-level encoder for extracting the latent features from a music signal and a tatum-level decoder for estimating a drum score from the latent features pooled at the tatum level. To capture the global repetitive structure of drum scores, which is difficult to learn with a recurrent neural network (RNN), we introduce a self-attention mechanism with tatum-synchronous positional encoding into the decoder. To mitigate the difficulty of training the self-attention-based model from an insufficient amount of paired data and to improve the musical naturalness of the estimated scores, we propose a regularized training method that uses a global structure-aware masked language (score) model with a self-attention mechanism pretrained from an extensive collection of drum scores. The experimental results showed that the proposed regularized model outperformed the conventional RNN-based model in terms of the tatum-level error rate and the frame-level F-measure, even when only a limited amount of paired data was available so that the non-regularized model underperformed the RNN-based model.

The CEO Problem With rth Power of Difference and Logarithmic Distortions

The CEO problem has received much attention since first introduced by Berger et al., but there are limited results on non-Gaussian models with non-quadratic distortion measures. In this work, we extend the quadratic Gaussian CEO problem to two non-Gaussian settings with general rth power of difference distortion. Assuming an identical observation channel across agents, we study the asymptotics of distortion decay as the number of agents and sum-rate, R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sum</sub> , grow without bound, while individual rates vanish. The first setting is a regular source-observation model with rth power of difference distortion, which subsumes the quadratic Gaussian CEO problem, and we establish that the distortion decays at <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sum</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-r/2</sup> ) when r ≥ 2. We use sample median estimation after the Berger-Tung scheme for achievability. The other setting is a non-regular source-observation model, including uniform additive noise models, with rth power of difference distortion for which estimation-theoretic regularity conditions do not hold. The distortion decay <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sum</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-r</sup> ) when r ≥ 1 is obtained for the non-regular model by midrange estimator following the Berger-Tung scheme. We also provide converses based on the Shannon lower bound for the regular model and the Chazan-Zakai-Ziv bound for the non-regular model, respectively. Lastly, we provide a sufficient condition for the regular model, under which quadratic and logarithmic distortions are asymptotically equivalent by an entropy power relationship as the number of agents grows. This proof relies on the Bernstein-von Mises theorem.

Predictive probability matching priors for a certain non-regular model

Probability matching priors for Bayesian prediction in non-regular case are considered. For one-parameter family of distributions, the resulting priors match the posterior predictive quantile with the frequentist one up to the order of o(n−2), and they are solutions of a certain differential equation (denoted by matching equation). Although predictive probability matching priors depend on a nominal rate α in general, we provide a prior which satisfy the matching equation for every nominal rate α in non-regular location and scale models. A multi-parameter extension including location-scale model is also discussed.

Reference priors via [formula omitted]-divergence for a certain non-regular model in the presence of a nuisance parameter

This paper presents reference priors for non-regular model whose support depends on an unknown parameter. A multi-parameter family which includes both regular and non-regular structures is considered. The resulting priors are obtained by asymptotically maximizing the expected α-divergence between the prior and the corresponding posterior distribution. Some examples of reference priors for typical multi-parameter non-regular distributions such as the location-scale family of distributions and the truncated Weibull distribution are also given.

Change-point detection, segmentation, and related topics

Recent contributions to change-point detection, segmentation and inference for non-regular models are presented. Various problems are considered including the multiple change-point estimation with adaptive penalty for time series with different dependency structures, estimation of the singularity point in cusp-type models, inference for thresholded autoregressive models, and cross-segmentation of matrices.

On optimal designs for nonregular models

Classically, Fisher information is the relevant object in defining optimal experimental designs. However, for models that lack certain regularity, the Fisher information does not exist and, hence, there is no notion of design optimality available in the literature. This article seeks to fill the gap by proposing a so-called Hellinger information, which generalizes Fisher information in the sense that the two measures agree in regular problems, but the former also exists for certain types of non-regular problems. We derive a Hellinger information inequality, showing that Hellinger information defines a lower bound on the local minimax risk of estimators. This provides a connection between features of the underlying model---in particular, the design---and the performance of estimators, motivating the use of this new Hellinger information for non-regular optimal design problems. Hellinger optimal designs are derived for several non-regular regression problems, with numerical results empirically demonstrating the efficiency of these designs compared to alternatives.

Neural Architecture Search by Estimation of Network Structure Distributions

The influence of deep learning is continuously expanding across different domains, and its new applications are ubiquitous. The question of neural network design thus increases in importance, as traditional empirical approaches are reaching their limits. Manual design of network architectures from scratch relies heavily on trial and error, while using existing pretrained models can introduce redundancies or vulnerabilities. Automated neural architecture design is able to overcome these problems, but the most successful algorithms operate on significantly constrained design spaces, assuming the target network to consist of identical repeating blocks. While such approach allows for faster search, it does so at the cost of expressivity. We instead propose an alternative probabilistic representation of a whole neural network structure under the assumption of independence between layer types. Our matrix of probabilities is equivalent to the population of models, but allows for discovery of structural irregularities, while being simple to interpret and analyze. We construct an architecture search algorithm, inspired by the estimation of distribution algorithms, to take advantage of this representation. The probability matrix is tuned towards generating high-performance models by repeatedly sampling the architectures and evaluating the corresponding networks, while gradually increasing the model depth. Our algorithm is shown to discover non-regular models which cannot be expressed via blocks, but are competitive both in accuracy and computational cost, while not utilizing complex dataflows or advanced training techniques, as well as remaining conceptually simple and highly extensible.

Moment matching priors for non-regular models

This paper presents moment matching priors for non-regular models whose supports depend on an unknown parameter. Both one-parameter and multi-parameter cases are considered. The resulting priors are given by matching the posterior mean and bias-adjusted maximum likelihood estimator up to the higher order. Some examples of proposed priors are also given.

On Bayesian Oracle Properties

When model uncertainty is handled by Bayesian model averaging (BMA) or Bayesian model selection (BMS), the posterior distribution possesses a desirable “oracle property” for parametric inference, if for large enough data it is nearly as good as the oracle posterior, obtained by assuming unrealistically that the true model is known and only the true model is used. We study the oracle properties in a very general context of quasi-posterior, which can accommodate non-regular models with cubic root asymptotics and partial identification. Our approach for proving the oracle properties is based on a unified treatment that bounds the posterior probability of model mis-selection. This theoretical framework can be of interest to Bayesian statisticians who would like to theoretically justify their new model selection or model averaging methods in addition to empirical results. Furthermore, for non-regular models, we obtain nontrivial conclusions on the choice of prior penalty on model complexity, the temperature parameter of the quasi-posterior, and the advantage of BMA over BMS.

Monte Carlo Confidence Sets for Identified Sets

In complicated/nonlinear parametric models, it is generally hard to know whether the model parameters are point identified. We provide computationally attractive procedures to construct confidence sets (CSs) for identified sets of full parameters and of subvectors in models defined through a likelihood or a vector of moment equalities or inequalities. These CSs are based on level sets of optimal sample criterion functions (such as likelihood or optimally-weighted or continuously-updated GMM criterions). The level sets are constructed using cutoffs that are computed via Monte Carlo (MC) simulations directly from the quasi-posterior distributions of the criterions. We establish new Bernstein-von Mises (or Bayesian Wilks) type theorems for the quasi-posterior distributions of the quasi-likelihood ratio (QLR) and profile QLR in partially-identified regular models and some non-regular models. These results imply that our MC CSs have exact asymptotic frequentist coverage for identified sets of full parameters and of subvectors in partially-identified regular models, and have valid but potentially conservative coverage in models with reduced-form parameters on the boundary. Our MC CSs for identified sets of subvectors are shown to have exact asymptotic coverage in models with singularities. We also provide results on uniform validity of our CSs over classes of DGPs that include point and partially identified models. We demonstrate good finite-sample coverage properties of our procedures in two simulation experiments. Finally, our procedures are applied to two non-trivial empirical examples: an airline entry game and a model of trade flows.

Monte Carlo Confidence Sets for Identified Sets

In complicated/nonlinear parametric models, it is generally hard to know whether the model parameters are point identified. We provide computationally attractive procedures to construct confidence sets (CSs) for identified sets of full parameters and of subvectors in models defined through a likelihood or a vector of moment equalities or inequalities. These CSs are based on level sets of optimal sample criterion functions (such as likelihood or optimally-weighted or continuously-updated GMM criterions). The level sets are constructed using cutoffs that are computed via Monte Carlo (MC) simulations directly from the quasi-posterior distributions of the criterions. We establish new Bernstein-von Mises (or Bayesian Wilks) type theorems for the quasi-posterior distributions of the quasi-likelihood ratio (QLR) and profile QLR in partially-identified regular models and some non-regular models. These results imply that our MC CSs have exact asymptotic frequentist coverage for identified sets of full parameters and of subvectors in partially-identified regular models, and have valid but potentially conservative coverage in models with reduced-form parameters on the boundary. Our MC CSs for identified sets of subvectors are shown to have exact asymptotic coverage in models with singularities. We also provide results on uniform validity of our CSs over classes of DGPs that include point and partially identified models. We demonstrate good finite-sample coverage properties of our procedures in two simulation experiments. Finally, our procedures are applied to two non-trivial empirical examples: an airline entry game and a model of trade flows.

Bayesian estimation of the half-normal regression model with deterministic frontier

A regression model with deterministic frontier is considered. This type of model has hardly been studied, partly owing to the difficulty in the application of maximum likelihood estimation since this is a non-regular model. As an alternative, the Bayesian methodology is proposed and analysed. Through the Gibbs algorithm, the inference of the parameters of the model and of the individual efficiencies are relatively straightforward. The results of the simulations indicate that the utilized method performs reasonably well.

Analysis of a Regularized Bingham Model with Pressure-Dependent Yield Stress

The goal of this article is to provide some essential results for the solution of a regularized viscoplastic frictional flow model adapted from the extensive mathematical analysis of the Bingham model. The Bingham model is a standard for the description of viscoplastic flows and it is widely used in many application areas. However, wet granular viscoplastic flows necessitate the introduction of additional non-linearities and coupling between velocity and stress fields. This article proposes a step toward a frictional coupling, characterized by a dependence of the yield stress to the pressure field. A regularized version of this viscoplastic frictional model is analysed in the framework of stationary flows. Existence, uniqueness and regularity are investigated, as well as finite-dimensional and algorithmic approximations. It is shown that the model can be solved and approximated as far as a frictional parameter is small enough. Getting similar results for the non-regularized model remains an issue. Numerical investigations are postponed to further works.

On the consistency of the maximum likelihood estimator for the three parameter lognormal distribution

The three parameter log-normal distribution is a popular non-regular model, but surprisingly, whether the local maximum likelihood estimator (MLE) for parameter estimation is consistent or not has been speculated about since the 1960s. This note gives a rigorous proof for the existence of a consistent MLE for the three parameter log-normal distribution, which solves a problem that has been recognized and unsolved for 50 years. Our results also imply a uniform local asymptotic normality condition for the three parameter log-normal distribution. In addition, we give results on the asymptotic normality and the uniqueness of the local MLE.

The Bernstein–von Mises theorem and nonregular models

We study the asymptotic behaviour of the posterior distribution in a broad class of statistical models where the "true" solution occurs on the boundary of the parameter space. We show that in this case Bayesian inference is consistent, and that the posterior distribution has not only Gaussian components as in the case of regular models (the Bernstein-von Mises theorem) but also has Gamma distribution components whose form depends on the behaviour of the prior distribution near the boundary and have a faster rate of convergence. We also demonstrate a remarkable property of Bayesian inference, that for some models, there appears to be no bound on efficiency of estimating the unknown parameter if it is on the boundary of the parameter space. We illustrate the results on a problem from emission tomography.

Thermodynamic Re-optimization of the Binary Pb-Sb System

Hassam et al. recently reported a thermodynamic assessment of the binary Pb-Sb system. Experimental data reveal that the properties of liquid lead-antimony alloy differ slightly from an ideal solution. However, Hassam et al. utilized a non-regular model with six parameters for liquid description, which seems to be not reasonable and can result in bad behavior of the extrapolation of the calculation into areas not covered by the experiments. In this work, a new thermodynamic description of the binary Pb-Sb system was proposed with good agreement between experimental data and predicted results.