Heavy-tailed Research Articles

Stars everywhere: Revealing the prevalence of star performers using empirical data published in entrepreneurship research

Scholars have long called for moving beyond a narrow focus on average performance toward a more direct investigation of the variance in performance. While a few studies have evaluated star entrepreneurs, most empirical research continues to focus on average performers. This lacuna has constrained not only the development of theories but also the accumulation of data on the distribution of performance. In response, this study uses simulations and heuristics to extract distributional information from descriptive statistics commonly reported in published research (i.e., mean, standard deviation, and sample size). Applying this approach to studies recently published in high-impact entrepreneurship journals shows that (a) the suggested methodology can provide rough estimates of the skew and shape of performance distributions, and (b) right-skewed, heavy-tailed distributions featuring star performers are ubiquitous in entrepreneurship, thus reinforcing calls for more direct studies of performance distributions in entrepreneurship.

Multidimensional and Nonlinear Fractional Langevin Equations: Advancing the Theory in of Anomalous Diffusion

This paper presents a comprehensive theoretical analysis of fractional Langevin equations (FLEs) and their applications in modeling anomalous diffusion processes. We extend the classical FLE framework in several significant directions to address the complexities observed in various physical, biological, and social systems. First, we derive a multidimensional extension of the FLE, providing a robust tool for studying anomalous diffusion in higher-dimensional spaces. Second, we incorporate non-Gaussian Lévy α-stable noise into the FLE, enabling the description of systems characterized by heavy-tailed distributions and extreme events. Third, we develop a nonlinear variant of the FLE to model more intricate restoring forces and potential landscapes. Finally, we analyze a generalized FLE with different orders of fractional derivatives for the inertial and friction terms, offering a flexible framework capable of capturing a wider range of anomalous diffusion phenomena. For each extension, we prove the existence and uniqueness of solutions, derive the corresponding probability distributions, and analyze the scaling behavior of the mean squared displacement in the long-time limit. Our results demonstrate how these generalized FLEs can describe various regimes of anomalous diffusion, from subdiffusive to superdiffusive behavior. This work provides a unified theoretical foundation for understanding and modeling complex diffusive processes across diverse disciplines, paving the way for more accurate descriptions of systems exhibiting memory effects, non-Gaussian statistics, and multiscale dynamics.

Heavy-Tail Phenomenon in Decentralized SGD

Recent theoretical studies have shown that heavy-tails can emerge in stochastic optimization due to ‘multiplicative noise’, even under surprisingly simple settings, such as linear regression with Gaussian data. While these studies have uncovered several interesting phenomena, they consider conventional stochastic optimization problems, which exclude decentralized settings that naturally arise in modern machine learning applications. In this paper, we study the emergence of heavy-tails in decentralized stochastic gradient descent (DE-SGD), and investigate the effect of decentralization on the tail behavior. We first show that, when the loss function at each computational node is twice continuously differentiable and strongly convex outside a compact region, the law of the DE-SGD iterates converges to a distribution with polynomially decaying (heavy) tails. To have a more explicit control on the tail exponent, we then consider the case where the loss at each node is a quadratic function, and show that the tail-index can be estimated as a function of the step-size, batch-size, and the topological properties of the network of the computational nodes. Then, we provide theoretical and empirical results showing that DE-SGD has heavier tails than centralized SGD. We also compare DE-SGD to disconnected SGD where nodes distribute the data but do not communicate. Our theory uncovers an interesting interplay between the tails and the network structure: we identify two regimes of parameters (stepsize and network size), where DE-SGD can have lighter or heavier tails than disconnected SGD depending on the regime. Finally, to support our theoretical results, we provide numerical experiments conducted on linear regression and neural networks that suggest the heavier tails is correlated with better generalization in the decentralized setting.

Deep regression learning with optimal loss function

In this paper, we develop a novel efficient and robust nonparametric regression estimator under a framework of a feedforward neural network (FNN). There are several interesting characteristics for the proposed estimator. First, the loss function is built upon an estimated maximum likelihood function, which integrates the information from observed data as well as the information from the data distribution. Consequently, the resulting estimator has desirable optimal properties, such as efficiency. Second, different from the traditional maximum likelihood estimation (MLE), the proposed method avoids the specification of the distribution, making it adaptable to various distributions such as heavy tails and multimodal or heterogeneous distributions. Third, the proposed loss function relies on probabilities rather than direct observations as in least square loss, contributing to the robustness of the proposed estimator. Finally, the proposed loss function involves a nonparametric regression function only. This enables the direct application of the existing packages, simplifying the computational and programming requirements. We establish the large sample property of the proposed estimator in terms of its excess risk and minimax near-optimal rate. The theoretical results demonstrate that the proposed estimator is equivalent to the true MLE where the density function is known in terms of excess risk. Our simulation studies show that the proposed estimator outperforms the existing methods based on prediction accuracy, efficiency and robustness. Particularly, it is comparable to the MLE with the known density and even gets slightly better as the sample size increases. This implies that the adaptive and data-driven loss function from the estimated density may offer an additional avenue for capturing valuable information. We further apply the proposed method to four real data examples, resulting in significantly reduced out-of-sample prediction errors compared to existing methods.

Improved Bald Eagle Search Optimization Algorithm for the Inverse Kinematics of Robotic Manipulators.

The inverse kinematics of robotic manipulators involves determining an appropriate joint configuration to achieve a specified end-effector position. This problem is challenging because the inverse kinematics of manipulators are highly nonlinear and complexly coupled. To address this challenge, the bald eagle search optimization algorithm is introduced. This algorithm combines the advantages of evolutionary and swarm techniques, making it more effective at solving nonlinear problems and improving search efficiency. Due to the tendency of the algorithm to fall into local optima, the Lévy flight strategy is introduced to enhance its performance. This strategy adopts a heavy-tailed distribution to generate long-distance jumps, thereby preventing the algorithm from becoming trapped in local optima and enhancing its global search efficiency. The experiments first evaluated the accuracy and robustness of the proposed algorithm based on the inverse kinematics problem of manipulators, achieving a solution accuracy of up to 10-18 m. Subsequently, the proposed algorithm was compared with other algorithms using the CEC2017 test functions. The results showed that the improved algorithm significantly outperformed the original in accuracy, convergence speed, and stability. Specifically, it achieved over 70% improvement in both standard deviation and mean for several test functions, demonstrating the effectiveness of the Lévy flight strategy in enhancing global search capabilities. Furthermore, the practicality of the proposed algorithm was verified through two real engineering optimization problems.

Different copula types and reliability applications for a new fisk probability model

In this research endeavor, the authors introduce a novel lifetime probability model. This distribution is meticulously examined and characterized, offering insights into its behavior and applicability in various contexts. The proposed new density function of this distribution has various heavy tail forms that are useful in the field of reliability, insurance and statistical modeling. This allows for the representation and modeling of a wide range of data sets that are diverse in their form and nature. The new distribution is characterized by having different patterns of risk or failure rates. The researchers extend the new distribution to the bivariate domain through different methods, including the Morgenstern-Farley-Gumbel distribution, the modified Morgenstern-Farley-Gumbel distribution, the famous Clayton mathematical versions, and the Rennie versions. These extensions enhance the usefulness of the proposed distribution in modeling multivariate age and reliability data and dependencies between variables. The study presents some statistical modeling experiments on reliability data and some important comparisons are presented within the framework of some statistical comparison criteria.

A Pareto tail plot without moment restrictions

We propose a mean functional that exists for arbitrary probability distributions and characterizes the Pareto distribution within the set of distributions with finite left endpoint. This is in sharp contrast to the mean excess plot, which is meaningless for distributions without an existing mean and has non-standard behavior when the mean is finite, but the second moment does not exist. The construction of the plot is based on the principle of a single huge jump, which differentiates between distributions with moderately heavy and super heavy tails. We present an estimator of the tail function based on U-statistics and study its large sample properties. Several loss datasets illustrate the use of the new plot.

Enhanced Diffusion and Non-Gaussian Displacements of Colloids in Quasi-2D Suspensions of Motile Bacteria

In the real world, active agents interact with surrounding passive objects, thus introducing additional degrees of complexity. The relative contributions of far-field hydrodynamic and near-field contact interactions to the anomalous diffusion of passive particles in suspensions of active swimmers remain a subject of ongoing debate. We constructed a quasi-two-dimensional microswimmer–colloid mixed system by taking advantage of Serratia marcescens’ tendency to become trapped at the air–water interface to investigate the origins of the enhanced diffusion and non-Gaussianity of the displacement distributions of passive colloidal tracers. Our findings reveal that the diffusion behavior of colloidal particles exhibits a strong dependence on bacterial density. At moderate densities, the collective dynamics of bacteria dominate the diffusion of tracer particles. In dilute bacterial suspensions, although there are multiple dynamic types present, near-field contact interactions such as collisions play a major role in the enhancement of colloidal transport and the emergence of non-Gaussian displacement distributions characterized by heavy exponential tails in short times. Despite the distinct types of microorganisms and their diverse self-propulsion mechanisms, a generality in the diffusion behavior of passive colloids and their underlying dynamics is observed.

Robust Permutation Tests in Linear Instrumental Variables Regression

This paper develops permutation versions of identification-robust tests in linear instrumental variables regression. Unlike the existing randomization and rank-based tests in which independence between the instruments and the error terms is assumed, the permutation Anderson-Rubin (AR), Lagrange Multiplier (LM) and Conditional Likelihood Ratio (CLR) tests are asymptotically similar and robust to conditional heteroskedasticity under standard exclusion restriction i.e. the orthogonality between the instruments and the error terms. Moreover, when the instruments are independent of the structural error term, the permutation AR tests are exact, hence robust to heavy tails. As such, these tests share the strengths of the rank-based tests and the wild bootstrap AR tests. Numerical illustrations corroborate the theoretical results.

Mises-Fisher similarity-based boosted additive angular margin loss for breast cancer classification

To enhance the accuracy of breast cancer diagnosis, current practices rely on biopsies and microscopic examinations. However, this approach is known for being time-consuming, tedious, and costly. While convolutional neural networks (CNNs) have shown promise for their efficiency and high accuracy, training them effectively becomes challenging in real-world learning scenarios such as class imbalance, small-scale datasets, and label noises. Angular margin-based softmax losses, which concentrate on the angle between features and classifiers embedded in cosine similarity at the classification layer, aim to regulate feature representation learning. Nevertheless, the cosine similarity’s lack of a heavy tail impedes its ability to compactly regulate intra-class feature distribution, limiting generalization performance. Moreover, these losses are constrained to target classes when margin penalties are applied, which may not always optimize effectiveness. Addressing these hurdles, we introduce an innovative approach termed MF-BAM (Mises-Fisher Similarity-based Boosted Additive Angular Margin Loss), which extends beyond traditional cosine similarity and is anchored in the von Mises-Fisher distribution. MF-BAM not only penalizes the angle between deep features and their corresponding target class weights but also considers angles between deep features and weights associated with non-target classes. Through extensive experimentation on the BreaKHis dataset, MF-BAM achieves outstanding accuracies of 99.92%, 99.96%, 100.00%, and 98.05% for magnification levels of ×40, ×100, ×200, and ×400, respectively. Furthermore, additional experiments conducted on the BACH dataset for breast cancer classification, as well as on the LFW and YTF datasets for face recognition, affirm the generalization capability of our proposed loss function.

Multiscale Change Point Detection for Univariate Time Series Data with Missing Value

This paper studies the autoregressive integrated moving average (ARIMA) state space model combined with Kalman smoothing to impute missing values in a univariate time series before detecting change points. We estimate a scale-dependent time-average variance constant that depends on the length of the data section and is robust to mean shifts under serial dependence. The consistency of the proposed estimator is shown under the assumption allowing heavy tailedness. Integrating the proposed estimator with the moving sum and wild binary segmentation procedures to determine the number and locations of change points is discussed. Furthermore, the performance of the proposed methods is evaluated through extensive simulation studies and applied to the Beijing multi-site air quality dataset to impute missing values and detect mean changes in the data.

High Probability Bounds for Stochastic Subgradient Schemes with Heavy Tailed Noise]

High Probability Bounds for Stochastic Subgradient Schemes with Heavy Tailed Noise]

Robust Regression with Covariate Filtering: Heavy Tails and Adversarial Contamination

We study the problem of linear regression where both covariates and responses are potentially (i) heavy-tailed and (ii) adversarially contaminated. Several computationally efficient estimators have been proposed for the simpler setting where the covariates are sub-Gaussian and uncontaminated; however, these estimators may fail when the covariates are either heavy-tailed or contain outliers. In this work, we show how to modify the Huber regression, least trimmed squares, and least absolute deviation estimators to obtain estimators which are simultaneously computationally and statistically efficient in the stronger contamination model. Our approach is quite simple, and consists of applying a filtering algorithm to the covariates, and then applying the classical robust regression estimators to the remaining data. We show that the Huber regression estimator achieves near-optimal error rates in this setting, whereas the least trimmed squares and least absolute deviation estimators can be made to achieve near-optimal error after applying a postprocessing step. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Robust estimation for Threshold Autoregressive Moving-Average models

Threshold autoregressive moving-average (TARMA) models extend the popular TAR model and are among the few parametric time series specifications to include a moving average in a non-linear setting. The state dependent reactions to shocks is particularly appealing in Economics and Finance. However, no theory is currently available when the data present heavy tails or anomalous observations. Here we provide the first theoretical framework for robust M-estimation for TARMA models and study its practical relevance. Under mild conditions, we show that the robust estimator for the threshold parameter is super-consistent, while the estimators for autoregressive and moving-average parameters are strongly consistent and asymptotically normal. The Monte Carlo study shows that the M-estimator is superior, in terms of both bias and variance, to the least squares estimator, which can be heavily affected by outliers. The findings suggest that robust M-estimation should be generally preferred to the least squares method. We apply our methodology to a set of commodity price time series; the robust TARMA fit presents smaller standard errors and superior forecasting accuracy. The results support the hypothesis of a two-regime non-linearity characterised by slow expansions and fast contractions.

Emergence of extreme events from randomly chirped condensate

We explore rogue wave emergence in generic wave systems governed by the nonlinear Schrödinger equation. We highlight the interplay between the stochastic focusing induced by a random chirp of a continuous wave condensate and modulation instability of the condensate to periodic and/or random perturbations. We show that regardless of its statistical properties, the random chirp of the condensate leads to a non-Gaussian, heavy-tailed probability density distribution of peak powers of the waves excited atop of the condensate, which is a statistical signature of rogue waves, or more broadly, extreme events in physics.

Regularisation and central limit theorems for an inverse problem in network sampling applications

ABSTRACT An inverse problem motivated by packet sampling in communication networks and edge sampling in directed complex networks is studied through the operator perspective. The problem is shown to be ill-posed, with the resulting naive estimator potentially having very heavy tails, satisfying non-Gaussian central limit theorem and showing poor statistical performance. Regularisation of the problem leads to the Gaussian central limit theorem and superior performance of the regularised estimator, as a result of desirable properties of underlying operators. The limiting variance and convergence rates of the regularised estimator are also investigated. The results are illustrated on synthetic and real data from communication and complex networks.

A data-driven maximum entropy method for probability uncertainty analysis based on the B-spline theory

The probability density function (PDF) is quite important for structural reliability analysis; thus, accurate PDF modeling methods draw increasing attention. This paper proposes a novel metaheuristic data-driven paradigm of the maximum entropy method (MEM) based on the B-spline function theory. Firstly, a B-spline proxy of the MEM PDF is proposed for probability uncertainty analysis. We derive the parameter calculation formulation and calculate the undetermined parameters via the raw data of structural responses. Then, to determine the knots of the B-spline functions, we propose a novel data-driven approach with the aid of a powerful metaheuristic algorithm and the response data information. Different from the traditional MEM, the proposed method is a complete data-driven solution approach and does not involve the statistical moment calculation and the nonlinear equations composed of statistical moments. Combining the advantages of the B-spline theory and the MEM, the proposed method can reconstruct the response PDF with a complex shape, such as the PDF with multiple peaks or heavy tails. For verification, two numerical examples and one engineering example are analyzed, and compared with some classical PDF modeling methods. The results show that the proposed method is superior to the compared methods in terms of computational accuracy, when the same sample data is used.

Deep-water ambient sound over the Atlantis II seamounts in the Northwest Atlantica).

Ambient sound was continuously recorded for 52 days by three synchronized, single-hydrophone, near-bottom receivers. The receivers were moored at depths of 2573, 2994, and 4443 m on flanks and in a trough between the edifices of the Atlantis II seamounts. The data reveal the power spectra and intermittency of the ambient sound intensity in a 13-octave frequency band from 0.5 to 4000 Hz. Statistical distribution of sound intensity exhibits much heavier tails than in the expected exponential intensity distribution throughout the frequency band of observations. It is established with high statistical significance that the data are incompatible with the common assumption of normally distributed ambient noise in deep water. Spatial variability of the observed ambient sound appears to be controlled by the seafloor properties, bathymetric shadowing, and nonuniform distribution of the noise sources on the sea surface. Temporal variability of ambient sound is dominated by changes in the wind speed and the position of the Gulf Stream relative to the experiment site. Ambient sound intensity increases by 4-10 dB when the Gulf Stream axis is within 25 km from the receivers. The sound intensification is attributed to the effect of the Gulf Stream current on surface wave breaking.

Epidemics on critical random graphs with heavy-tailed degree distribution

We study the susceptible-infected-recovered (SIR) epidemic on a random graph chosen uniformly over all graphs with certain critical, heavy-tailed degree distributions. We prove process level scaling limits for the number of individuals infected on day h on the largest connected components of the graph. The scaling limits contain non-negative jumps corresponding to some vertices of large degree. These weak convergence techniques allow us to describe the height profile of the α-stable continuum random graph (Goldschmidt et al., 2022; Conchon–Kerjan and Goldschmidt, 2023), extending results known in the Brownian case (Miermont and Sen, 2022). We also prove model-independent results that can be used on other critical random graph models.

Information FOMO: The Unhealthy Fear of Missing Out on Information-A Method for Removing Misleading Data for Healthier Models.

Misleading or unnecessary data can have out-sized impacts on the health or accuracy of Machine Learning (ML) models. We present a Bayesian sequential selection method, akin to Bayesian experimental design, that identifies critically important information within a dataset while ignoring data that are either misleading or bring unnecessary complexity to the surrogate model of choice. Our method improves sample-wise error convergence and eliminates instances where more data lead to worse performance and instabilities of the surrogate model, often termed sample-wise "double descent". We find these instabilities are a result of the complexity of the underlying map and are linked to extreme events and heavy tails. Our approach has two key features. First, the selection algorithm dynamically couples the chosen model and data. Data is chosen based on its merits towards improving the selected model, rather than being compared strictly against other data. Second, a natural convergence of the method removes the need for dividing the data into training, testing, and validation sets. Instead, the selection metric inherently assesses testing and validation error through global statistics of the model. This ensures that key information is never wasted in testing or validation. The method is applied using both Gaussian process regression and deep neural network surrogate models.