Robust Covariance Research Articles

Decreased prevalence of alcohol use and related sociodemographic factors in an urban district of Shanghai, China: Evidence from three cross-sectional surveys (2010–2021)

BackgroundAlcohol use contributes considerably to the global impact of disease. However, limited research has been conducted recently to explore the changes in the prevalence of alcohol use, especially in urban Shanghai, China. ObjectivesTo estimate the prevalence changes of alcohol use and explore related sociodemographic factors among urban residents of Xuhui district, Shanghai, China. MethodsUsing a multistage probability proportionate-to-size sampling method, three waves of household investigations via a structured questionnaire were conducted in Xuhui district, Shanghai, China, between 2010 and 2021. Data were collected by face-to-face interviews among residents aged 15 or older who responded to the 2010, 2015, and 2021 Health Status and Health Service Utilization Survey. Age and sex standardized prevalence of alcohol use was calculated, and the linear trend was tested using logistic regression models. Multilevel mixed-effects logistic regression models with robust covariance matrix estimators were used to estimate related factors of alcohol use. ResultsA total of 24 552 participants aged 15–109 years were included. The standardized overall prevalence of current regular alcohol use statistically declined from 8.8 % in 2010 to 5.5 % in 2021. This decreasing trend was statistically significant, especially among men aged 30–59 and ≥ 75 years old. However, the prevalence of overall alcohol use, regardless of frequency, increased from 19.6 % in 2015 to 24.0 % in 2021. Multivariate regression analysis showed that men had a significantly higher likelihood of regular alcohol use than women. Also, age was positively related to regular alcohol use, while education level was negatively related to this behavior. ConclusionsAlcohol use was still common, although a significant decreasing trend in the prevalence of regular alcohol use was observed overall, in this study population. The findings call for strengthened health education and alcohol control in the context of the harmful effects of alcohol use at any level.

Anomaly detection using unsupervised machine learning algorithms: A simulation study

This study presents a comprehensive evaluation of five prominent unsupervised machine learning anomaly detection algorithms: One-Class Support Vector Machine (One-Class SVM), One-Class SVM with Stochastic Gradient Descent (SGD), Isolation Forest (iForest), Local Outlier Factor (LOF), and Robust Covariance (Elliptic Envelope). Through systematic analysis on a synthetically simulated dataset, the study assessed each algorithm’s predictive performance using accuracy, precision, recall, and F1 score specifically for outlier detection. The evaluation reveals that One-Class SVM, Isolation Forest, and Robust Covariance are more effective in identifying outliers in the synthetic simulated dataset, with Isolation Forest slightly outperforming the other algorithms in terms of balancing precision and recall. One-Class SVM with SGD shows promise in precision but needs adjustment to improve recall. Local Outlier Factor may require parameter tuning or may not be as suitable for this particular dataset’s characteristics. The findings reveal significant variations in performance, highlighting the strengths and limitations of each method in identifying anomalies. This research contributes to the field of machine learning by demonstrating that the selection of an anomaly detection algorithm should be a considered decision, taking into account the specific characteristics of the data and the operational context of its application. Future work should explore parameter optimization, the impact of dataset characteristics on model performance, and the application of these models to real-world datasets to validate their efficacy in practical anomaly detection scenarios.

Robust Estimation of Variance Using Supplementary Information

This study introduces robust exponential ratio-type estimators for finite population variance, incorporating the minimum covariance determinant (M.C.D.) and minimum volume ellipsoid (M.V.E.) robust covariance matrices in the context of simple random sampling. We utilize a first-order approximation to derive expressions for the bias and mean square error (M.S.E.) of these proposed estimators. Through a comprehensive review of existing literature, it is evident that these robust estimators outperform other alternatives in terms of efficiency. Both the M.C.D. and M.V.E. methods are known for their resilience to outliers, making them particularly effective when such anomalies are present in the data. Simulation studies and empirical results consistently demonstrate that the proposed robust exponential ratio-type estimators yield lower mean square errors compared to traditional estimators under simple random sampling. Additionally, the efficiency of these estimators is further validated through simulated studies and real-world data sets. Theoretical analysis and numerical evidence collectively affirm that the proposed class of estimators consistently outperforms competing methods across various scenarios.

Robust Portfolio Mean-Variance Optimization for Capital Allocation in Stock Investment Using the Genetic Algorithm: A Systematic Literature Review

Traditional mean-variance (MV) models, considered effective in stable conditions, often prove inadequate in uncertain market scenarios. Therefore, there is a need for more robust and better portfolio optimization methods to handle the fluctuations and uncertainties in asset returns and covariances. This study aims to perform a Systematic Literature Review (SLR) on robust portfolio mean-variance (RPMV) in stock investment utilizing genetic algorithms (GAs). The SLR covered studies from 1995 to 2024, allowing a thorough analysis of the evolution and effectiveness of robust portfolio optimization methods over time. The method used to conduct the SLR followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. The result of the SLR presented a novel strategy to combine robust optimization methods and a GA in order to enhance RPMV. The uncertainty parameters, cardinality constraints, optimization constraints, risk-aversion parameters, robust covariance estimators, relative and absolute robustness, and parameters adopted were unable to develop portfolios capable of maintaining performance despite market uncertainties. This led to the inclusion of GAs to solve the complex optimization problems associated with RPMV efficiently, as well as fine-tuning parameters to improve solution accuracy. In three papers, the empirical validation of the results was conducted using historical data from different global capital markets such as Hang Seng (Hong Kong), Data Analysis Expressions (DAX) 100 (Germany), the Financial Times Stock Exchange (FTSE) 100 (U.K.), S&P 100 (USA), Nikkei 225 (Japan), and the Indonesia Stock Exchange (IDX), and the results showed that the RPMV model optimized with a GA was more stable and provided higher returns compared with traditional MV models. Furthermore, the proposed method effectively mitigated market uncertainties, making it a valuable tool for investors aiming to optimize portfolios under uncertain conditions. The implications of this study relate to handling uncertainty in asset returns, dynamic portfolio parameters, and the effectiveness of GAs in solving portfolio optimization problems under uncertainty, providing near-optimal solutions with relatively lower computational time.

Which Components of the Haemodynamic Response to Active Stand Predict Cardiovascular Disease and Mortality? Data From The Irish Longitudinal Study on Ageing.

An integrated haemodynamic response during standing may serve as an integrative marker of neuro-cardiovascular function. Individual components of both heart rate (HR) and blood pressure (BP) responses to active stand (AS) have been linked with cardiovascular disease (CVD) and mortality. We assessed longitudinal associations between entire HR/BP response curves during AS, incident CVD and mortality over 12 years. Beat-to-beat measurements of dynamic HR/BP responses to AS were conducted in 4,336 individuals (61.5±8.2 years; 53.7% female). Functional Principal Components Analysis was applied to HR/BP response curves and their association with CVD and mortality assessed. We hypothesised that integrating BP/HR information from the entire haemodynamic response curve may uncover novel associations with both CVD and mortality. Higher systolic BP (SBP) before AS and blunted recovery of SBP during AS was associated with all-cause mortality over 12-years (Hazard Ratio [HR]: 1.14; 1.04, 1.26; p=0.007). Higher baseline/peak HR and lower HR from 30 seconds post stand onwards were associated with lower mortality due to circulatory causes (HR: 0.78; 0.64, 0.95; p = 0.013). Higher HR throughout AS was associated with mortality from other causes (HR: 1.48; 1.22, 1.80; p<0.001). Findings persisted on robust covariate adjustment. We observed distinct relationships between HR/BP responses to AS and 12-year incident CVD and mortality. Integrating the entire haemodynamic response may reveal more nuanced relationships between HR/BP responses to AS, CVD and mortality - serving as an integrative marker of neuro-cardiovascular health in midlife and beyond.

Canonical correlation analysis based on robust covariance matrix by using deterministic of minimum covariance determinant

Canonical correlation analysis (CCA) study the linear combinations between a two multivariate set of variable that have the maximum association among these two sets of variables. The main computation of the CCA is depend on the sample mean and covariance matrices, where they are very sensitive and highly effected by presence of outliers. In this study a new procedure is used on CCA to obtain the robust canonical correlation analysis (RCCA) to control and conquer the deformities of the sample mean and covariance matrices in contaminated data set. The deterministic of minimum covariance determinant (DetMCD) of Mia et al. (1936) is applied on CCA and the superiority performance of RCCA is demonstrated CCA method. A simulation is running for both methods on real and generation data.

Robust speckle covariance matrix estimation of sea clutter based on spectral symmetry

Speckle covariance matrix estimation plays an important role in adaptive target detection of maritime radars. Spatial heterogeneity of sea clutter incurs insufficient secondary data in estimation, which degrades detection performance. It is an effective way of estimation improvement to exploit the Doppler spectrum knowledge of sea clutter for structural dimension reduction of the speckle covariance matrix. This paper proposes a speckle covariance matrix estimation method based on the spectral symmetry knowledge of sea clutter, indicating that the spectrum of sea clutter is symmetric with respect to the Doppler offset. First of all, the spectral symmetry of sea clutter is analyzed by using the measured data from several opened databases and the results show that most of the data excluding sea spikes have symmetric Doppler spectra. Secondly, the spectral symmetry constrains the speckle covariance matrix into the Hadamard product of a special rank-one Toeplitz matrix and a real positive definite symmetric matrix. The degrees of freedom are reduced to almost half of that of a Hermitian matrix and thus fewer secondary data are required. Thirdly, a robust iterative maximum likelihood estimator is proposed to estimate the speckle covariance matrix with structural constraint and is free of the clutter texture distribution. The performance of the estimator is verified by the measured data and the results show that it is superior to traditional estimators, particularly in fewer secondary data.

High dimensional T-type Estimator for robust covariance matrix estimation with applications to elliptical factor models

High dimensional T-type Estimator for robust covariance matrix estimation with applications to elliptical factor models

A new robust covariance matrix estimation for high‐dimensional microbiome data

SummaryMicrobiome data typically lie in a high‐dimensional simplex. One of the key questions in metagenomic analysis is to exploit the covariance structure for this kind of data. In this paper, a framework called approximate‐estimate‐threshold (AET) is developed for the robust basis covariance estimation for high‐dimensional microbiome data. To be specific, we first construct a proxy matrix , which is almost indistinguishable from the real basis covariance matrix . Then, any estimator satisfying some conditions can be used to estimate . Finally, we impose a thresholding step on to obtain the final estimator . In particular, this paper applies a Huber‐type estimator , and achieves robustness by only requiring the boundedness of 2+ moments for some . We derive the convergence rate of under the spectral norm, and provide theoretical guarantees on support recovery. Extensive simulations and a real example are used to illustrate the empirical performance of our method.

Outlier Detection for Permanent Magnet Synchronous Motor (PMSM) Fault Detection and Severity Estimation

Today, Permanent Magnet Synchronous Motors (PMSMs) are a dominant choice in industry applications. During operation, different possible faults in the system can occur, so early and automated fault detection and severity estimation are required to ensure smooth operation and optimal maintenance planning. In this direction, outlier detection methods are employed in this paper. The motor’s current signals are used to extract useful indicators of the fault, along with d-q transform. Statistical indicators in both time and frequency domains are selected to describe fault-related patterns. Based on the extracted features, three outlier detection methods are investigated: the Isolation Forest, the One Class Support Vector Machine, and the Robust Covariance Ellipse. Each method is investigated through different model parameters to evaluate fault detection and severity estimation capabilities. Finally, an ensemble approach is proposed based on decisions and outlier score ensemble. The proposed methodology is verified through different operating conditions in a PMSM test bench.

COMPARISON MRCD AND ORACLE FOR ESTIMATING THE DETERMINANT OF HIGH DIMENSIONAL COVARIANCE MATRIX

Estimating the variance matrix has an important role in statistical applications and conclusions, in high–dimensional matrices if the number of variables is greater than the number of observations P &gt; n, the traditional statistical methods are not reliable because they give uncontrolled estimates. Shrinkage methods are used to estimate the high–dimensional variance matrix. In this research, the high–dimensional variance matrix was estimated using the robust Nonparametric method Minimum Regularized Covariance Determinant (MRCD), which is based on Mahalanobis distance, and compared with the variance matrix estimated by the Oracle method, which is based on the Frobenius criterion

Minimum Regularized Covariance Trace Estimator and Outlier Detection for Functional Data

We propose the Minimum Regularized Covariance Trace (MRCT) estimator, a novel method for robust covariance estimation and functional outlier detection designed primarily for dense functional data. The MRCT estimator employs a subset-based approach that prioritizes subsets exhibiting greater centrality based on the generalization of the Mahalanobis distance, resulting in a fast-MCD type algorithm. Notably, the MRCT estimator handles high-dimensional datasets without the need for preprocessing or dimension reduction techniques, due to the internal smoothening whose amount is determined by the regularization parameter α > 0 . The selection of α is automated. An extensive simulation study demonstrates the efficacy of the MRCT estimator in terms of robust covariance estimation and automated outlier detection, emphasizing the balance between noise exclusion and signal preservation achieved through appropriate selection of α. The method converges fast in practice and performs favorably when compared to other functional outlier detection methods.

Multiple behavioral mechanisms shape development in a highly social cichlid fish

Early-life social experiences shape adult phenotype, yet the underlying behavioral mechanisms remain poorly understood. We manipulated early-life social experience in the highly social African cichlid fish Astatotilapia burtoni to investigate the effects on behavior and stress axis function in juveniles. Juveniles experienced different numbers of social partners in stable pairs (1 partner), stable groups (6 fish; 5 partners), and socialized pairs (a novel fish was exchanged every 5 days; 5 partners). Treatments also differed in group size (groups vs. pairs) and stability (stable vs. socialized). We then measured individual behavior and water-borne cortisol to identify effects of early-life experience. We found treatment differences in behavior across all assays: open field exploration, social cue investigation, dominant behavior, and subordinate behavior. Treatment did not affect cortisol. Principal components (PC) analysis revealed robust co-variation of behavior across contexts, including with cortisol, to form behavioral syndromes sensitive to early-life social experience. PC1 (25.1 %) differed by social partner number: juveniles with more partners (groups and socialized pairs) were more exploratory during the social cue investigation, spent less time in the territory, and were more interactive as dominants. PC5 (8.5 %) differed by stability: socialized pairs were more dominant, spent less time in and around the territory, were more socially investigative, and had lower cortisol than stable groups or pairs. Observations of the home tanks provided insights into the social experiences that may underlie these effects. These results contribute to our understanding of how early-life social experiences are accrued and exert strong, lasting effects on phenotype.

Evaluation of covariate effects in item response theory models.

Item response theory (IRT) models are usually the best way to analyze composite or rating scale data. Standard methods to evaluate covariate or treatment effects in IRT models do not allow to identify item-specific effects. Finding subgroups of patients who respond differently to certain items could be very important when designing inclusion or exclusion criteria for clinical trials, and aid in understanding different treatment responses in varying disease manifestations. We present a new method to investigate item-specific effects in IRT models, which is based on inspection of residuals. The method was investigated in a simulation exercise with a model for the Epworth Sleepiness Scale. We also provide a detailed discussion as a guidance on how to build a robust covariate IRT model.

Impact of manager’s motivation on fraudulent accounting: An empirical study

This study empirically examines the motives of Indian firms’ managers to violate Generally Accepted Accounting Principles (GAAP) and engage in so-called fraudulent accounting. Managers’ motivations for fraudulent accounting rather than profit adjustment are empirically analyzed using data from Indian-listed companies. The sample includes 91 firms and the period of study is March 2001 to March 2022. The tests were conducted using single and multiple variables by the empirical methods used in other studies on profit adjustment. In the case of single variables, the tests are the chi-square test of independence for dummy variables and the significance test of the difference between the mean and median for continuous variables. In the case of multiple variables, the sample firms with a dependent variable of 1 and the control firms with a dependent variable of 0 are analyzed using the logit model. The estimation is done by the robust covariance method. The findings indicate that firms that engage in fraudulent accounting are significantly worse off than other firms in terms of their financial position and operating results, have significantly higher financing needs, and significantly more frequently conduct initial public offerings (IPOs).

Sexual pleasure and HIV-related worry in female sex workers on oral pre-exposure prophylaxis in south-western Uganda.

Female sex workers (FSWs) contribute disproportionately to HIV transmission in Uganda, and pre-exposure prophylaxis (PrEP) is effective in preventing HIV among cisgender women. Psychological factors are important for PrEP uptake, but few studies have examined psychosocial changes due to PrEP use in Uganda. In 2021, we recruited 524 FSWs in three Trans-African Highway towns and four fishing communities in south-western Uganda. We conducted structured interviews among women who were attending routine PrEP follow-up visits in six health units. Bivariable and multivariable modified regression using a robust covariance matrix estimator were used to identify factors associated with experiencing increased sexual pleasure and less worry about HIV because of PrEP. Overall, 80.9% participants reported that sex was more pleasurable because of taking PrEP. There were statistical trends for sex being more pleasurable when taking PrEP or when having condomless sex with casual paying partners (aPR=1.19, 95% CI=1.07-1.32, P=0.001). Almost three-quarters of the participants (76.3%) were less worried about getting HIV because of PrEP. Condomless sex with casual paying partners (aPR=1.17, 95% CI=1.05-1.31, P=0.032, P=0.003) and being On PrEP for the past 1-2years (aPR=1.18, 95% CI=1.00-1.38, P=0.032) was significantly associated with HIV-related worry (aPR=1.17, 95% CI=1.05-1.31, P=0.032, P=0.003)Conclusions:We found a positive impact of PrEP in Ugandan FSWs on two key psychosocial dimensions: (1) more pleasurable sex; and (2) less worry about acquiring HIV. Interventions aiming to increase PrEP uptake may find it useful to focus on psychosocial dimensions.

Spatially Smoothed Robust Covariance Estimation for Local Outlier Detection

Most multivariate outlier detection procedures ignore the spatial dependency of observations, which is present in many real datasets from various application areas. This article introduces a new outlier detection method that accounts for a (continuously) varying covariance structure, depending on the spatial neighborhood of the observations. The underlying estimator thus constitutes a compromise between a unified global covariance estimation, and local covariances estimated for individual neighborhoods. Theoretical properties of the estimator are presented, in particular related to robustness properties, and an efficient algorithm for its computation is introduced. The performance of the method is evaluated and compared based on simulated data and for a dataset recorded from Austrian weather stations. Supplemental materials to the article are available online.

Robust second-order stationary spatial blind source separation using generalized sign matrices

Consider a spatial blind source separation model in which the observed multivariate spatial data are assumed to be a linear mixture of latent stationary spatially uncorrelated random fields. The objective is to recover an unknown mixing procedure as well as the latent random fields. Recently, spatial blind source separation methods that are based on the simultaneous diagonalization of two or more scatter matrices were proposed. In cases involving uncontaminated data, such methods can solve the blind source separation problem, however, in the presence of outlying observations, these methods perform poorly. We propose a robust blind source separation method that employs robust global and local covariance matrices based on generalized spatial signs in simultaneous diagonalization. Simulation studies are employed to illustrate the robustness and efficiency of the proposed methods in various scenarios.

DESI mock challenge: constructing DESI galaxy catalogues based on FastPM simulations

ABSTRACT Together with larger spectroscopic surveys such as the Dark Energy Spectroscopic Instrument (DESI), the precision of large-scale structure studies and thus the constraints on the cosmological parameters are rapidly improving. Therefore, one must build realistic simulations and robust covariance matrices. We build galaxy catalogues by applying a halo occupation distribution (HOD) model upon the FastPM simulations, such that the resulting galaxy clustering reproduces high-resolution N-body simulations. While the resolution and halo finder are different from the reference simulations, we reproduce the reference galaxy two-point clustering measurements – monopole and quadrupole – to a precision required by the DESI Year 1 emission line galaxy sample down to non-linear scales, i.e. $k\lt 0.5\, h\, \mathrm{Mpc}^{-1}$ or $s\gt 10\, \mathrm{Mpc}\, h^{-1}$. Furthermore, we compute covariance matrices based on the resulting FastPM galaxy clustering – monopole and quadrupole. We study for the first time the effect of fitting on Fourier conjugate (e.g. power spectrum) on the covariance matrix of the Fourier counterpart (e.g. correlation function). We estimate the uncertainties of the two parameters of a simple clustering model and observe a maximum variation of 20 per cent for the different covariance matrices. Nevertheless, for most studied scales the scatter is between 2 and 10 per cent. Consequently, using the current pipeline we can precisely reproduce the clustering of N-body simulations and the resulting covariance matrices provide robust uncertainty estimations against HOD fitting scenarios. We expect our methodology will be useful for the coming DESI data analyses and their extension for other studies.

Do ESG ratings and COVID-19 severity score predict stock behavior and market perception? Evidence from emerging markets

PurposeThe purpose of this study is to examine the impact of firm environmental, social and governance (ESG) rating scores on market perception and stock behavior from 2017 to 2021 while controlling for COVID-19 severity score.Design/methodology/approachThe authors used panel regression models with robust standard errors based on cross-country and cross-industry sample of 1,324 ESG firms from 25 emerging countries across four regions. Four separate regression analyses are used. Hausman test is used to determine whether fixed-effect (FE) or random-effect approaches should be used in regression models. Lagrange multiplier test is used to test for time FEs, and F-test for individual effects to choose between pooled ordinary least squares model and FE. Two-unit root tests are conducted to check stationarity. Heteroskedasticity and serial correlation were controlled through a robust covariance matrix estimation.FindingsThe authors provide evidence that the stakeholder theory persists in emerging countries. Overall, the results suggest that firms’ stock behavior is positively associated with the level of environmental and social performance in the region. However, the results do not provide empirical evidence to support the link between ESG performance and stock market perception proxied by the price-to-sales ratio. The results suggest that Refinitiv and Bloomberg ESG rating scores have a positive impact on stock performance in emerging markets, albeit the Bloomberg rating score is insignificant.Practical implicationsFavorable impact of environmental and social performance on stock performance suggests that policymakers should take initiatives to raise awareness toward investments in ESG projects. Evidence shows that ESG stock performance in emerging markets does not insulate firms from the COVID-19 severity. Furthermore, this study highlights the inconsistency in calculating the ESG ratings, therefore, a more standardized approach is recommended to support investors seeking sustainable investments.Social implicationsThe findings have social implications for investors with proenvironmental preferences and nonpecuniary motives for ethical investments. Asset fund managers should develop ESG investment strategies to promote investor preferences that are linked to the proenvironmental and prosocial attitudes by increasing their investments in stocks of firms that behave ethically and support the environment. Furthermore, the findings show that investors pay a price for ethical and socially responsible investments as they are evaluating the environmental and social activities, hence, the firm ESG profile influences equity valuation and risk assessment.Originality/valueThe study extends the literature and provides evidence from the unique setting of emerging markets by analyzing the relationship between ESG rating scores and the COVID-19 severity scores on one hand, and stock behavior and market perception on the other.