Assumption Of Independence Research Articles

The relationship between smoking and recurrent aphthous stomatitis: A Mendelian randomization study.

Existing research suggests an association between smoking and the incidence of recurrent aphthous stomatitis (RAS); however, the causal relationship remains ambiguous. We employed Mendelian randomization (MR) to clarify the potential causal association between smoking and the risk of developing RAS. We utilized genome-wide association study (GWAS) sequencing data related to smoking from the Finnish database as instrumental variables (IVs) and GWAS data for RAS from the UK Biobank (UKB) as the outcome to perform a two-sample MR analysis. The selection of IVs was rigorously controlled according to the three principal assumptions of relevance, independence, and exclusivity. The primary analytical methods utilized were inverse variance weighting (IVW) and weighted median (WM), supplemented by MR-Egger, simple mode, and weighted mode techniques to infer causality between smoking and RAS. Sensitivity analyses were conducted using MR-PRESSO, Cochran's Q, and the MR-Egger intercept to ensure the robustness of the findings. The findings from the IVW and WM analyses suggest a causal association between smoking and an elevated risk of RAS (IVW: OR=1.003; 95% CI: 1.0002-1.005, p=0.033; WM: OR=1.003; 95% CI: 1.00006-1.007, p=0.044). Compared to non-smokers, smokers have a 0.3% increase in the risk of RAS. Furthermore, the sensitivity analysis did not reveal any inconsistencies that would contradict the MR results. Our findings provide preliminary evidence of a potential causal relationship between smoking and the risk of RAS, which may contribute to a deeper understanding of the underlying mechanisms. Further research is needed to confirm these results and explore their implications for clinical practice.

The impact of co-housing on murine aging studies.

Analysis of preclinical lifespan studies often assume that outcome data from co-housed animals are independent. In practice, treatments, such as controlled feeding or putative life-extending compounds, are applied to whole housing units, and as a result, the outcomes are potentially correlated within housing units. We consider intra-class (here, intra-cage) correlation in three published and two unpublished lifespan studies of aged mice encompassing more than 20,000 observations. We show that the independence assumption underlying common analytic techniques does not hold in these data, particularly for traits associated with frailty. We describe and demonstrate various analytical tools available to accommodate this study design and highlight a limitation of standard variance components models (i.e., linear mixed models) which are the usual statistical tools for handling correlated errors. Through simulations, we examine the statistical biases resulting from intra-cage correlations with similar magnitudes as observed in these case studies and discuss implications for power and reproducibility.

Implementing A Naïve Bayes Classifier on Iris Data Using MATLAB, A Classification Method by Using Grid Parameters Optimization

Data mining categorization is crucial for predicting outcomes. Naive Bayes Classification (NBC) is a prominent approach used in data mining for classification purposes. It has the ability to anticipate outcomes and is generally more efficient than other techniques of categorization. Several Naive Bayes classification methods exhibit subpar performance when used to classification and regression issues. One of the factors contributing to the success of NBC is the reliance on assumptions of independence among predictors and the initial hyperparameters. Nevertheless, this robust assumption results in a decrease in precision. This work introduces a novel approach to enhance the precision of NBC. The suggested approach employs a grid search to enhance the accuracy of Naïve Bayes classification. The results obtained indicate that the technique utilised achieved a significant degree of accuracy in forecasting compared to the conventional built-in Matlab NBC. Thus, by assuming conditional independence, the accuracy of the NBC may be enhanced.

A sparse latent class model incorporating response times.

Diagnostic models (DM) have been widely used to classify respondents' latent attributes in cognitive and non-cognitive assessments. The integration of response times (RTs) with DM presents additional evidence to understand respondents' problem-solving behaviours. While recent research has explored using sparse latent class models (SLCM) to infer the latent structure of items based on item responses, the incorporation of RT data within these models remains underexplored. This study extends the SLCM framework to include RT, relaxing the conditional independence assumption between RT and latent attributes given individual speed. This adaptation provides a more flexible framework for jointly modelling RT and item responses. While the proposed model holds promise for applications in educational assessment, this study applied the model to the Fisher Temperament Inventory, yielding findings that provide a novel perspective on utilizing DM with RT in personality assessments. Additionally, a Gibbs sampling algorithm is proposed for parameter estimation. Results from Monte Carlo simulations demonstrate the algorithm's accuracy and efficiency.

How do immune cells shape type 1 diabetes? Insights from Mendelian randomization.

The role of immune cells in type 1 diabetes (T1D) is unclear. The aim of this study was to assess the causal effect of different immune cells on T1D using Mendelian randomization (MR). A dataset of immune cell phenotypes (numbered from GCST0001391 to GCST0002121) was obtained from the European Bioinformatics Institute, while a T1D dataset (numbered finngen_R10_T1D) was obtained from FinnGen. Single nucleotide polymorphisms meeting the conditions were screened stepwise according to the assumptions of association, independence, and exclusivity. Inverse variance weighted was used as the main method for the MR analysis. MR-Egger was used to assess the horizontal pleiotropy of the results. Cochran's Q and the leave-one-out method were respectively used for the heterogeneity analysis and the sensitivity analysis of the results. MR analysis showed that effector memory (EM) double-negative (DN) (CD4-CD8-) %T cells [odds ratio (OR) = 1.157, 95% confidence interval (95% CI) = 1.016-1.318, p = 0.028, false discovery rate (FDR) = 0.899], EM CD8br %T cells (OR = 1.049, 95% CI = 1.003-1.098, p = 0.037, FDR = 0.902), CD28 on CD28+CD45RA+CD8br (OR = 1.334, 95% CI = 1.132-1.571, p = 0.001, FDR = 0.044), IgD+CD38dim %lymphocytes (OR = 1.045, 95% CI = 1.002-1.089, p = 0.039, FDR = 0.902), CD80 on monocytes (OR = 1.084, 95% CI = 1.013-1.161, p = 0.020, FDR = 0.834), SSC-A on plasmacytoid dendritic cells (pDCs) (OR = 1.174, 95% CI = 1.004-1.372, p = 0.044, FDR = 0.902), and FSC-A on pDCs (OR = 1.182, 95% CI = 1.011-1.382, p = 0.036, FDR = 0.902) were associated with an increased genetic susceptibility to T1D. Cochran's Q showed that there was heterogeneity for CD28 on the CD28+CD45RA+CD8br results (p = 0.043), whereas there was no heterogeneity for the other results (p ≥ 0.05). The sensitivity analysis showed that the MR analysis results were robust. The MR analysis demonstrated that seven immune cell phenotypes were associated with an increased genetic susceptibility to T1D. These findings provide a new direction for the pathogenesis of and the drug development for T1D.

PSYCHOMETRIC ANALYSIS OF MATHEMATICS READINESS ASSESSMENT TOOL FOR ELEMENTARY SCHOOL STUDENTS: A RASCH MODEL APPROACH

This study examines the psychometric properties of a Mathematics Readiness Assessment Tool for elementary school students using Rasch analysis through the Winstep program. The analysis involved 214 students responding to 15 items. Results show strong psychometric characteristics with an item reliability of 0.95 and person reliability of 0.69. The instrument demonstrates good construct validity with INFIT and OUTFIT MNSQ values falling within the acceptable range (0.5-1.5). Analysis of unidimensionality showed satisfactory results with raw variance explained by measures at 26.8% and unexplained variance in the first contrast at 7.6%. Local independence assumption was met with residual correlations ranging from -0.22 to 0.20. Item difficulty ranges from -1.33 to 1.56 logits, indicating a good spread of item difficulties. The test information function shows optimal measurement precision for students of average ability levels, though less precise for extremely high or low abilities. Differential Item Functioning (DIF) analysis revealed some items requiring revision, particularly items 13-15. The instrument's Alpha Cronbach value of 0.73 indicates good internal consistency. Overall, the assessment tool demonstrates adequate psychometric properties for measuring mathematics readiness in elementary school students, though some improvements are recommended for optimal measurement across all ability levels.

Directional predictability tests

. This article proposes new tests of predictability for non Gaussian sequences that may display general non linear dependence in higher order properties. We test the null of martingale difference against parametric alternatives which can introduce linear or non linear dependence as generated by ARMA and all-pass restricted ARMA models, respectively. We also develop tests to check for linear predictability under the white noise null hypothesis parameterized by an all-pass model driven by martingale difference innovations and tests of non linear predictability on ARMA residuals. Our Lagrange Multiplier tests are developed from a loss function based on pairwise dependence measures of model residuals. The new tests have standard pivotal null asymptotic distribution and we discuss consistency against parametric and non parametric alternatives. We provide finite sample analysis of the properties of the new LM tests and investigate the predictability of several series of financial returns. We confirm that our tests are able to detect predictability in NYSE, AMEX, and NASDAQ returns while allowing for higher order dependence which cannot be properly addressed by previous methods based on serial independence assumptions.

Differential item functioning between English, German, and Spanish PROMIS® physical function ceiling items.

We investigated the validity of the German and Spanish translations of 35 new high functioning items added to the Patient Reported Outcomes Measurement Information System (PROMIS®) Physical Function item bank 2.0. We assessed differential item functioning (DIF) between three general population samples from Argentina, Germany, and the United States. PROMIS Physical Function data was collected in online panels from 3601 individuals (mean age, 41.6years old; range, 18-88years; 53.7% female). Of these, 1001 participants completed the Spanish version, 1000 completed the German version, and 1600 completed the English version. DIF was assessed by a multiverse analysis that systematically varied analytic choices across the entire range of plausible options within the logistic ordinal regression framework. Translated items generally met the assumptions of unidimensionality, monotonicity, and local independence. The 272 different analyses suggest consistent DIF between languages in four items. Test characteristic curves suggested that the magnitude and impact of DIF on the test scores were negligible for all items at the test level. After correcting for potential DIF, we observed greater scoring for physical functioning in Argentina compared to the US, Cohen's d = 0.25, [0.17, 0.33], and Argentina compared to Germany, Cohen's d = 0.23, [0.15, 0.32]. Our findings support the universal applicability of PROMIS Physical Function items across general populations in Argentina, Germany, and the U.S. The sensitivity analyses indicate that the identification of DIF items was robust for different data analytic decisions. Multiverse analysis is a promising approach to address lack of clear cutoffs in DIF identification.

Optimasi Klasifikasi Sentimen pada Komentar Online menggunakan Multinomial Naïve Bayes dan Ekstraksi Fitur TF-IDF serta N-grams

The Naïve Bayes (NB) algorithm is a classifier method that calculates simple probabilities and it is suitable for text classification in the context of sentiment analysis. The classic variant of NB is Multinomial Naïve Bayes (MNB). The weakness of the MNB algorithm is the assumption of feature independence. This research uses a dataset of comments and reviews from various online platforms. This study uses the proposed method to handle the weakness of the MNB algorithm, namely the use of TF-IDF feature extraction and N-grams (1-gram to 5-gram), and the use of Chi-Square feature selection, as well as handling dataset imbalance using SMOTE (oversampling and undersampling method). The results of this study show that the use of pentagram (5-gram) with data that has been oversampled by SMOTE produces the highest accuracy value of 94% and an Area Under Curve (AUC) value of 100%

Machine learning in predicting heart failure survival: a review of current models and future prospects.

Heart failure is a complex and prevalent condition with significant implications for patient management and survival prediction. Traditional predictive models often fall short in accuracy due to their reliance on pre-specified predictors and assumptions of variable independence. This review aims to assess the role of machine learning (ML) algorithms in predicting heart failure survival, comparing their performance with traditional statistical methods and identifying key predictive features. We conducted a review of studies utilizing ML algorithms for heart failure survival prediction. Data were sourced from PubMed/MEDLINE, Google Scholar, ScienceDirect, Embase, DOAJ, and the Cochrane Library, covering studies published until July 2024. A total of 10 studies were reviewed, encompassing 468,171 patients with heart failure. ML algorithms, particularly random forests and gradient boosting methods, demonstrated superior performance compared to traditional statistical models. These algorithms effectively identified key risk factors and stratified patients into risk categories with high accuracy. Notably, extreme learning machine (ELM) and CatBoost models showed exceptional predictive capabilities, as indicated by metrics such as Harrell's concordance index (C-index) and area under the curve (AUC). Key predictive features included ejection fraction (EF), serum creatinine (S Cr), and blood urea nitrogen (BUN). ML algorithms offer significant advantages in predicting heart failure survival by uncovering complex patterns and improving risk stratification. Their integration into clinical practice could lead to more personalized treatment strategies and enhanced patient outcomes. However, challenges such as data quality, model interpretability, and integration into clinical workflows need to be addressed.

Detecting departures from the conditional independence assumption in diagnostic latent class models: a simulation study

BackgroundLatent class models can be used to estimate diagnostic accuracy without a gold standard test. Early studies often assumed independence between tests given the true disease state, however this can lead to biased estimates when there are inter-test dependencies. Residual correlation plots and chi-squared statistics have been commonly utilized to assess the validity of the conditional independence assumption and, when it does not hold, identify which test pairs are conditionally dependent. We aimed to assess the performance of these tools with a simulation study covering a wide range of scenarios.MethodsWe generated data sets from a model with four tests and a dependence between tests 1 and 2 within the diseased group. We varied sample size, prevalence, covariance, sensitivity and specificity, with 504 combinations of these in total, and 1000 data sets for each combination. We fitted the conditional independence model in a Bayesian framework, and reported absolute bias, coverage, and how often the residual correlation plots, G2 and χ2 statistics indicated lack-of-fit globally or for each test pair.ResultsAcross all settings, residual correlation plots, pairwise G2 and χ2 detected the correct correlated pair of tests only 12.1%, 10.3%, and 10.3% of the time, respectively, but incorrectly suggested dependence between tests 3 and 4 64.9%, 49.7%, and 49.5% of the time. We observed some variation in this across parameter settings, with these tools appearing to perform more as intended when tests 3 and 4 were both much more accurate than tests 1 and 2. Residual correlation plots, G2 and χ2 statistics identified a lack of overall fit in 74.3%, 64.5% and 67.5% of models, respectively. The conditional independence model tended to overestimate the sensitivities of the correlated tests (median bias across all scenarios 0.094, 2.5th and 97.5th percentiles -0.003, 0.397) and underestimate prevalence and the specificities of the uncorrelated tests.ConclusionsResidual correlation plots and chi-squared statistics cannot be relied upon to identify which tests are conditionally dependent, and also have relatively low power to detect lack of overall fit. This is important since failure to account for conditional dependence can lead to highly biased parameter estimates.

Multiple Imputation for Systematically Missing Partner Variables in Survey Data

Systematically missing information on secondary respondents is a frequent problem in multiactor surveys. Budget and time constraints often prevent all variables collected for primary respondents (e.g., anchors) from being collected for secondary respondents (e.g., partners). Thus, a subset of variables are systematically missing for secondary respondents. This can severely limit the analysis potential of multiactor data, ruling out all research questions that would require (the same) information on primary and secondary respondents. The problem of systematically missing data is also present in other settings, for example, after changes in measurement instruments in repeated surveys or in ex post survey harmonization if one or more surveys did not include a specific variable. In these cases, using multiple imputation (MI) techniques to impute the missing variables is a common approach. The authors explore whether MI can be used when data on secondary respondents are systematically missing. Results from simulation studies show that imputation under the assumption of conditional independence for primary and secondary respondents variables leads to a strong bias toward zero in the estimated partial correlation between primary and secondary respondents. However, external data in the form of bridging studies can be used to estimate the partial correlation between the observed variable for the primary and the unobserved variable for the secondary respondent, leading to estimates with less bias after MI.

Measuring Spatial Ability: Analysis of Spatial Ability Test for Gulf State Students Using Item Response Theory

This study evaluates the psychometric properties of the spatial ability test using the three-parameter logistic model within item response theory. The final version of the scale comprised 29 dichotomous items, administered to a sample of 2,694 male and female students from grades 5 and 6 across schools in the Arab Gulf region. The test adhered to the three-parameter model, satisfying the assumptions of unidimensionality and local independence. The item difficulty parameters ranged from -1.541 to 1.735, discrimination parameters spanned from 0.419 to 5.252, and guessing parameters varied between 0.00 and 0.346. With a marginal reliability coefficient of 0.86, the scale demonstrated strong stability. These findings indicate that the test items align with established measurement principles, supporting the spatial ability test as a valid and reliable assessment tool for measuring spatial abilities in the Gulf region. The results have important implications for educational assessment in the Arab Gulf and could guide the development of similar assessments in other educational contexts. Further research is recommended to improve the test’s precision and explore its application in diverse educational settings.

Disentangled Representation Learning.

Disentangled Representation Learning (DRL) aims to learn a model capable of identifying and disentangling the underlying factors hidden in the observable data in representation form. The process of separating underlying factors of variation into variables with semantic meaning benefits in learning explainable representations of data, which imitates the meaningful understanding process of humans when observing an object or relation. As a general learning strategy, DRL has demonstrated its power in improving the model explainability, controlability, robustness, as well as generalization capacity in a wide range of scenarios such as computer vision, natural language processing, and data mining. In this article, we comprehensively investigate DRL from various aspects including motivations, definitions, methodologies, evaluations, applications, and model designs. We first present two well-recognized definitions, i.e., Intuitive Definition and Group Theory Definition for disentangled representation learning. We further categorize the methodologies for DRL into four groups from the following perspectives, the model type, representation structure, supervision signal, and independence assumption. We also analyze principles to design different DRL models that may benefit different tasks in practical applications. Finally, we point out challenges in DRL as well as potential research directions deserving future investigations. We believe this work may provide insights for promoting the DRL research in the community.

Assessing the robustness of the Galactic rotation curve inferred from the Jeans equations using Gaia DR3 and cosmological simulations

Context. Several authors have recently applied Jeans modelling to Gaia-based datasets to infer the circular velocity curve for the Milky Way. These works have consistently found evidence for a continuous decline in the rotation curve beyond ~15 kpc, which may indicate the existence of a light dark matter (DM) halo. Aims. Using a large sample of Gaia DR3 data, we aim to derive the rotation curve of the Milky Way using the Jeans equations, and to quantify the role of systematic effects, both in the data and those inherent to the Jeans methodology under the assumptions of axisym-metry and time independence. Methods. We used data from the Gaia DR3 radial velocity spectrometer sample, supplemented with distances inferred through Bayesian frameworks, to determine the radial variation of the second moments of the velocity distribution for stars close to the Galactic plane. We used these profiles to determine the rotation curve using the Jeans equations under the assumption of axisym-metry and explored how they vary with azimuth and position above and below the plane of the Galactic disc. We applied the same methodology to an N-body simulation of a Milky Way-like galaxy impacted by a satellite akin the Sagittarius dwarf, and to the Auriga suite of cosmological simulations. Results. The circular velocity curve we infer for the Milky Way is consistent with previous findings out to ~15 kpc, where our statistics are robust. Due to the larger number of stars in our sample, we are able to reveal evidence of disequilibrium and deviations from axisymmetry closer in. For example, we find that the second moment of VR flattens out at R ≳ 12.5 kpc, and that the second moment of Vϕ is different above and below the plane for R ≳ 11 kpc. Our exploration of the simulations indicates that these features are typical of galaxies that have been perturbed by external satellites. From the simulations, we also estimate that the difference between the true circular velocity curve and that inferred from Jeans equations can be as high as 15%, but that it is likely of the order of 10% for the Milky Way. This is higher than the systematic uncertainties associated with the observations or those linked to most modelling assumptions when using the Jeans equations. However, if the density of the tracer population were truncated at large radii instead of being exponential as often assumed, this could lead to the erroneous conclusion of a steeply declining rotation curve. Conclusions. We find that steady-state axisymmetric Jeans modelling becomes less robust at large radii, indicating that particular caution must be exercised when interpreting the rotation curve inferred in those regions. A more careful and sophisticated approach may be necessary for precision measurements of the DM content of our Galaxy.

Factors affecting farmers' choice to adopt risk management strategies: The application of multivariate and multinomial probit models

This study investigates the factors that impact farmers' adoption of risk management strategies (RMS) in Pakistan during times of uncertainty. The study examines farmers' RMS adoption choices using both multinomial probit (MNP) and multivariate probit (MVP). Data were collected from 382 sampled farmers from four districts in Khyber-Pakhtunkhwa (KP) province of Pakistan via a multistage sampling technique. This study utilizes the MNP model, considering the assumption of Independence of Irrelevant Alternatives (IIA) and incorporating correlated error terms. The objective is to understand farmers' behavior in risky situations and determine if there is heterogeneity. Results are compared with the MVP model to assess robustness and gain insights into farmers' decision-making processes.The research findings reveal that our results are robust, and farmers behave homogeneously in different settings of the provision of risk management strategies. Farmers adopt RMS individually or in combination to mitigate the adverse effects of natural calamities on their livelihood. The risk-averse farmers, who perceive weather-related risks as a threat, access credits and information, and have farms close to a river are more likely to adopt RMS, irrespective of the format of the strategies available. Moreover, the predicted probabilities and correlation of the RMS and RM categories further strengthened our model estimation. These findings provide insights into the behavior of farmers in adopting RMS which are helpful for policymakers and stakeholders in developing strategies to mitigate the impacts of natural calamities on farmers.

Hierarchical Nearest Neighbor Gaussian Process models for discrete choice: Mode choice in New York City

Standard Discrete Choice Models (DCMs) assume that unobserved effects that influence decision-making are independently and identically distributed among individuals. When unobserved effects are spatially correlated, the independence assumption does not hold, leading to biased standard errors and potentially biased parameter estimates. This paper proposes an interpretable Hierarchical Nearest Neighbor Gaussian Process (HNNGP) model to account for spatially correlated unobservables in discrete choice analysis. Gaussian Processes (GPs) are often regarded as lacking interpretability due to their non-parametric nature. However, we demonstrate how to incorporate GPs directly into the latent utility specification to flexibly model spatially correlated unobserved effects without sacrificing structural economic interpretation. To empirically test our proposed HNNGP models, we analyze binary and multinomial mode choices for commuting to work in New York City. For the multinomial case, we formulate and estimate HNNGPs with and without independence from irrelevant alternatives (IIA). Building on the interpretability of our modeling strategy, we provide both point estimates and credible intervals for the value of travel time savings in NYC. Finally, we compare the results from all proposed specifications with those derived from a standard logit model and a probit model with spatially autocorrelated errors (SAE) to showcase how accounting for different sources of spatial correlation in discrete choice can significantly impact inference. We also show that the HNNGP models attain better out-of-sample prediction performance when compared to the logit and probit SAE models, especially in the multinomial case.

A flamelet-based Eulerian transported PDF method for the modeling and simulation of supersonic combustion

This paper presents a high-efficiency and high-fidelity approach to model supersonic combustion using the extended flamelet-generated manifold (FGM) and the Eulerian transported probability density function (PDF), also known as the Eulerian stochastic fields (ESF) method. The efficiency benefits from the FGM, where the compressibility effects induced by shock waves are considered using two extra control variables, i.e., the pressure (p) and the absolute internal energy of the oxidizer (Eox), in addition to the mixture fraction (Z) and progress variable (Yc) in traditional flamelet tables. The joint PDF for these control variables is modeled using transported PDF based on the ESF method. The ESF method enhances accuracy in the prediction of turbulence-chemistry interactions, avoiding complexity induced by the presumed PDF in the flamelet table and ad-hoc presumed and independent joint PDF assumptions, e.g., the typical presumed β-PDF for Z and δ-PDF for Yc. This FGM-ESF method is tested in large eddy simulations of two canonical hydrogen supersonic flames: a strut-stabilized hydrogen supersonic flame (DLR case) and a transverse hydrogen jet flame in a high-enthalpy incoming flow (Stanford case). For both cases, results show that including the compressibility effects in the FGM table is essential for properly describing the flame behaviors near shock waves. The sub-grid PDF of control variables significantly influences near-wall and shear-layer combustion, and the ESF method demonstrates superior performance in predicting the near-wall reaction zone and the shear-layer reaction zone compared to the perfectly micro-mixed sub-grid model (δ-PDF) for the Stanford case. This study marks the first application of the FGM-ESF approach with a Z-Yc-Eox-p FGM table to simulate supersonic flames, offering a novel perspective for future modeling efforts in this domain.Novelty and Significance StatementThe novelty of this research lies in the development and application of an efficient computational approach that for the first time couples the extended flamelet-generated manifold (FGM) method with the Eulerian stochastic fields (ESF) to simulate supersonic flames. This combined FGM-ESF method incorporates pressure and oxidizer energy as additional control variables in the FGM tabulation to account for compressibility effects. The ESF is used to describe the sub-grid scale joint probability density function without relying on complex presumed functions. Application to a canonical strut-stabilized hydrogen supersonic flame and transverse hydrogen jet flames in a high-enthalpy incoming flow demonstrate the superior predictive capabilities of the FGM-ESF method in capturing flame dynamics. The comparison between four-dimensional and two-dimensional FGM tables provides new insights into the importance of compressibility effects for these configurations.

Exploring the impact of spatial autocorrelation on optimistic bias in cross-validation and assessing the effectiveness of spatial cross-validation

ABSTRACT Spatial autocorrelation is a fundamental property of spatial data, which violates the assumption of independence between training and test datasets in general cross-validation (CV). Previous studies have reported strong positive spatial autocorrelation generally leads to optimistic biases in general CV results. Spatial CV methods have been developed to address this bias, but their effectiveness remains controversial owing to their potential for excessively pessimistic estimations. This study examines the impact of spatial autocorrelation on general CV results and validates the effectiveness of spatial CV. The first simulation explores the impact of varying spatial autocorrelation levels on the general CV results. Specifically, strong and moderate positive spatial autocorrelation introduces optimistic biases, whereas weak positive or negative spatial autocorrelations have no significant impact. The second simulation shows spatial CV methods can mitigate the optimistic biases in general CV results when dealing with spatial data having strong and moderate positive spatial autocorrelations. However, the hyperparameters of spatial CV should be adjusted based on the level of spatial autocorrelation to avoid excessively pessimistic estimations.

A bivariate load-sharing model

The motivation of this work came from a data set obtained from an experiment performed on diabetic patients, with diabetic retinopathy disorder. The aim of this experiment is to test whether there is any significant difference between two different treatments which are being used for this disease. The two eyes can be considered as a two-component load-sharing system. In a two-component load-sharing system after the failure of one component, the surviving component has to shoulder extra load. Hence, it is prone to failure at an earlier time than what is expected under the original model. It may also happen sometimes that the failure of one component may release extra resources to the survivor, thus delaying the failure. In most of the existing literature, it has been assumed that at the beginning the lifetime distributions of the two components are independently distributed, which may not be very reasonable in this case. In this paper, we have introduced a new bivariate load-sharing model where the independence assumptions of the lifetime distributions of the two components at the beginning have been relaxed. In this present model, they may be dependent. Further, there is a positive probability that the two components may fail simultaneously. If the two components do not fail simultaneously, it is assumed that the lifetime of the surviving component changes based on the tampered failure rate assumption. The proposed bivariate distribution has a singular component. The likelihood inference of the unknown parameters has been provided. Simulation results and the analysis of the data set have been presented to show the effectiveness of the proposed model.