Aligned Rank Tests Research Articles

A multi-objective comparison of CNN architectures in Arctic human-built infrastructure mapping from sub-meter resolution satellite imagery

ABSTRACTRisk assessment of infrastructure exposed to ice-rich permafrost hazards is essential for climate change adaptation in the Arctic. As this process requires up-to-date, comprehensive, high-resolution maps of human-built infrastructure, gaps in such geospatial information and knowledge of the applications required to produce it must be addressed. Therefore, this study highlights the ongoing development of a deep learning approach to efficiently map the Arctic built environment by detecting nine different types of structures (detached houses, row houses, multi-story blocks, non-residential buildings, roads, runways, gravel pads, pipelines, and storage tanks) from recently-acquired Maxar commercial satellite imagery (<1 m resolution). We conducted a multi-objective comparison, focusing on generalization performance and computational cost, of nine different semantic segmentation architectures. K-fold cross validation was used to estimate the average F1-score of each architecture and the Friedman Aligned Ranks test with the Bergmann-Hommel post-hoc procedure was applied to test for significant differences in generalization performance. ResNet-50-UNet++ performs significantly better than five out of the other eight candidate architectures; no significant difference was found in the pairwise comparisons of ResNet-50-UNet++ to ResNet-50-MANet, ResNet-101-MANet, and ResNet-101-UNet++. We then conducted a high-performance computing scaling experiment to compare the number of service units and runtime required for model inferencing on a hypothetical pan-Arctic scale dataset. We found that the ResNet-50-UNet++ model could save up to ~ 54% on service unit expenditure, or ~ 18% on runtime, when considering operational deployment of our mapping approach. Our results suggest that ResNet-50-UNet++ could be the most suitable architecture (out of the nine that were examined) for deep learning-enabled Arctic infrastructure mapping efforts. Overall, our findings regarding the differences between the examined CNN architectures and our methodological framework for multi-objective architecture comparison can provide a foundation that may propel future pan-Arctic GeoAI mapping efforts of infrastructure.

Pairwise comparisons using ranks in block designs

Friedman’s rank test and the associated aligned rank test are the standard rank alternatives to the classical linear models F statistic for the randomized complete block design (RCBD). However, in current practice there are no good rank alternatives to the Tukey–Kramer all pairwise comparisons procedure for the normal linear model. For example, the standard rank method found in Section 7.3 of Hollander et al. [John Wiley & Sons, Inc., Hoboken, New Jersey; 2014] is based on the permutation distribution under the complete null of no differences and thus cannot strongly control the family-wise error rate (FWER). It also has low power. However, we show that the closed method introduced by Marcus et al. [Biometrika 63 (3);1976:655–60] applied to aligned ranks has strong FWER control and good power compared to the Tukey–Kramer method for long-tailed error distributions.

Empirical Power and Type I Error of Covariate Adjusted Nonparametric Methods

In clinical trials, practitioners collect baseline covariates for enrolled patients prior to treatment assignment. In recent guidance from Food and Drug Administration and European Medicines Agency, regulators encourage practitioners to utilize baseline information at the analysis stage to improve the efficiency. However, the current guidance focused on linear or non-linear modelling approach. Nonparametric statistical methods were not focus in the guidance. In this article, we conducted simulations of several covariate-adjusted nonparametric statistical tests. Wilcoxon rank sum test is a widely used method for non-normally distributed response variables between two groups but its original form does not take into account the possible effect of covariates. We investigated the empirical power and the type I error of the Wilcoxon type test statistics under various settings of covariate adjustments commonly encountered in clinical trials. In addition to Wilcoxon type test statistics, we also compared simulation results to more advanced nonparametric test statistics such as the aligned rank test and Jaeckel, Hettmansperger-McKean test. The simulation result shows when there is covariate imbalance, applying Wilcoxon rank sum test without adjusting the covariates will become problematic. The survey of the covariate adjustments for varies tests under investigation gives brief guidance to trial practitioners in real practice, particularly whose baseline covariates are not well balanced.

A Comparative Analysis on Prediction Performance of Regression Models during Machining of Composite Materials.

Modeling the interrelationships between the input parameters and outputs (responses) in any machining processes is essential to understand the process behavior and material removal mechanism. The developed models can also act as effective prediction tools in envisaging the tentative values of the responses for given sets of input parameters. In this paper, the application potentialities of nine different regression models, such as linear regression (LR), polynomial regression (PR), support vector regression (SVR), principal component regression (PCR), quantile regression, median regression, ridge regression, lasso regression and elastic net regression are explored in accurately predicting response values during turning and drilling operations of composite materials. Their prediction performance is also contrasted using four statistical metrics, i.e., mean absolute percentage error, root mean squared percentage error, root mean squared logarithmic error and root relative squared error. Based on the lower values of those metrics and Friedman rank and aligned rank tests, SVR emerges out as the best performing model, whereas the prediction performance of median regression is worst. The results of the Wilcoxon test based on the drilling dataset identify the existence of statistically significant differences between the performances of LR and PCR, and PR and median regression models.

Protocol - Evaluating the Effect of ACA Medicaid Expansion on Mortality During the COVID-19 Pandemic Using County-level Matching.

States are able to choose whether to expand Medicaid as part of the Affordable Care Act (ACA); thus it is of interest to understand the impact of this policy choice. In this protocol, we outline a study on the impact of Medicaid expansion as part of the ACA on mortality during the COVID-19 pandemic in the United States. County-level matching using full, optimal matching with a propensity score model is used to estimate causal effects in this observational study. Due to the provisional nature of mortality data in 2020 as reported by the CDC, we outline a modified aligned rank test to account for censored data as well as reporting lags for different states. We aim to make connections between statistical and ethnographic methodologies by particularly examining adjacent counties and similar counties that are in the same region of the US and in vastly different regions of the US. Finally, we aim to add to the growing literature about the effect of ACA Medicaid expansion on mortality by calculating effects, disaggregating by race.

Structure\u2013activity relationship-based chemical classification of highly imbalanced Tox21 datasets

The specificity of toxicant-target biomolecule interactions lends to the very imbalanced nature of many toxicity datasets, causing poor performance in Structure–Activity Relationship (SAR)-based chemical classification. Undersampling and oversampling are representative techniques for handling such an imbalance challenge. However, removing inactive chemical compound instances from the majority class using an undersampling technique can result in information loss, whereas increasing active toxicant instances in the minority class by interpolation tends to introduce artificial minority instances that often cross into the majority class space, giving rise to class overlapping and a higher false prediction rate. In this study, in order to improve the prediction accuracy of imbalanced learning, we employed SMOTEENN, a combination of Synthetic Minority Over-sampling Technique (SMOTE) and Edited Nearest Neighbor (ENN) algorithms, to oversample the minority class by creating synthetic samples, followed by cleaning the mislabeled instances. We chose the highly imbalanced Tox21 dataset, which consisted of 12 in vitro bioassays for > 10,000 chemicals that were distributed unevenly between binary classes. With Random Forest (RF) as the base classifier and bagging as the ensemble strategy, we applied four hybrid learning methods, i.e., RF without imbalance handling (RF), RF with Random Undersampling (RUS), RF with SMOTE (SMO), and RF with SMOTEENN (SMN). The performance of the four learning methods was compared using nine evaluation metrics, among which F1 score, Matthews correlation coefficient and Brier score provided a more consistent assessment of the overall performance across the 12 datasets. The Friedman’s aligned ranks test and the subsequent Bergmann-Hommel post hoc test showed that SMN significantly outperformed the other three methods. We also found that a strong negative correlation existed between the prediction accuracy and the imbalance ratio (IR), which is defined as the number of inactive compounds divided by the number of active compounds. SMN became less effective when IR exceeded a certain threshold (e.g., > 28). The ability to separate the few active compounds from the vast amounts of inactive ones is of great importance in computational toxicology. This work demonstrates that the performance of SAR-based, imbalanced chemical toxicity classification can be significantly improved through the use of data rebalancing.

Stratified two‐sample design: A review on nonparametric methods

AbstractIn this article, a comparison between the most promising nonparametric tests in a two‐sample stratified design for practical uses is performed. We compared methods that exhibit good small‐sample properties in order to be used with the most common stratum sizes. From the literature we identified as promising the following solutions: the aligned rank test, a small‐sample approximation for the ANOVA‐type statistic based on an unweighted average of all the distributions, and an asymptotic permutation distribution for the Wald‐type statistic. We also developed a permutation version of the aligned rank test and another permutation testing procedure based on the Mann‐Whitney statistic using the nonparametric combination procedure. All selected methods were compared by means of a simulation study. The results show that the aligned rank test and its permutation version perform better in most of the considered situations. Data from a genuine industrial problem were used for illustration purposes and to confirm the simulation results.

Data allocation in distributed database systems: a novel hybrid method based on differential evolution and variable neighborhood search

Data allocation problem (DAP) in distributed database systems is a NP-hard optimization problem with significant importance in parallel processing environments. The solution of problem aims to minimize the total cost of transactions and settlement of queries in which he main cost regards to the data transmission through the distributed system. These costs are affected by the strategy how to allocate fragments to the sites. Researchers have been solving this challenging problem by applying soft computing methods especially evolutionary algorithms. This study proposes a novel hybrid method based on differential evolution (DE) algorithm and variable neighborhood search (VNS) mechanism for solving DAP problem. The suggested hybrid method (DEVNS) aims to increase the performance of DE algorithm by applying effective selection and crossover operators. Moreover, DEVNS goals to improve the solutions found so far using VNS technique. By applying VNS, more promising parts of search space can be extracted. Eventually, the introduced DEVNS explores the search space via DE and fulfills more exploitation by neighborhood search mechanism. Performance of proposed DEVNS is experimentally evaluated against nine state-of-the-art methods using well-known benchmarks reported in literature. Obtained results exhibits that proposed DEVNS takes the first position for 13 of 20 test problems. Likewise, Friedman aligned rank test is carried out to demonstrate that there is significant statistical difference between all methods.

A novel chaotic Jaya algorithm for unconstrained numerical optimization

Jaya algorithm is one of the recent algorithms developed to solve optimization problems. The basic concept of this algorithm consists in moving the obtained solution, for a given problem, toward the best solution and avoiding the worst one. However, it severely suffers from premature convergence problem and therefore can be easily trapped in local optimums. This study aimed to alleviate these drawbacks and improve the performance of the original Jaya algorithm. Here, three new mutation strategies were implemented in the original Jaya to improve both its global and local search abilities. Chaotic maps were proved to be able to boost the search capabilities of meta-heuristic algorithms. Therefore, after demonstrating its chaotic behavior through the sensitivity to initial conditions, topological transitivity and the density of periodic points, we proposed a new 2D cross chaotic map. The chaotic sequences provided by the proposed chaotic map were embedded into the original Jaya algorithm to generate the initial population and control the search equations. It is worth mentioning that the modifications incorporated in the original algorithm did not affect its two essential characteristics, i.e., simplicity and nonrequirement of additional control parameters. As case studies, sixteen benchmark functions were used to evaluate the performance of the proposed chaotic Jaya algorithm (C-Jaya) regarding solution accuracy and convergence speed. Comparisons with some other meta-heuristic algorithms for low-, middle- and high-dimensional benchmark functions show that the proposed C-Jaya algorithm enhances the performance of original Jaya significantly. Moreover, it offers the fastest global convergence, the highest solution quality and it is the most robust on almost all the test functions among all the algorithms. Nonparametric statistical procedures, i.e., Friedman test, Friedman aligned ranks test and Quade test, conducted to analyze the obtained results, show the superiority of the proposed algorithm.

Validation of the Oral Hygiene Habits Scale: Relationships with sociodemographic variables in the general and clinical population of Monterrey, Mexico.

Several socioeconomic factors are associated with poor oral hygiene habits. A version of the Oral Hygiene Habits Scale (OHHS) was developed in Mexico to measure these factors; however, its relationship with sociodemographic variables has not been studied. The verification of these relationships could contribute to the validation of the scale. Objective: To evaluate the relationship between oral hygiene habits and sociodemographic variables of sex, age, schooling, self-defined socioeconomic stratum, occupation and marital status in the general and clinical population of Monterrey, Mexico. Materials and Methods: A general population sample (GPS) of 256participants and a clinical sample (CPS) of 240 participants were studied. The OHHS consisted of an eight-item Likert scale of 4 points ranging from 0 to 4. A descriptive correlational study was performed with a cross-sectional design. Data were analyzed using the Mann-Whitney U test, Kruskal-Wallis test, Spearman correlation coefficient, Cramer's V coefficient, and multivariate aligned rank test. Results: In GPS and CPS groups, OHHS was related to sex, schooling, socioeconomic stratum, occupation and marital status, but not to age. There were no significant interactions between the samples (GPS and CPS) and sociodemographic variables. Conclusion: There is a statistically significant relationship between oral hygiene habits and some sociodemographic variables in the general and clinical population. This relationship supports the validity of the OHHS.

Aligned rank tests in measurement error model

Aligned rank tests are introduced in the linear regression model with possible measurement errors. Unknown nuisance parameters are estimated first and then classical rank tests are applied on the residuals. Two situations are discussed: testing about an intercept in the linear regression model considering the slope parameter as nuisance and testing of parallelism of several regression lines, i.e. whether the slope parameters of all lines are equal. Theoretical results are derived and the simulation study is also made to illustrate good performance of introduced tests.

Count Data Analysis in Randomised Clinical Trials

Choosing the best model for analysis of count data in randomised clinical trials is complicated. In this paper, we review count data analysis with different parametric and non-parametric methods used in randomised clinical trials, and we define procedures for choosing between the two methods and their subtypes. We focus on analysis of simple count data and do not consider methods for analyzing longitudinal count data or Bayesian statistical analysis. We recommend that: (1) a detailed statistical analysis plan is published prior to access to trial data; (2) if there is lack of evidence for a parametric model, both non-parametric tests (either the van Elteren test or the Tadap2 test, based on an aligned rank test with equal stratum weights) and bootstrapping should be used as default methods; and (3) if more than two intervention groups are compared, then the Kruskal–Wallis test may be considered. If our recommendations are followed, the risk of biased results ensuing from analysis of count data in randomised clinical trials is expected to decrease.

Non‐parametric three‐way mixed ANOVA with aligned rank tests

Research problems that require a non-parametric analysis of multifactor designs with repeated measures arise in the behavioural sciences. There is, however, a lack of available procedures in commonly used statistical packages. In the present study, a generalization of the aligned rank test for the two-way interaction is proposed for the analysis of the typical sources of variation in a three-way analysis of variance (ANOVA) with repeated measures. It can be implemented in the usual statistical packages. Its statistical properties are tested by using simulation methods with two sample sizes (n = 30 and n = 10) and three distributions (normal, exponential and double exponential). Results indicate substantial increases in power for non-normal distributions in comparison with the usual parametric tests. Similar levels of Type I error for both parametric and aligned rank ANOVA were obtained with non-normal distributions and large sample sizes. Degrees-of-freedom adjustments for Type I error control in small samples are proposed. The procedure is applied to a case study with 30 participants per group where it detects gender differences in linguistic abilities in blind children not shown previously by other methods.

Robust permutation tests for homogeneity of fingerprint patterns of dioxin congener profiles

Fingerprint analysis comparing the polychlorinated dibenzo‐p‐dioxin and dibenzofuran (PCDD/F) congener profile patterns of collected samples with those of potential dioxin emission source(s) is an important tool for identifying environmental dioxin pollution. The constraint that the proportions of the 17 PCDD/F congeners comprising a fingerprint sum up to one motivates a multivariate gamma distribution, which leads to a Dirichlet distribution. Because of the complexity in restricted likelihood ratio tests and typical sample size limitations resulting from laboratory analysis costs, permutation test procedures are employed for hypothesis testing of the homogeneity of congener profiles. Pearson‐type chi‐squared tests based on the Dirichlet distribution (DM) assumption, the generalized form of DM using the arithmetic mean and geometric mean of the proportions, and the robust aligned rank test, are proposed and compared through simulations. PCDD/F samples collected from the stack of a local municipal solid waste incinerator and from ambient air near the municipal solid waste incinerator in Taiwan were illustrated as an example. The simulation results showed that the aligned rank test, followed by the DM‐based test, was generally robust to distributional assumptions and had high statistical power. The arithmetic‐mean‐based and geometric‐mean‐based tests outperformed one another in different conditions, dependent on the underlying distribution. Copyright © 2012 John Wiley &amp; Sons, Ltd.

Rank-Based Analyses of Stratified Experiments: Alternatives to the van Elteren Test

The van Elteren test, a popular extension of the Wilcoxon rank sum test for comparing two treatments in a stratified experiment, uses within-stratum ranks and assigns larger weight to rank sums from smaller strata. It performs well if the treatment effect is constant across strata, but it can be grossly inefficient otherwise. Existing alternatives to the van Elteren test include tests that use overall ranks and/or equal stratum weights, and the aligned rank test of Hodges and Lehmann. We develop two new testing procedures. The first replaces the stratum weights in the van Elteren test with corresponding weights based on estimated treatment effects and sample sizes. The second uses the aligned rank test and a version of the van Elteren test with equal stratum weights in a sequential manner via an adaptive multiple testing strategy. The new and existing tests are compared to the van Elteren test via extensive simulations. The results show that the aligned rank test and the proposed adaptive testing procedure convincingly outperform the van Elteren test in terms of statistical power; the latter is recommended as a default based on overall performance. Data from two clinical trials are used for illustration.

Aligned Rank Tests for Interactions in Split-Plot Designs: Distributional Assumptions and Stochastic Heterogeneity

Three aligned rank methods for transforming data from multiple group repeated measures (split-plot) designs are reviewed. Univariate and multivariate statistics for testing the interaction in split-plot designs are elaborated. Computational examples are presented to provide a context for performing these ranking procedures and statistical tests. SAS/IML and SPSS syntax code to perform the procedures is included in the Appendix.

JMASM 26: Hettmansperger and Mckean Linear Model Aligned Rank Test for the Single Covariate and One-Way ANCOVA Case (SAS)

A SAS program (SAS 9.1.3 release, SAS Institute, Cary, N.C.) is presented to implement the Hettmansperger and McKean (1983) linear model aligned rank test (nonparametric ANCOVA) for the single covariate and one-way ANCOVA case. As part of this program, SAS code is also provided to derive the residuals from the regression of Y on X (which is step 1 in the Hettmansperger and McKean procedure) using either ordinary least squares regression (proc reg in SAS) or robust regression with MM estimation (proc robustreg in SAS).

Aligned Rank Tests As Robust Alternatives For Testing Interactions In Multiple Group Repeated Measures Designs With Heterogeneous Covariances

Data simulation was used to investigate whether tests performed on aligned ranks (Beasley, 2002) could be used as robust alternatives to parametric methods for testing a split-plot interaction with non-normal data and heterogeneous covariance matrices. Results indicated the aligned rank method do not have any distinct advantage over parametric methods in this situation.

Affine-invariant aligned rank tests for the multivariate general linear model with VARMA errors

We develop optimal rank-based procedures for testing affine-invariant linear hypotheses on the parameters of a multivariate general linear model with elliptical VARMA errors. We propose a class of optimal procedures that are based either on residual (pseudo-)Mahalanobis signs and ranks, or on absolute interdirections and lift-interdirection ranks, i.e., on hyperplane-based signs and ranks. The Mahalanobis versions of these procedures are strictly affine-invariant, while the hyperplane-based ones are asymptotically affine-invariant. Both versions generalize the univariate signed rank procedures proposed by Hallin and Puri (J. Multivar. Anal. 50 (1994) 175), and are locally asymptotically most stringent under correctly specified radial densities. Their AREs with respect to Gaussian procedures are shown to be convex linear combinations of the AREs obtained in Hallin and Paindaveine (Ann. Statist. 30 (2002) 1103; Bernoulli 8 (2002) 787) for the pure location and purely serial models, respectively. The resulting test statistics are provided under closed form for several important particular cases, including multivariate Durbin–Watson tests, VARMA order identification tests, etc. The key technical result is a multivariate asymptotic linearity result proved in Hallin and Paindaveine (Asymptotic linearity of serial and nonserial multivariate signed rank statistics, submitted).

Rank Regression in Ranked-Set Samples

The use of statistical methods based on ranked-set sampling can lead to a substantial improvement over analog methods associated with simple random sampling schemes. This article develops a rank-based estimator and testing procedures for linear models for ranked-set samples. The estimator is defined as the minimizer of the rank dispersion function with Wilcoxon scores. It is shown that the estimator of the regression parameter is asymptotically normal and has higher Pitman asymptotic efficiency than a simple random-sample rank regression estimator. Three testing procedures are developed to test a general linear hypothesis: dispersion, Wald, and aligned rank tests. It is shown that all of these test statistics converge to a chi-squared distribution and the aligned rank test reduces to a simple random-sample analog of Kruskal-Wallis test for one-way analysis of variance. Under the assumption of perfect judgment ranking, optimal allocation of order statistics are constructed for set sizes smaller than 7. The optimal allocation procedures quantify middle observation(s) for symmetric unimodal distributions and smallest (largest) observation for right- (left-) skewed distributions.