Χ2 Distribution Research Articles

Personalised news filtering and recommendation system using Chi-square statistics-based K-nearest neighbour (χ2SB-KNN) model

ABSTRACTRecommendation problem has been extensively studied by researchers in the field of data mining, database and information retrieval. This study presents the design and realisation of an automated, personalised news recommendations system based on Chi-square statistics-based K-nearest neighbour (χ2SB-KNN) model. The proposed χ2SB-KNN model has the potential to overcome computational complexity and information overloading problems, reduces runtime and speeds up execution process through the use of critical value of χ2 distribution. The proposed recommendation engine can alleviate scalability challenges through combined online pattern discovery and pattern matching for real-time recommendations. This work also showcases the development of a novel method of feature selection referred to as Data Discretisation-Based feature selection method. This is used for selecting the best features for the proposed χ2SB-KNN algorithm at the preprocessing stage of the classification procedures. The implementation of the proposed χ2SB-KNN model is achieved through the use of a developed in-house Java program on an experimental website called OUC newsreaders’ website. Finally, we compared the performance of our system with two baseline methods which are traditional Euclidean distance K-nearest neighbour and Naive Bayesian techniques. The result shows a significant improvement of our method over the baseline methods studied.

Confidence Intervals of Fit Indexes by Inverting a Bootstrap Test

Fit indexes are an important tool in the evaluation of model fit in structural equation modeling (SEM). Currently, the newest confidence interval (CI) for fit indexes proposed by Zhang and Savalei (2016) is based on the quantiles of a bootstrap sampling distribution at a single level of misspecification. This method, despite a great improvement over naive and model-based bootstrap methods, still suffers from unsatisfactory coverage. In this work, we propose a new method of constructing bootstrap CIs for various fit indexes. This method directly inverts a bootstrap test and produces a CI that involves levels of misspecification that would not be rejected in a bootstrap test. Similar in rationale to a parametric CI of root mean square error of approximation (RMSEA) based on a noncentral χ2 distribution and a profile-likelihood CI of model parameters, this approach is shown to have better performance than the approach of Zhang and Savalei (2016), with more accurate coverage and more efficient widths.

Empirical likelihood for quantile regression models with response data missing at random

Abstract This paper studies quantile linear regression models with response data missing at random. A quantile empirical-likelihood-based method is proposed firstly to study a quantile linear regression model with response data missing at random. It follows that a class of quantile empirical log-likelihood ratios including quantile empirical likelihood ratio with complete-case data, weighted quantile empirical likelihood ratio and imputed quantile empirical likelihood ratio are defined for the regression parameters. Then, a bias-corrected quantile empirical log-likelihood ratio is constructed for the mean of the response variable for a given quantile level. It is proved that these quantile empirical log-likelihood ratios are asymptotically χ2 distribution. Furthermore, a class of estimators for the regression parameters and the mean of the response variable are constructed, and the asymptotic normality of the proposed estimators is established. Our results can be used directly to construct the confidence intervals (regions) of the regression parameters and the mean of the response variable. Finally, simulation studies are conducted to assess the finite sample performance and a real-world data set is analyzed to illustrate the applications of the proposed method.

Clinical characteristics and outcomes of patients with atrial fibrillation: impact of an alternative primary diagnosis

Objective To explore the impact of atrial fibrillation (AF) recognized at primary diagnosis on clinical features and outcomes of patients with AF in emergency service. Methods Data were collected from consecutive patients admitted in resuscitation room in the Department of Emergency (ED) of a major comprehensive teaching hospital, from January 1, 2011 through December 31, 2015. Patients were checked by electrocardiogram examination and / or monitored in resuscitation room after admission, and were divided into patients with AF recognized at a primary diagnosis and those with AF judged by alternative primary diagnoses in ED. The main criteria of prognosis were the length of resuscitation room stay, number of repeated ED visits, and outcome scale (such as death, transferred to intensive units, transferred to general wards, or direct discharge). Non-paired student t test, χ2, and circular distribution analysis were performed using SPSS 10.0 and EXCEL 2007 software. Results A total of 929 patients with mean age of (70.3±12.7)years, and 502 (54.0%) female were enrolled. There were 122 cases with AF not recognized at primary diagnosis but by an alternative primary diagnosis (non-primary group, NPG), and 807 cases with AF recognized at primary diagnosis (primary group, PG). Compared with the PG, the patients were older [(76.9±9.3)vs.(68.7±14.4), P<0.01], had more comorbidities [(1.75±1.26)vs.(0.08±0.39), P<0.01], higher APACHE Ⅱ scores [(17.89±8.19)vs.(8.64±4.15), P<0.01], longer resuscitation room stay (P<0.01), higher mortality (11.5% vs. 0.2%, OR=52.176, 95%CI: 11.698-232.710, χ2=78.928, P<0.01) and a higher percentage of transferring to intensive care unit (14.8% vs. 5.1%, OR=3.234, 95%CI: 1.791-5.838, χ2=16.674, P<0.01) in NPG. There were no significant difference in number of repeated-visits in ED between the PG and the NPG. Conclusion Patients with AF in the ED judged by alternative primary diagnosis are older and have more comorbidities, higher mortality and higher probability to be transferred to intensive care unit than patients with AF directly recognized by a primary diagnosis. This cohort of patients with special characteristics should be meticulously cared for and be distinguished from the patients with AF crystal clear at a primary diagnosis. Future studies are needed to examine the specific impact of AF on outcomes in the setting of primary diagnoses in ED. Key words: Atrial fibrillation; Emergency; Non-primary diagnosis; Mortality

On the statistical power of Baarda’s outlier test and some alternative

Abstract Baarda’s outlier test is one of the best established theories in geodetic practice. The optimal test statistic of the local model test for a single outlier is known as the normalized residual. Also other model disturbances can be detected and identified with this test. It enjoys the property of being a uniformly most powerful invariant (UMPI) test, but is not a uniformly most powerful (UMP) test. In this contribution we will prove that in the class of test statistics following a common central or non-central χ2 distribution, Baarda’s solution is also uniformly most powerful, i.e. UMPχ2 for short. It turns out that UMPχ2 is identical to UMPI, such that this proof can be seen as another proof of the UMPI property of the test. We demonstrate by an example that by means of the Monte Carlo method it is even possible to construct test statistics, which are regionally more powerful than Baarda’s solution. They follow a so-called generalized χ2 distribution. Due to high computational costs we do not yet propose this as a ”new outlier detection method”, but only as a proof that it is in principle possible to outperform the statistical power of Baarda’s test.

Influence of axial-force errors on the deformation of the 4 m lightweight mirror and its correction.

The lightweight primary mirror of the four meter Chinese telescope is supported axially by a passive hydraulic system, in parallel with an active system. The figure is sensitive to the errors in the axial forces, which can be effectively corrected using active optics. The squared RMS of this figure approximately follows the χ2 distribution, as verified by Monte Carlo analysis. The probability distributions of the squared norm of the active-forces vector, of the squared norm of the leg-lengths vector in the hexapod platform, and of the squared RMS of the residual figure are discussed in terms of the bending modes obtained by singular-value decomposition of the influence matrix, the optical sensitivity matrix deduced through ray trace, and the Jacobian matrix of the hexapod platform. The results show that, within the 90% confidence interval, the RMS of the figure caused by the axial-force errors is less than 232.38nm and is corrected to less than 3.26nm by the active optics. The maximum values of the corresponding squared norms of the active-force vector and of the leg-lengths vector are deduced.

Comparing Multilevel and Classical Confirmatory Factor Analysis Parameterizations of Multirater Data: A Monte Carlo Simulation Study

This simulation study assesses the statistical performance of two mathematically equivalent parameterizations for multitrait–multimethod data with interchangeable raters—a multilevel confirmatory factor analysis (CFA) and a classical CFA parameterization. The sample sizes of targets and raters, the factorial structure of the trait factors, and rater missingness are varied. The classical CFA approach yields a high proportion of improper solutions under conditions with small sample sizes and indicator-specific trait factors. In general, trait factor related parameters are more sensitive to bias than other types of parameters. For multilevel CFAs, there is a drastic bias in fit statistics under conditions with unidimensional trait factors on the between level, where root mean square error of approximation (RMSEA) and χ2 distributions reveal a downward bias, whereas the between standardized root mean square residual is biased upwards. In contrast, RMSEA and χ2 for classical CFA models are severely upwardly biased in conditions with a high number of raters and a small number of targets.

Community adaptations to an impending food desert in rural Appalachia, USA

The United States Department of Agriculture (USDA) describes a food desert as an urban neighborhood or rural town without ready access to fresh, healthy, and affordable food. An estimated 2.3 million rural Americans live in food deserts. One goal of the USDA is to eliminate food deserts. However, at a time when some food deserts are being eliminated, hundreds of grocery stores are closing, causing other food deserts to arise. The literature is scarce on how a community adapts to an impending food desert. Alderson, West Virginia, USA (population 1184) rallied to face an impending food desert when the only grocery store in town closed in December 2014. This study investigated how this small rural community adapted to its oncoming food desert. A community member survey was administered to 155 Alderson families (49%) to determine how the new food desert affected family food acquisition and storage behaviors. A restaurant survey was given to the town's four restaurants to determine how the food desert affected their businesses. Sales data for a new food hub (Green Grocer) was obtained to see if this new initiative offset the negative effects of the food desert. ANOVA and <i>t</i>-tests were used to compare group numerical data. Two group response rates were compared by testing the equality of two proportions. Categorical data were analyzed with the &#967;2 or frequency distribution analysis. Group averages are reported as mean ± standard error of the mean. Significance for all analyses was set at <i>p</i><0.05. Even though 86% of the population shopped at the new Green Grocer, 77% did most of their shopping at a store at least 17.7 km (11 miles) from home. The number of long-distance monthly shopping trips made after the food desert (3.3±0.4) did not change significantly (<i>p</i>=0.16) from the number before the food desert (2.8±0.3). Price comparisons among the Green Grocer and three distant supermarkets showed a 30% savings by traveling to distant supermarkets. Frequency of monthly restaurant visits did not change after the emergence of the food desert (2.98±0.54 vs 3.05±0.51, <i>p</i>=0.85). However, restaurant patrons requested to buy fresh produce and dairy from the restaurants to use for their own home cooking. Food pantry use increased by 43%, with community members requesting more fresh produce, meat, and dairy. The food desert triggered a 21% increase in home gardening and an 11% increase in home food preservation. Opening a Green Grocer offset only some of the effects of the food desert, because community members use it as a convenience store to purchase fresh produce and dairy products that families may lack before their next long-distance trip to a supermarket. Alderson's low-income residents now rely more heavily on food pantry assistance, while a small number of other residents have started gardening and food preservation. The first factor governing food acquisition behavior in rural Appalachia is food pricing, with the proximity of food access coming in second. How to overcome these two major barriers to food security in the midst of current economics and marketing remains to be answered.

A comparison of likelihood ratio tests and Rao's score test for three separable covariance matrix structures.

The problem of testing the separability of a covariance matrix against an unstructured variance-covariance matrix is studied in the context of multivariate repeated measures data using Rao's score test (RST). The RST statistic is developed with the first component of the separable structure as a first-order autoregressive (AR(1)) correlation matrix or an unstructured (UN) covariance matrix under the assumption of multivariate normality. It is shown that the distribution of the RST statistic under the null hypothesis of any separability does not depend on the true values of the mean or the unstructured components of the separable structure. A significant advantage of the RST is that it can be performed for small samples, even smaller than the dimension of the data, where the likelihood ratio test (LRT) cannot be used, and it outperforms the standard LRT in a number of contexts. Monte Carlo simulations are then used to study the comparative behavior of the null distribution of the RST statistic, as well as that of the LRT statistic, in terms of sample size considerations, and for the estimation of the empirical percentiles. Our findings are compared with existing results where the first component of the separable structure is a compound symmetry (CS) correlation matrix. It is also shown by simulations that the empirical null distribution of the RST statistic converges faster than the empirical null distribution of the LRT statistic to the limiting χ2 distribution. The tests are implemented on a real dataset from medical studies.

On scheduling of deception attacks for discrete-time networked systems equipped with attack detectors

This paper is concerned with the attack scheduling of deception attacks for discrete-time systems with attack detection. Specifically, for a class of Kalman filters with χ2 detectors, an attacker with the given attack task needs to decide how many the maximum number is or how much the attack probability is for different kinds of attack scenarios (i.e. consecutive deception attacks or randomly launched deception attacks). Firstly, in light of the property of χ2 distribution, the predictions of detection probabilities under attacks are calculated in mean-square sense for these two attack scenarios. Then, in terms of predicted probabilities, the deception attack schemes are designed in the framework of Kalman filtering. For the case of consecutive attacks, the maximum number of attacks is determined via recursive Riccati-like difference equations. For the case of randomly launched deception attacks, the desired attack probability is obtained by solving a Riccati-like equation. Finally, a numerical example on target tracking is presented to demonstrate the effectiveness of the proposed suboptimal attack schemes.

Biostatistics Series Module 4: Comparing Groups - Categorical Variables.

Categorical variables are commonly represented as counts or frequencies. For analysis, such data are conveniently arranged in contingency tables. Conventionally, such tables are designated as r × c tables, with r denoting number of rows and c denoting number of columns. The Chi-square (χ2) probability distribution is particularly useful in analyzing categorical variables. A number of tests yield test statistics that fit, at least approximately, a χ2 distribution and hence are referred to as χ2 tests. Examples include Pearson's χ2 test (or simply the χ2 test), McNemar's χ2 test, Mantel–Haenszel χ2 test and others. The Pearson's χ2 test is the most commonly used test for assessing difference in distribution of a categorical variable between two or more independent groups. If the groups are ordered in some manner, the χ2 test for trend should be used. The Fisher's exact probability test is a test of the independence between two dichotomous categorical variables. It provides a better alternative to the χ2 statistic to assess the difference between two independent proportions when numbers are small, but cannot be applied to a contingency table larger than a two-dimensional one. The McNemar's χ2 test assesses the difference between paired proportions. It is used when the frequencies in a 2 × 2 table represent paired samples or observations. The Cochran's Q test is a generalization of the McNemar's test that compares more than two related proportions. The P value from the χ2 test or its counterparts does not indicate the strength of the difference or association between the categorical variables involved. This information can be obtained from the relative risk or the odds ratio statistic which is measures of dichotomous association obtained from 2 × 2 tables.

Empirical likelihood-based serial correlation testing in partially varying coefficient single-index models

ABSTRACTPartially varying coefficient single-index models (PVCSIM) are a class of semiparametric regression models. One important assumption is that the model error is independently and identically distributed, which may contradict with the reality in many applications. For example, in the economical and financial applications, the observations may be serially correlated over time. Based on the empirical likelihood technique, we propose a procedure for testing the serial correlation of random error in PVCSIM. Under some regular conditions, we show that the proposed empirical likelihood ratio statistic asymptotically follows a standard χ2 distribution. We also present some numerical studies to illustrate the performance of our proposed testing procedure.

Stochastic unit root processes: Maximum likelihood estimation, and new Lagrange multiplier and likelihood ratio tests

We show in this study that the maximum likelihood estimators of stochastic unit root (STUR) processes are consistent and asymptotically normally distributed. We also present two new tests for STUR. We first propose a Lagrange multiplier test and show that it has a standard χ2 distribution asymptotically. We also propose a likelihood ratio test and show that it has an asymptotic distribution of 50–50 mixture of χ2 and a point mass at 0. As an empirical example, we test the existence of STUR in the Canadian real exchange rate and explore the implication of STUR on the validity of purchasing power parity.

New confidence interval estimator of the signal-to-noise ratio based on asymptotic sampling distribution

ABSTRACTIn this paper, the asymptotic distribution of the signal-to-noise ratio (SNR) is derived and a new confidence interval for the SNR is introduced. An evaluation of the performance of the new interval compared to Sharma and Krishna (S–K) (1994) confidence interval for the SNR using Monte Carlo simulations is conducted. Data were randomly generated from normal, log-normal, χ2, Gamma, and Weibull distributions. Simulations revealed that the performance of S–K interval is totally dependent on the amount of noise introduced and that it has a constant width for a given sample size. The S–K interval performs poorly in four of the distributions unless the SNR is around one. It is recommended against using the S–K interval for data from log-normal distribution even with SNR = 1. Unlike the S–K interval which does not account for skewness and kurtosis of the distribution, the new confidence interval for the SNR outperforms S–K for all five distributions discussed, especially when SNR ⩾ 2. The proposed ranked set sampling (RSS) instead of simple random sampling (SRS) has improved the performance of both intervals as measured by coverage probability.

Generalized information and entanglement entropy, gravitation and holography

A nonextensive statistical mechanics entropy that depends only on the probability distribution is proposed in the framework of superstatistics. It is based on a Γ(χ2) distribution that depends on β and also on pl. The corresponding modified von Neumann entropy is constructed; it is shown that it can also be obtained from a generalized Replica trick. We further demonstrate a generalized H-theorem. Considering the entropy as a function of the temperature and volume, it is possible to generalize the equation of state of an ideal gas. Moreover, following the entropic force formulation a generalized Newton's law is obtained, and following the proposal that the Einstein equations can be deduced from the Clausius law, we discuss on the structure that a generalized Einstein's theory would have. Lastly, we address the question whether the generalized entanglement entropy can play a role in the gauge/gravity duality. We pay attention to 2d CFT and their gravity duals. The correction terms to the von Neumann entropy result more relevant than the usual UV ones and also than those due to the area dependent AdS3 entropy which result comparable to the UV ones. Then the correction terms due to the new entropy would modify the Ryu–Takayanagi identification between the CFT entanglement entropy and the AdS entropy in a different manner than the UV ones or than the corrections to the AdS3 area dependent entropy.

Association test with the principal component analysis in case-parents studies.

In this report, we proposed a statistic with the principal component analysis (PCA) to test association between multiple markers and the disease-susceptibility gene in case–parents data set. The proposed statistic is based on a difference vector calculated by comparing the genotypes of affected offspring with the hypothetical siblings who carry the parental genotypes not present in the affected offspring. The statistic here focuses on all the principal components and is asymptotically distributed as a χ2 distribution with one degree of freedom. Simulation studies showed that, when the number of markers is not very large, the proposed statistic has higher power than the APRICOT test which is based on the same method of the PCA. The rapid advancement of genotyping technologies and the availability of enormous quantities of genotype or haplotype data provide an unprecedented opportunity for identifying genes underlying complex traits. When multiple markers are available, haplotype-based methods and genotype-based methods are commonly used for conducting association between complex traits and a series of possibly linked markers. Owing to the information from individual markers as well as the linkage disequilibrium (LD) structure between the markers, the haplotype-based association study is considered to be a potentially superior strategy (Akey et al. 2001). However, haplotype-based methods are challenged by a large number of distinct haplotypes, which results in a large number of degrees of freedom, and some haplotypebased methods need estimating haplotype phases only when genotype data at multiple marker loci are collected. On the other hand, genotype-based methods have the advantage of not requiring phase information and in many situations they have higher power than haplotype-based methods (Chapman et al. 2003; Xu et al. 2006; Rakovski et al. 2007; Yu and Wang 2011).

Empirical likelihood for the class of single index hazard regression models

Based on the B spline approximation technique and right censored data, we consider the empirical likelihood inference for the index parameters and its partial components in a class of single index hazard regression models. Under some regular conditions, we show that our proposed empirical likelihood ratio statistics follow the standard χ2 distribution. Some numerical studies are given to illustrate our proposed methodology.

Low energy physical activity recognition system on smartphones.

An innovative approach to physical activity recognition based on the use of discrete variables obtained from accelerometer sensors is presented. The system first performs a discretization process for each variable, which allows efficient recognition of activities performed by users using as little energy as possible. To this end, an innovative discretization and classification technique is presented based on the χ2 distribution. Furthermore, the entire recognition process is executed on the smartphone, which determines not only the activity performed, but also the frequency at which it is carried out. These techniques and the new classification system presented reduce energy consumption caused by the activity monitoring system. The energy saved increases smartphone usage time to more than 27 h without recharging while maintaining accuracy.

Confidence in the Mass Hierarchy Determination from Reactor Neutrino Experiments

In the next decade, a number of experiments will attempt to determine the neutrino mass hierarchy. We will discuss some of the challenges that these experiments will face, focusing on the reactor neutrino experiments at intermediate baselines. Here, for example, even a small nonlinear energy response can severely affect the sensitivity to the hierarchy. Using MC simulations, we will show the probability of successfully determining the hierarchy for some realistic locations of the detector. We will also show that, since the two hierarchies are non-nested hypothesis, the statistic Δχ2 does not follow a one-degree-of-freedom χ2 distribution and hence the sensitivity to the hierarchy cannot be estimated by taking the square root of the expected Δχ2; we will present the correct formula for the sensitivity.

PBOOST: a GPU-based tool for parallel permutation tests in genome-wide association studies.

The importance of testing associations allowing for interactions has been demonstrated by Marchini et al. (2005). A fast method detecting associations allowing for interactions has been proposed by Wan et al. (2010a). The method is based on likelihood ratio test with the assumption that the statistic follows the χ(2) distribution. Many single nucleotide polymorphism (SNP) pairs with significant associations allowing for interactions have been detected using their method. However, the assumption of χ(2) test requires the expected values in each cell of the contingency table to be at least five. This assumption is violated in some identified SNP pairs. In this case, likelihood ratio test may not be applicable any more. Permutation test is an ideal approach to checking the P-values calculated in likelihood ratio test because of its non-parametric nature. The P-values of SNP pairs having significant associations with disease are always extremely small. Thus, we need a huge number of permutations to achieve correspondingly high resolution for the P-values. In order to investigate whether the P-values from likelihood ratio tests are reliable, a fast permutation tool to accomplish large number of permutations is desirable. We developed a permutation tool named PBOOST. It is based on GPU with highly reliable P-value estimation. By using simulation data, we found that the P-values from likelihood ratio tests will have relative error of >100% when 50% cells in the contingency table have expected count less than five or when there is zero expected count in any of the contingency table cells. In terms of speed, PBOOST completed 10(7) permutations for a single SNP pair from the Wellcome Trust Case Control Consortium (WTCCC) genome data (Wellcome Trust Case Control Consortium, 2007) within 1 min on a single Nvidia Tesla M2090 device, while it took 60 min in a single CPU Intel Xeon E5-2650 to finish the same task. More importantly, when simultaneously testing 256 SNP pairs for 10(7) permutations, our tool took only 5 min, while the CPU program took 10 h. By permuting on a GPU cluster consisting of 40 nodes, we completed 10(12) permutations for all 280 SNP pairs reported with P-values smaller than 1.6 × 10⁻¹² in the WTCCC datasets in 1 week. The source code and sample data are available at http://bioinformatics.ust.hk/PBOOST.zip. gyang@ust.hk; eeyu@ust.hk Supplementary data are available at Bioinformatics online.