Accounting for the effect of correlations by modulo averaging as part of neural network integration of statistical tests for small samples

Abstract

The Aim of the paper is to demonstrate the advantages of taking into consideration real correlations by means of their symmetrization, which is significantly better than completely ignoring real correlations in cases of statistical estimation using small samples. Methods . Instead of real correlation numbers different in sign and modulo, identical values of correlation numbers moduli are used. It is shown that the equivalence of transformation to symmetrization is subject to the condition of identical probabilities of errors of the first and second kind for asymmetrical and equivalent symmetrical correlation matrices. The authors examine the procedure of accurate calculation of equal data correlation coefficients by trial and error and procedure of approximate calculation of symmetrical coefficients by averaging the moduli of real correlation numbers of an asymmetrical matrix. Results . The paper notes a practically linear dependence of equal probabilities of errors of the first and second kind from the dimension of the symmetrized problem being solved under logarithmic scale of the variables taken into consideration. That ultimately allows performing the examined calculations in table form using low-bit, low-power, inexpensive microcontrollers. The examined transformations have a quadratic computational complexity and come down to using pre-constructed 8-bit binary tables that associate the expected probability of errors of the first and second kind with the parameter of equal correlation of data. All the table calculations are correct and do not accumulate input data round-off errors. Conclusions . The now widely practiced complete disregard of the correlations when performing statistical analysis is very detrimental. It would be more correct to replace the matrices of real correlation numbers with symmetrical equivalents. The approximation error caused by simple averaging of the moduli of coefficient of asymmetrical matrices decreases as the square of their dimension or the square of the number of neurons that generalize classical statistical tests. When 16 and more neurons are used, the approximation error becomes negligible and can be disregarded.