Random Measurement Error Research Articles

The impact of respondent attentiveness on reliability and validity

ABSTRACTThis study aims to assess whether respondent inattentiveness causes systematic and unsystematic measurement error that influences survey data quality. To determine the impact of (in)attentiveness on the reliability and validity of target measures, we compared respondents from a German online survey (N = 5205) who had passed two attention checks with those who had failed. Our results show that inattentiveness induces both random and systematic measurement error, which impacts estimates of the reliability and validity of multi-item scales. In addition, we conducted a sensitivity analysis, which revealed that the impact of inattentiveness on analyses can be substantial.

Variability of native T1 values: implication for defining regional myocardial changes using MRI.

The aim of the present study was to establish T1 variation (T1v) thresholds for duplicated measurements of regional T1 values in left ventricle (LV) using magnetic resonance imaging (MRI). Eighteen healthy volunteers were recruited to undergo two consecutive cardiac MRI scans using modified Look-Locker Inversion recovery (MOLLI) with two spatial resolutions on different days to repeat T1 measurements on LV. The absolute differences (d) and standard deviations (SDs) of regional T1 values were acquired with the two scans and two readers. T1v threshold (mean difference + 2SD), intra-class correlation coefficient (ICC) and coefficient of variation (CoV) were calculated. T1 mapping using the MOLLI sequence (with multiple spatial resolutions) was successfully performed in all 18 volunteers twice. On a per-slice basis, ICCs for intra-observer, inter-observer, inter-resolution and inter-study T1v were 0.988, 0.899, 0.763 and 0.6. CoVs were 0.72, 2.39, 3.90 and 4.28%. T1v thresholds were 22, 66, 118 and 120ms. On a per-segment basis, ICCs for intra-observer, inter-observer, inter-resolution and inter-studyT1v were 0.974, 0.859, 0.711 and 0.594. CoVs were 1.09, 3.36, 4.69 and 5.01%. T1v thresholds were 33, 94, 140 and 144ms. Those thresholds may be useful for discriminating disease-initiated T1v from random errors of T1 measurements.

Determining anisotropic elastic parameters of transversely isotropic rocks through single torsional shear test and theoretical analysis

Deformation properties of sedimentary rock is quite important for evaluating the industrial production of shale gas and the effective design of tunnel constructions, deep rock foundations for nuclear power plant and deep geological disposal. Anisotropic sedimentary rocks, such as shale, can exhibit significantly different material properties in different directions. Knowledge of the anisotropic features of such rocks is, therefore, essential. When a hollow cylinder of an anisotropic material such as a sedimentary rock is torsionally-sheared, the axisymmetry is violated and the stress varies in the rotational direction. Utilizing such stress distributions, we propose a method to determine the parameters of the transversely isotropic elasticity of such a material, including the dominant orientation, via a single torsional shear test using a single hollow cylindrical specimen. A newly designed instrumented cap is also employed. Unlike existing methods, the specimen can be sampled in any arbitrary direction. An overview of the method is presented and its validity is checked both theoretically and numerically. The strain and stress distributions in a hollow cylindrical specimen of a transversely isotropic material is first investigated theoretically, and the stress and strain fields (to which random measurement errors are added) are back-analyzed using the proposed method. The results demonstrate that five elastic parameters, as well as the orientation of the transverse isotropy plane, can be determined uniquely via the proposed method.

Interrater and Intrarater Reliability of the Balance Computerized Adaptive Test in Patients With Stroke

ObjectiveTo examine the interrater and intrarater reliability of the Balance Computerized Adaptive Test (Balance CAT) in patients with chronic stroke having a wide range of balance functions. DesignRepeated assessments design (1wk apart). SettingSeven teaching hospitals. ParticipantsA pooled sample (N=102) including 2 independent groups of outpatients (n=50 for the interrater reliability study; n=52 for the intrarater reliability study) with chronic stroke. InterventionsNot applicable. Main Outcome MeasuresBalance CAT. ResultsFor the interrater reliability study, the values of intraclass correlation coefficient, minimal detectable change (MDC), and percentage of MDC (MDC%) for the Balance CAT were .84, 1.90, and 31.0%, respectively. For the intrarater reliability study, the values of intraclass correlation coefficient, MDC, and MDC% ranged from .89 to .91, from 1.14 to 1.26, and from 17.1% to 18.6%, respectively. ConclusionsThe Balance CAT showed sufficient intrarater reliability in patients with chronic stroke having balance functions ranging from sitting with support to independent walking. Although the Balance CAT may have good interrater reliability, we found substantial random measurement error between different raters. Accordingly, if the Balance CAT is used as an outcome measure in clinical or research settings, same raters are suggested over different time points to ensure reliable assessments.

Estimating the Index of Increase via Balancing Deterministic and Random Data

We introduce and explore an empirical index of increase that works in both deterministic and random environments, thus allowing to assess monotonicity of functions that are prone to random measurement errors. We prove consistency of the index and show how its rate of convergence is influenced by deterministic and random parts of the data. In particular, the obtained results suggest a frequency at which observations should be taken in order to reach any pre-specified level of estimation precision.We illustrate the index using data arising from purely deterministic and error-contaminated functions, which may or may not be monotonic.

The Applicability of Standard Error of Measurement and Minimal Detectable Change to Motor Learning Research-A Behavioral Study.

Motor learning studies face the challenge of differentiating between real changes in performance and random measurement error. While the traditional p-value-based analyses of difference (e.g., t-tests, ANOVAs) provide information on the statistical significance of a reported change in performance scores, they do not inform as to the likely cause or origin of that change, that is, the contribution of both real modifications in performance and random measurement error to the reported change. One way of differentiating between real change and random measurement error is through the utilization of the statistics of standard error of measurement (SEM) and minimal detectable change (MDC). SEM is estimated from the standard deviation of a sample of scores at baseline and a test–retest reliability index of the measurement instrument or test employed. MDC, in turn, is estimated from SEM and a degree of confidence, usually 95%. The MDC value might be regarded as the minimum amount of change that needs to be observed for it to be considered a real change, or a change to which the contribution of real modifications in performance is likely to be greater than that of random measurement error. A computer-based motor task was designed to illustrate the applicability of SEM and MDC to motor learning research. Two studies were conducted with healthy participants. Study 1 assessed the test–retest reliability of the task and Study 2 consisted in a typical motor learning study, where participants practiced the task for five consecutive days. In Study 2, the data were analyzed with a traditional p-value-based analysis of difference (ANOVA) and also with SEM and MDC. The findings showed good test–retest reliability for the task and that the p-value-based analysis alone identified statistically significant improvements in performance over time even when the observed changes could in fact have been smaller than the MDC and thereby caused mostly by random measurement error, as opposed to by learning. We suggest therefore that motor learning studies could complement their p-value-based analyses of difference with statistics such as SEM and MDC in order to inform as to the likely cause or origin of any reported changes in performance.

Heat flux and temperature determination in a cylindrical element with the use of Finite Volume Finite Element Method

This paper presents a method to determine the local heat flux and temperature distribution on a thick cylindrical surface through the solution of an inverse heat conduction problem (IHCP). In particular, the method is developed for cases when traditional measurements cannot be performed due to difficulties in accessing the surface; a common situation in engineering and research practice. In particular, inverse methods are sensitive to measurement errors, for example, those caused by a mispositioned temperature sensor on a solid body. Hence, this paper proposes a method that is insensitive to, and can accurately be used, even in presence of these errors. The proposed method lowers the error in the local heat flux and temperature determination hence making the inverse methods more reliable.The proposed method is based on the finite element-finite volume method (FEMFVM) approach. It is tested using measurement data obtained on a laboratory stand. The measurements are performed using thermocouples installed inside a solid cylindrical wall.The method is tested and compared to other numerical methods, including the finite volume method (FVM) and the finite element method (FEM). Extensive additional calculations are carried out to evaluate the impact of random measurement errors on the accuracy of the proposed method.The calculation results prove that the new method is suitable and useful in the case of complex geometrical shapes and enables accurate determination of parameters such as the heat flux, and the outer surface temperature.

Random measurement error: Why worry? An example of cardiovascular risk factors.

With the increased use of data not originally recorded for research, such as routine care data (or ‘big data’), measurement error is bound to become an increasingly relevant problem in medical research. A common view among medical researchers on the influence of random measurement error (i.e. classical measurement error) is that its presence leads to some degree of systematic underestimation of studied exposure-outcome relations (i.e. attenuation of the effect estimate). For the common situation where the analysis involves at least one exposure and one confounder, we demonstrate that the direction of effect of random measurement error on the estimated exposure-outcome relations can be difficult to anticipate. Using three example studies on cardiovascular risk factors, we illustrate that random measurement error in the exposure and/or confounder can lead to underestimation as well as overestimation of exposure-outcome relations. We therefore advise medical researchers to refrain from making claims about the direction of effect of measurement error in their manuscripts, unless the appropriate inferential tools are used to study or alleviate the impact of measurement error from the analysis.

Comparison of experimental methods for estimating matrix diffusion coefficients for contaminant transport modeling

Diffusion cell and diffusion wafer experiments were conducted to compare methods for estimating effective matrix diffusion coefficients in rock core samples from Pahute Mesa at the Nevada Nuclear Security Site (NNSS). A diffusion wafer method, in which a solute diffuses out of a rock matrix that is pre-saturated with water containing the solute, is presented as a simpler alternative to the traditional through-diffusion (diffusion cell) method. Both methods yielded estimates of effective matrix diffusion coefficients that were within the range of values previously reported for NNSS volcanic rocks. The difference between the estimates of the two methods ranged from 14 to 30%, and there was no systematic high or low bias of one method relative to the other. From a transport modeling perspective, these differences are relatively minor when one considers that other variables (e.g., fracture apertures, fracture spacings) influence matrix diffusion to a greater degree and tend to have greater uncertainty than effective matrix diffusion coefficients. For the same relative random errors in concentration measurements, the diffusion cell method yields effective matrix diffusion coefficient estimates that have less uncertainty than the wafer method. However, the wafer method is easier and less costly to implement and yields estimates more quickly, thus allowing a greater number of samples to be analyzed for the same cost and time. Given the relatively good agreement between the methods, and the lack of any apparent bias between the methods, the diffusion wafer method appears to offer advantages over the diffusion cell method if better statistical representation of a given set of rock samples is desired.

Modal Analysis, Metrology, and Error Budgeting of a Precision Motion Stage

In this study, a precision motion stage, whose design utilizes a single shaft supported from the bottom by an air bearing and voice coil actuators in complementary double configuration, is evaluated for its dynamic properties, motion accuracy, and potential machining force response, through modal testing, laser interferometric metrology, and spectral analysis, respectively. Modal testing is carried out using two independent methods, which are both based on impact hammer testing. Results are compared with each other and with the predicted natural frequencies based on design calculations. Laser interferometry has been used with varying optics to measure the geometric errors of motion. Laser interferometry results are merged with measured servo errors, estimated thermal errors, and the predicted dynamic response to machining forces, to compile the error budget. Overall accuracy of the stage is calculated as peak-to-valley 5.7 μm with a 2.3 μm non-repeatable part. The accuracy measured is in line with design calculations which incorporated the accuracy grade of the encoder scale and the dimensional tolerances of structural components. The source of the non-repeatable errors remains mostly equivocal, as they fall in the range of random errors of measurement in laser interferometry like alterations of the laser wavelength due to air turbulence.

‘Phantom’ compositional effects in English school value-added measures: the consequences of random baseline measurement error

A compositional effect is when pupil attainment is associated with the characteristics of their peers, over and above their own individual characteristics. Pupils at academically selective schools, for example, tend to out-perform similar-ability pupils who are educated with mixed-ability peers. Previous methodological studies however have shown that ‘phantom’ compositional effects can arise purely from measurement error. Through simulations using English National Pupil Database data, this paper demonstrates that moderate rates of measurement error in baseline scores can produce a systemic bias in value-added (‘Progress’) scores that favours higher ability pupils. This bias is large enough to explain the ostensibly superior performance of selective ‘grammar’ schools that is seen in the English data and create an appreciable bias across the school performance distribution. This paper describes how such biases can arise from ‘regression attenuation’ and discusses the more general methodological implications of the results. The paper concludes that the consequences of baseline measurement error on school value-added scores is greatly mitigated by controls for school-level prior attainment, but that the English ‘Progress’ value-added measures, as they do not control for prior attainment at school-level, are seriously biased and that the observed ‘grammar school effect’ is likely to be largely or wholly spurious.

Non-destructive testing based on vibrations in the low to mid-frequency range

Recently, it has been shown that a Time Reversal MUltiple Signal Classification (TR-MUSIC) algorithm can be employed to detect defects in samples, which are both challenging in terms of material and in terms of geometrical complexity. This can be achieved by lowering the detection frequency as compared to most other TR and TR-MUSIC applications. In this case, the method operates in a low to mid-frequency range, where accurate models are still realizable. The method relies on the measurement of the frequency response function (FRF) between multiple excitation and sensor locations. These are gathered in a matrix, which then is decomposed in its singular vectors, serving as input for a MUSIC algorithm. In order to improve the applicability of this method, it is shown how to adapt the algorithm, in order to reduce the number of excitation locations or the number of sensors significantly. This results in a considerable speed-up for the application of this non-destructive testing strategy. Furthermore, it is investigated numerically how the robustness of the detection result under the influence of random measurement errors behaves if the number of excitation/sensor locations is reduced. Additionally, the dependency between, the decrease of the number of excitation/sensor locations and the types of defects, which can be detected is studied.

Methodological characteristics of fiber-optical position sensors

The questions of normalized values control of fiber-optic strain transducers output parameters, position and displacements are considered. Knowledge of the normalized values of these parameters makes it possible to correct the results and thereby increase the accuracy (reduce the uncertainty) of the measurements.The main parameters characterizing fiber-optic sensors are: the conversion coefficient of the rotation angle / displacement into the output signal and the zero drift. The drift caused mainly by the shot noise of the photo detectors, and the dispersion of the conversion coefficients is the possible boundary value of the sensors effective application, i.e. their accuracy. In this case, for each manufactured batch of products it is necessary to note the certified value and the margin of error and / or the expanded uncertainty of this value.Studying the set of sensors under repeatability conditions, the dispersion of the results is due only to the influence of a random variable-the drift of the zero line. The variance of the overall average of the results of m-fold measurements of the characteristics of n selected sensor instances will also depend on the dispersion of the characteristics of the sensors array. For a separate estimation of the influence of the random measurement error and the sensors inhomogeneity one-way dispersion analysis is used on the basis of which an estimate of the variance between the sensors in the manufactured batch of products is obtained. On the basis of this variance estimate and average value, the upper and lower limit values are set in which contain the normalized parameter with the probability of 0,95.

IMPROVEMENT OF QUALITY OF THE EVALUATION OF MODEL PARAMETERS BY INTEGRATED METHOD OF LEAST SQUARES

Modern systems of management, optimization and forecasting of production require new algorithms of work and evaluation of systems and processes. When solving problems, in the absence of a priori information, there is a need for the use of effective methods of parametric identification. The accuracy of the existing classical methods of identification depends on the availability of certain information regarding the characteristics of the signals, in particular, on the law of the distribution of random error of measurement, and therefore in the real processes are ineffective. There is a need to apply methods that provide more accurate estimates of the parameters of a mathematical model of the object under study in time-limited and non-sensitive data, about noise variables and control impacts. The estimation of parameters is carried out using the integrated method of least squares, which provides smoothing of the external influences of the model under study on the results. The effectiveness of the method under consideration is confirmed by comparison with the least squares method. The optimization of the parameters of the weight function according to the external criterion has been made, at least the norm of the difference between the estimates of the parameters of the pair and odd sequences. The analysis of the dependence of the accuracy of the estimations of parameters on the choice of coefficients of the weight function is carried out.

Test-Retest Reliability and Minimal Detectable Change of the D2 Test of Attention in Patients with Schizophrenia.

The d2 Test of Attention (D2) is a commonly used measure of selective attention for patients with schizophrenia. However, its test-retest reliability and minimal detectable change (MDC) are unknown in patients with schizophrenia, limiting its utility in both clinical and research settings. The aim of the present study was to examine the test-retest reliability and MDC of the D2 in patients with schizophrenia. A rater administered the D2 on 108 patients with schizophrenia twice at a 1-month interval. Test-retest reliability was determined through the calculation of the intra-class correlation coefficient (ICC). We also carried out Bland-Altman analysis, which included a scatter plot of the differences between test and retest against their mean. Systematic biases were evaluated by use of a paired t-test. The ICCs for the D2 ranged from 0.78 to 0.94. The MDCs (MDC%) of the seven subscores were 102.3 (29.7), 19.4 (85.0), 7.2 (94.6), 21.0 (69.0), 104.0 (33.1), 105.0 (35.8), and 7.8 (47.8), which represented limited-to-acceptable random measurement error. Trends in the Bland-Altman plots of the omissions (E1), commissions (E2), and errors (E) were noted, presenting that the data had heteroscedasticity. According to the results, the D2 had good test-retest reliability, especially in the scores of TN, TN-E, and CP. For the further research, finding a way to improve the administration procedure to reduce random measurement error would be important for the E1, E2, E, and FR subscores.

ACCURACY OF SATELLITE OPTICAL OBSERVATIONS AND PRECISE ORBIT DETERMINATION

The monitoring of low-orbit space objects (LEO-objects) is performed in the Astronomical Observatory of Odessa I.I. Mechnikov National University (Ukraine) for many years. Decades-long arсhives of these observations are accessible within Ukrainian network of optical observers (UMOS). In this work, we give an example of orbit determination for the satellite with the 1500-km height of orbit based on angular observations in our observatory (Int. No. 086). For estimation of the measurement accuracy and accuracy of determination and propagation of satellite position, we analyze the observations of Ajisai satellite with the well-determined orbit. This allows making justified conclusions not only about random errors of separate measurements, but also to analyze the presence of systematic errors, including external ones to the measurement process. We have shown that the accuracy of one measurement has the standard deviation about 1 arcsec across the track and 1.4 arcsec along the track and systematical shifts in measurements of one track do not exceed 0.45 arcsec. Ajisai position in the interval of the orbit fitting is predicted with accuracy better than 30 m along the orbit and better than 10 m across the orbit for any its point.

Interrater Reliability of the Observable Movement Quality Scale for Children.

Purpose: The authors investigated the interrater reliability, the standard deviation of the random measurement error, and the limits of agreement (LoA) of the Observable Movement Quality (OMQ) scale in children. Movement quality is important in the recognition of motor problems, and the OMQ scale, a questionnaire used by paediatric physical therapists, has been developed for use with an age-specific motor test to observe movement quality and score relative to what is expected for a child's age. Method: Paediatric physical therapists (n=28; 2 men, 26 women) observed video-recorded assessments of age-related motor tests in children (n=9) aged 6 months to 6 years and filled in the OMQ scale (possible score range 15-75 points). For our analyses, we used linear mixed models without fixed effects. Results: The interrater reliability was moderate (intra-class correlation coefficient [ICC2,1]: 0.67, 95%CI: 0.47, 0.88); neither work setting nor work experience exerted any influence on it. The standard deviation of the random measurement error was 5.7, and the LoA was 31.5. Item agreement was good (proportion of observed agreement [Po] total 0.82-0.99). Conclusion: The OMQ scale showed moderate interrater reliability when being used by therapists who were unfamiliar with the questionnaire and who had received only 2 hours of training. Feedback from the participants suggested a need for more comprehensive training in using the OMQ scale in clinical practice.

Cramér-Rao Lower Bound for State-Constrained Nonlinear Filtering

This letter presents a mean-square error lower bound for state estimation of nonlinear stochastic systems under given differentiable state constraints. Its recursive formulation permits incorporation of random process and measurement errors and is shown to be a generalization of the known lower bound for unconstrained problems. The bound is evaluated for the example of locating a ground vehicle from noisy measurements of its horizontal position and velocity incorporating a roadmap.

Analysis of Capabilities for Satellite Monitoring of Atmospheric Gaseous Composition Using IRFS-2 Instrument

Capabilities of the total column monitoring of different minor gaseous compounds of the atmosphere with the satellite IRFS-2 Fourier interferometer have been studied. The possibilities of determining the СО2, О3, СH4, HNO3, N2O, CH3OH, HCFC-22, CFC-11, CFC-12, PAN, and ССl4 total columns have been investigated on the basis of line-by-line calculations of the forward problem operator and calculation of error matrices by the optimal estimation method. It has been shown that the IRFS-2 device could be used to measure the total columns of СО2, О3, N2O, СH4, and HNO3. In the information-gathering mode, it is also possible to retrieve the CH3OH, HCFC-22, CFC-11, CFC-12, PAN, and ССl4 total columns due to the suppression of random measurement errors.

A comparison between the original and Tablet-based Symbol Digit Modalities Test in patients with schizophrenia: Test-retest agreement, random measurement error, practice effect, and ecological validity

We aimed to compare the test-retest agreement, random measurement error, practice effect, and ecological validity of the original and Tablet-based Symbol Digit Modalities Test (T-SDMT) over five serial assessments, and to examine the concurrent validity of the T-SDMT in patients with schizophrenia. Sixty patients with chronic schizophrenia completed five serial assessments (one week apart) of the SDMT and T-SDMT and one assessment of the Activities of Daily Living Rating Scale III at the first time point. Both measures showed high test-retest agreement, similar levels of random measurement error over five serial assessments. Moreover, the practice effects of the two measures did not reach a plateau phase after five serial assessments in young and middle-aged participants. Nevertheless, only the practice effect of the T-SDMT became trivial after the first assessment. Like the SDMT, the T-SDMT had good ecological validity. The T-SDMT also had good concurrent validity with the SDMT. In addition, only the T-SDMT had discriminative validity to discriminate processing speed in young and middle-aged participants. Compared to the SDMT, the T-SDMT had overall slightly better psychometric properties, so it can be an alternative measure to the SDMT for assessing processing speed in patients with schizophrenia.