Small Standard Error Of Measurement Research Articles

Reliability and validity of the fulltrack AI application to determine cricket bowling line and length compared to 3D motion capture

ABSTRACT This study examined reliability and validity of the Fulltrack AI application to identify cricket ball landing position (line, length). Nine hundred and thirty-two deliveries were compared to 3D motion capture, the criterion measure, with 836 included in analysis (516 bowled (pace = 420, spin = 96), 320 SidearmTM; 301 facing a batter). Agreement analysis indicated an intraclass correlation coefficient of >0.96 for raw and filter 3D line and length data, compared to Fulltrack AI. The coefficient of variation was acceptable for length (<10%) and larger for line (23.82%), albeit with a smaller standard error of measurement (SEM = 0.05 m), improving with outliers removed. Bland−Altman plots confirmed good statistical agreement between devices, with limits of agreement largely within maximal allowable difference values. There are potential practical application considerations, given SEM = 0.47 m for length (diameter of seven cricket balls); with greater variability detecting length closer to the batters-end, and line closer to the bowlers-end. Validity, using a generalised additive model, showed no significant differences between devices (p > 0.05), with no condition-based interaction effects. The Fulltrack AI application enables ecologically valid assessment of bowling performance. Considering the trade-off between this and the accuracy of information is warranted when deciding how best to apply it to coaching environments to support augmented feedback.

Test-retest reliability and agreement of remote home-based functional capacity self-administered assessments in community-dwelling, socially isolated older adults.

To determine the test-retest reliability and agreement of home-based functional capacity self-administered assessments in socially isolated older adults. Fourteen community-dwelling older adults (eight females, 67.9 ± 7.7 years) volunteered for this study. Before testing, participants were screened online for eligibility and received instructional videos explaining test set-up and execution. Participants underwent the 30-second sit-to-standing test, gait speed tests at the usual pace, and timed-up-and-go tests administered 4 weeks apart. For the 30-second sit-to-standing protocol, participants were instructed to repeatedly sit and stand from a chair (with a height of ∼ 43 cm and without armrests) for 30 s, with the number of repetitions recorded. In the gait speed test protocol, participants were instructed to walk at their usual and comfortable pace, with the time taken recorded (seconds). In the timed-up-and-go, participants stood up from a chair, walked as fast as possible for 3 m, circled a marked point, and returned to the chair to sit down, completing the test, with the score recorded (seconds). A trained researcher conducted the scoring virtually via synchronous video. 30-second sit-to-standing, gait speed test, and timed-up-and-go showed excellent mean coefficient of variation values (2.0-4.9), small standard error of measurement (0.08-1.27), excellent intraclass coefficient (0.97-0.99), very strong correlations (0.97-0.99) and good agreement between the two days of testing. Self-administered functional capacity tests performed by older adults at home were reliable with good agreement. Healthcare professionals and older adults should take advantage of simple remote self-administered assessments performed at home to evaluate older adults' health status.

Cross-cultural adaptation and test-retest reliability of the Italian version of the Pediatric Balance Scale in children with typical and atypical development.

Balance disorders in children are complex and disabling. The pediatric balance scale assesses functional balance in the contest of everyday tasks. It is recommended for balance assessment in children. The cross-cultural validation of an assessment instrument allows to assess individuals in the appropriate cultural context. We designed our study with the aim of translating and cross-cultural adapting the Pediatric Balance Scale into Italian, and investigating the reliability of the translated version. Two forward translation of the Pediatric Balance Scale were conducted. Two blind backward translations were subsequently performed. A multidisciplinary group compared the content of the translations with those of the original Pediatric Balance Scale. Ambiguities and discrepancies were amended. To assess conceptual equivalence of the translation, we conducted cognitive debriefing involving physiotherapists, parents, and children. Suggestions for rewording were sought and considered for eventual modification. The Italian Pediatric Balance Scale was administered to 18 children with typical development (age 4.7-7.9 years, female 61.1%, scores 51-56), and to 18 children with atypical development (age 4.9-15.0 years, female 44.4%, scores 38-56). Test-retest reliability (i.e., intrarater and interrater reliability, and agreement between raters) of the scale was assessed by mean of single-rating, absolute-agreement, two-way mixed effects model intra-class correlation coefficients (ICC<inf>3,1</inf>) and the Bland-Altman method. The standard error of measurement and the minimal detectable change were subsequently computed. The forward and backward translations showed no substantial differences in content; wording discrepancies were resolved by consensus. The multidisciplinary group proposed some minor changes. Twenty-three physiotherapists, 36 parents, and 36 children participated in cognitive interviewing and further changes were made. Perfect agreement was observed in test-retest reliability assessment of the Italian pediatric balance scale in children with typical development. High reliability (intrarater: ICC<inf>3,1</inf> 0.998, 95% CI 0.994 to 0.999; interrater: ICC<inf>3,1</inf> 0.994, 95% CI 0.984 to 0.998) and agreement among raters (Bland-Altman plots: 89%, 95% CI 67% to 97%, of the data within the 95% CI limits of the bias estimate) were found assessing children with atypical development. Very small standard error of measurement and the minimal detectable change values were observed (intrarater: 0.212 and 0.588; interrater: 0.368 and 1.019). The present study provides a translated and cross-culturally adapted Italian version of the Pediatric Balance Scale. The full cross-cultural validity of the tool requires further steps to complete the psychometric testing.

The Common Instrument: an assessment to measure and communicate youth science engagement in out-of-school time

ABSTRACT There has been a growing need felt by practitioners, researchers, and evaluators to obtain a common measure of science engagement that can be used in different out-of-school time (OST) science learning settings. We report on the development and validation of a novel 10-item self-report instrument designed to measure, communicate, and ultimately help promote science engagement among youth: the Common Instrument (CI). When administered to 7,521 elementary and middle school students participating in OST science programming, CI items were found to have good psychometric properties – showing strong item discrimination and a range of difficulties, little difference in item functioning by grade-level, gender, or race/ethnicity, good unidimensionality, stability over time, and a small standard error of measurement over a large variety of science engagements. Given its properties, the CI is a reasonable way to collect data on science engagement in a wide range of OST science programs. Common measures, like the CI, that are reliable and valid provide a common language that enables programs to describe their strengths and challenges and make decisions about adapting and improving their approaches. Common measures are also essential for collective impact initiatives that need brief, easy-to-administer instruments to assess progress and impact of their change-making processes.

Cross-cultural adaptation, validity and reproducibility of the Back Beliefs Questionnaire among older Brazilians with acute low back pain. A cross-sectional study.

Low back pain (LBP) has emerging as an epidemic, multifactorial and multidimensional condition in older age. Assessment of attitudes and beliefs of patients with back pain is necessary for understanding the impact of psychosocial factors on pain perception and management. To cross-culturally adapt and examine the validity and reproducibility (intra and interrater reliability and agreement) of the Back Beliefs Questionnaire (BBQ) in older Brazilians with acute LBP. Cross-sectional methodological report conducted at the Department of Physical Therapy of the Universidade Federal de Minas Gerais, Belo Horizonte, Brazil. The present study was conducted for translating, adapting, and examining the psychometric properties of a questionnaire. Participants aged ≥ 60 years experiencing an acute episode of LBP were recruited. Coefficients of internal consistency, reliability and agreement were obtained using Cronbach's α, intraclass correlations, and standard error of measurement and the smallest detectable change, respectively. Twenty-six participants aged between 60-84 years and reporting a mean of 9.8 (4.3) years of schooling completed the study. The Brazilian Portuguese-language version of the BBQ (BBQ-Brazil) was proposed and presented with adequate conceptual, semantic, operational, and measurement equivalence from the original version. Intra and interrater evaluations showed moderate (0.74) and excellent (0.91) intraclass correlation coefficients, respectively, with small standard error of measurement for both evaluations. Internal consistency was considered adequate (0.70). BBQ-Brazil had consistent measurements of validity and reproducibility, and proved to be a valuable tool in clinical practice for addressing attitudes and beliefs of older patients with acute LBP.

Natural variability of daily physical activity measured by accelerometry in children with a mitochondrial disease

In this exploratory study we aimed to select the most valid and feasible accelerometer to measure daily physical activity at home in 10 children with mitochondrial disease. Using the experimentally-selected GENEActiv, good to excellent short- and long-term test-retest reliability of daily physical activity was found. Especially in children with more severe limitations daily physical activity seemed more stable and less susceptible to weather conditions. Moreover, small standard errors of measurement were found, indicating high precision of this measuring method. We conclude that measuring daily physical activity using accelerometry is a promising outcome measure for future studies in this unique population.

15-White Dots APP-Coo-Test: a reliable touch-screen application for assessing upper limb movement impairment in patients with cerebellar ataxias.

The use of objective measurements is essential to assess disease progression and to evaluate the effectiveness of rehabilitation protocols and clinical treatments. The purpose of this study was to develop a touch-screen application, that we named 15-White Dots APP-Coo-Test (15-WDACT), able to carry out quantitative and objective measurements of the rapid and coordinated upper limb movements, typically impaired in patients with cerebellar ataxias (CA). A total of 87 CA patients and 170 healthy subjects participated in this study. The subject was asked to touch with their index finger a white dot, appearing consecutively and randomly on the screen at different positions, for a total of 15 dots per session. The score is the execution time of a single session. 15-WDACT measurements have highly correlated with the scores obtained with the Scale for the Assessment and Rating of Ataxia (SARA), with the Composite Cerebellar Functional Severity (CCFS) and with the measurements obtained using two validated evaluating systems, i.e., the Nine Hole Pegboard test (9HPT) and the Click Test. We also observed high internal consistency and an excellent intra-rater and test-retest reliability. We found a small Standard Error of Measurement (SEM) and an excellent Minimal Detectable Change (MDC), indicating that even small variations in the 15-WDACT measurements are to be associated with real changes in performance. We have concluded that 15-WDACT is an easy, fast and reliable tool to assess the severity of the upper limb ataxia in patients with CA.

Measurement of anterior translation of the mandibular condyle using sonography.

[Purpose] To establish a standardized sonographic approach to quantify anterior translation of the mandibular condyle during mouth opening by examining the reliability of image acquisition and processing, and to understand how anterior condylar translation contributes to mouth opening. [Participants and Methods] Twenty-eight participants without temporomandibular disorder (TMD) participated. During day 1 of data collection, all participants performed maximal mouth opening while an examiner recorded anterior condylar translation using sonography. The mouth opening range of motion was also obtained. On day 2 of data collection, the same procedure was performed on 6 participants that participated in day 1 of data collection. To establish reliability of image processing, 3 examiners measured condylar translations on 2 separate days. To determine reliability of image acquisition, images obtained from 2 days of data collection were analyzed. [Results] Excellent intra-class correlation coefficients (ICCs) and small standard errors of measurement (SEMs) for image acquisition and processing were shown. A significant, linear model was found to describe the relationship between condylar anterior translations and mouth opening. [Conclusion] Anterior condylar translation during mouth opening can be reliably measured using sonography. The linear relation between condylar motion and mouth opening can be used to guide clinical practices.

Validity and Reliability of an Inertial Measurement Unit–based 3-Dimensional Angular Measurement of Cervical Range of Motion

ObjectiveThis study aimed to assess the reliability and validity of an inertial measurement unit (IMU)–based 3-dimensional (3D) angular measurement system for evaluating cervical range of motion. MethodsThirty-three healthy participants (21.9 ± 2.1 years; 162.0 ± 6.0 cm; 55.8 ± 9.0 kg; 21.2 ± 2.4 kg/m2) were evaluated. Kinematic data of the cervical joints were simultaneously obtained using the IMU 3D angular, goniometer, and photographic measurements during cervical flexion (0°, 30°, and 50°), extension (30°, 50°), side-bending (0°, 20°, 40°), and rotation (45°). Test–retest reliability was investigated in each measurement method. Concurrent validity was assessed with the direct comparison between the IMU 3D angular measurement and other methods. ResultsThe IMU 3D angular measurement showed mostly good to high test–retest reliability with relatively small standard error of measurement and the minimal detectable change values. The concurrent validity of IMU 3D angular measurements in the cervical range of motion was mostly reasonable. However, the measurement bias between the 2 methods tended to be larger at the end range of each plane. ConclusionUsing the IMU 3D angular measurement in cervical spine is recommended because of its mostly good to high reliability and reasonable validity. However, using the IMU 3D angular measurement at the end range of each plane should be carefully considered owing to the poorer validity.

THE DEVELOPMENT OF MULTIPLE CHOICE ACHIEVEMENT TESTS FOR EVALUATION OF GEOGRAPHY LEARNING OUTCOMES STUDENTS OF SECOND GRADE AT SENIOR HIGH SCHOOL

The ability of teachers who are still minimal in developing achievement tests with appropriate procedures results in tests that are of poor quality, ultimately impacting measurement results that are less reliable. This encourages researchers to develop achievement test instruments that are in accordance with the correct test development procedures. This study aims to develop a multiplechoice test that is valid and reliable, as well as practical to be used by evaluators as well as by evaluation target objects. The method used in this study is research and development methods, using the Instructional Development Institute model (IDI) with three stages of development namely 1) define, 2) develop and 3) evaluate. Product testing is limited to 38 students. The research data is obtained in quantitative form in the form of student responses to tests, expert validation and practical questionnaires by teachers and students. Data analysis techniques use classical theory, test validity tests, test reliability tests, and practical tests. The results of the study are in the form of products of multiple choice achievement test instruments. Based on the results of the tests, the test instruments that are developed had good grain quality with a moderate level of difficulty and high exposure power, the results of the measurements are valid and reliable or reliable and had a small standard error of measurement which is 2.82, which meant that the measurement results are accurate.

The clinical significance of 10-m walk test standardizations in Parkinson\u2019s disease

BackgroundThe 10-m walk test (10MWT) is a widely used measure of gait speed in Parkinson’s disease (PD). However, it is unclear if different standardizations of its conduct impact test results.Aim of the studyWe examined the clinical significance of two aspects of the standardization of the 10MWT in mild PD: static vs. dynamic start, and a single vs. repeated trials. Implications for fall prediction were also explored.Methods151 people with PD (mean age and PD duration, 68 and 4 years, respectively) completed the 10MWT in comfortable gait speed with static and dynamic start (two trials each), and gait speed (m/s) was recorded. Participants then registered all prospective falls for 6 months.ResultsAbsolute mean differences between outcomes from the various test conditions ranged between 0.016 and 0.040 m/s (effect sizes, 0.06–0.14) with high levels of agreement (intra-class correlation coefficients, 0.932–0.987) and small standard errors of measurement (0.032–0.076 m/s). Receiver operating characteristic curves showed similar discriminate abilities for prediction of future falls across conditions (areas under curves, 0.70–0.73). Cut-off points were estimated at 1.1–1.2 m/s.ConclusionsDifferent 10MWT standardizations yield very similar results, suggesting that there is no practical need for an acceleration distance or repeated trials when conducting this test in mild PD.

Reliability, validity, and minimal detectable change of the push-off test scores in assessing upper extremity weight-bearing ability

Study DesignClinical measurement study. IntroductionThe push-off test (POT) was recently conceived and found to be reliable and valid for assessing weight bearing through injured wrist or elbow. However, further research with larger sample can lend credence to the preliminary findings supporting the use of the POT. Purpose of the StudyThis study examined the interrater reliability, construct validity, and measurement error for the POT in patients with wrist conditions. MethodsParticipants with musculoskeletal (MSK) wrist conditions were recruited. The performance on the POT, grip isometric strength of wrist extensors was assessed. The shortened version of the Disabilities of the Arm, Shoulder and Hand and numeric pain rating scale were completed. The intraclass correlation coefficient assessed interrater reliability of the POT. Pearson correlation coefficients (r) examined the concurrent relationships between the POT and other measures. The standard error of measurement and the minimal detectable change at 90% confidence interval were assessed as measurement error and index of true change for the POT. ResultsA total of 50 participants with different elbow or wrist conditions (age: 48.1 ± 16.6 years) were included in this study. The results of this study strongly supported the interrater reliability (intraclass correlation coefficient: 0.96 and 0.93 for the affected and unaffected sides, respectively) of the POT in patients with wrist MSK conditions. The POT showed convergent relationships with the grip strength on the injured side (r = 0.89) and the wrist extensor strength (r = 0.7). The POT showed smaller standard error of measurement (1.9 kg). The minimal detectable change at 90% confidence interval for the POT was 4.4 kg for the sample. ConclusionsThis study provides additional evidence to support the reliability and validity of the POT. This is the first study that provides the values for the measurement error and true change on the POT scores in patients with wrist MSK conditions. Further research should examine the responsiveness and discriminant validity of the POT in patients with wrist conditions.

Within-day and between-day reliability of thickness measurements of abdominal muscles using ultrasound during abdominal hollowing and bracing maneuvers

Ultrasonography imaging has been used as a non-invasive method to estimate the thickness and relative activities of the abdominal muscles in patients with lower back pain (LBP). However, the statistical reliability of US thickness measurements of abdominal muscles, including transversus abdominis (TrA), internal oblique (IO) and external oblique (EO) muscles during abdominal hollowing (AH) and abdominal bracing (AB) maneuvers has not been well-investigated. This study was performed on a total of 20 female subjects (10 with LBP and 10 without LBP) in the age range of 25–55 years to assess within-day and between-day reliability of the measurements. US measurements on maneuvers were repeated after two hours for the within-day reliability and after five days for the between-day reliability assessment. High intra-class correlation coefficient (ICC) values (>0.75) for within-day and between-day reliability assessments during AH maneuver were concluded. The ICC values were moderate for reliability assessment during AB. The ICC values for AH were greater than AB both for within- and between-day reliabilities. The small standard error of measurement and minimal detectable change values (0.16–0.78 and 0.44 to 2.15, respectively) were found for both AH and AB. We recommend real-time US imaging as a reliable way of determining the thicknesses of the TrA and IO muscle (and to some extent, EO muscle) for both healthy and LBP patients.

Reliability, Validity, and Minimal Detectable Change of Balance Evaluation Systems Test and Its Short Versions in Older Cancer Survivors: A Pilot Study.

Cancer is primarily a disease of older adults. About 77% of all cancers are diagnosed in persons aged 55 years and older. Cancer and its treatment can cause diverse sequelae impacting body systems underlying balance control. No study has examined the psychometric properties of balance assessment tools in older cancer survivors, presenting a significant challenge in the selection of outcome measures for clinicians treating this fast-growing population. This study aimed to determine the reliability, validity, and minimal detectable change (MDC) of the Balance Evaluation System Test (BESTest), Mini-Balance Evaluation Systems Test (Mini-BESTest), and Brief-Balance Evaluation Systems Test (Brief-BESTest) in community-dwelling older cancer survivors. This study was a cross-sectional design. Twenty breast and 8 prostate cancer survivors participated [age (SD) = 68.4 (8.13) years]. The BESTest and Activity-specific Balance Confidence (ABC) Scale were administered during the first session. Scores of Mini-BESTest and Brief-BESTest were extracted on the basis of the scores of BESTest. The BESTest was repeated within 1 to 2 weeks by the same rater to determine the test-retest reliability. For the analysis of the inter-rater reliability, 21 participants were randomly selected to be evaluated by 2 raters. A primary rater administered the test. The 2 raters independently and concurrently scored the performance of the participants. Each rater recorded the ratings separately on the scoring sheet. No discussion among the raters was allowed throughout the testing. Intraclass correlation coefficients (ICCs), standard error of measurement, minimal detectable change (MDC), and Bland-Altman plots were calculated. Concurrent validity of these balance tests with the ABC Scale was examined using the Spearman correlation. The BESTest, Mini-BESTest, and Brief-BESTest had high test-retest (ICC = 0.90-0.94) and interrater reliability (ICC = 0.86-0.96), small standard error of measurement (0.86-2.47 points), and MDC (2.39-6.86 points). The Bland-Altman plot revealed no systematic errors. The scores of BESTest, Mini-BEST, and Brief-BEST were correlated significantly with those of ABC Scale (P < .01), supporting their concurrent validity. The BESTest, Mini-BESTest, and Brief-BESTest showed high interrater and test-retest reliability, and excellent concurrent validity with the ABC Scale for community-dwelling cancer survivors aged 55 years and older who had completed cancer treatments for at least 3 months. Future studies are necessary to determine the predictive values for determining fall risks using balance assessment tools in older cancer survivors. Clinicians can utilize the BESTest and its short versions to evaluate balance problems in community-dwelling older cancer survivors and apply the established MDC to assess the intervention outcomes.

New version of the Pediatric Evaluation of Disability Inventory (PEDI-CAT): translation, cultural adaptation to Brazil and analyses of psychometric properties.

ABSTRACTBackgroundThe Pediatric Evaluation of Disability Inventory-Computer Adaptive Test (PEDI-CAT), developed with innovative measurement methodologies, evaluates functioning of children and youth, from 0 to 21 years, with different health conditions. It is a revision of an earlier instrument (PEDI) that has been used in national and international clinical practice and research. It was felt to be necessary to make this new version (PEDI-CAT) available in Brazil.ObjectivesTranslate and culturally adapt the PEDI-CAT to the Brazilian-Portuguese language and test its psychometric properties.MethodThis methodological study was developed through the following stages: (1) translation, (2) synthesis, (3) back-translation, (4) revision by an expert committee, (5) testing of the pre-final version, and (6) evaluation of the psychometric properties. The 276 translated PEDI-CAT items were divided into three age groups (0-7, 8-14, and 15-21 years).ResultsThe PEDI-CAT translation followed all six stages. The adaptations incorporated cultural and socioeconomic class specificities. The PEDI-CAT/Brazil showed good indices of inter-examiner (intraclass correlation coefficient-ICC=0.83-0.89) and test-retest (ICC=0.96-0.97) reliability, good internal consistency (0.99) and small standard error of measurement in all three age groups (0.12-0.17). Factor analyses grouped the items from the three functional skills domains into one factor, and items from the responsibility scale into three factors, supporting the adequacy of these factor solutions to the conceptual structure of the instrument and the developmental model.ConclusionThe PEDI-CAT/Brazil is a theoretically consistent, culturally appropriate, and reliable instrument. Its availability in Brazil will contribute to the evaluation and measurement of functional outcomes from clinical interventions, longitudinal follow-up, and rehabilitation research.

SAT0422 Increased Supraspinatus Thickness with Reduced Acromohumeral Distance in People with Mild and Moderate Rotator Cuff Tendinopathy: A Reliability and Comparative Study

BackgroundSubacromial impingement syndrome has historically been ascribed to extrinsic compression of the supraspinatus tendon (SsT) due to altered acromial morphology. More recently intrinsic tendon pathology has been proposed as an...

Improving progress test score estimation using Bayesian statistics

Progress tests give a continuous measure of a student's growth in knowledge. However, the result at each test instance is subject to measurement error from a variety of sources. Previous tests contain useful information that might be used to reduce this error. A Bayesian statistical approach to using this prior information was investigated. We first developed a Bayesian model that used the result from only one preceding test to update both the current estimated test score and its standard error of measurement (SEM). This was then extended to include results from all previous tests. The Bayesian model leads to an exponentially weighted combination of test scores. The results show smoothing of test scores when all previous tests are included in the model. The effective sample size is doubled, leading to a 30% reduction in measurement error. A Bayesian approach can give improved score estimates and smaller SEMs. The method is simple to use with large cohorts of students and frequent tests. The smoothing of raw scores should give greater consistency in rank ordering of students and hence should better identify both high-performing students and those in need of remediation.

Validation of the Spanish SIRS With Monolingual Hispanic Outpatients

Psychologists are faced with formidable challenges in making their assessment methods relevant to growing numbers of Hispanic clients for whom English is not the primary or preferred language. Among other clinical issues, the determination of malingering has profound consequences for clients. In this investigation, we evaluated a Spanish translation of the Structured Interview of Reported Symptoms (SIRS; Rogers, Bagby, & Dickens, 1992) with 80 Spanish-speaking Hispanic American outpatients. Using a between-subjects simulation design, the Spanish SIRS was found to produce reliable results with small standard errors of measurement. Regarding validity, very large effect sizes (mean Cohen’s d= 2.00) were observed between feigners and honest responders for the SIRS primary scales. We consider the potential role of the Spanish SIRS with reference to Spanish translations for other assessment instruments.

The standard error of measurement is a more appropriate measure of quality for postgraduate medical assessments than is reliability: an analysis of MRCP(UK) examinations

BackgroundCronbach's alpha is widely used as the preferred index of reliability for medical postgraduate examinations. A value of 0.8-0.9 is seen by providers and regulators alike as an adequate demonstration of acceptable reliability for any assessment. Of the other statistical parameters, Standard Error of Measurement (SEM) is mainly seen as useful only in determining the accuracy of a pass mark. However the alpha coefficient depends both on SEM and on the ability range (standard deviation, SD) of candidates taking an exam. This study investigated the extent to which the necessarily narrower ability range in candidates taking the second of the three part MRCP(UK) diploma examinations, biases assessment of reliability and SEM.Methodsa) The interrelationships of standard deviation (SD), SEM and reliability were investigated in a Monte Carlo simulation of 10,000 candidates taking a postgraduate examination. b) Reliability and SEM were studied in the MRCP(UK) Part 1 and Part 2 Written Examinations from 2002 to 2008. c) Reliability and SEM were studied in eight Specialty Certificate Examinations introduced in 2008-9.ResultsThe Monte Carlo simulation showed, as expected, that restricting the range of an assessment only to those who had already passed it, dramatically reduced the reliability but did not affect the SEM of a simulated assessment. The analysis of the MRCP(UK) Part 1 and Part 2 written examinations showed that the MRCP(UK) Part 2 written examination had a lower reliability than the Part 1 examination, but, despite that lower reliability, the Part 2 examination also had a smaller SEM (indicating a more accurate assessment). The Specialty Certificate Examinations had small Ns, and as a result, wide variability in their reliabilities, but SEMs were comparable with MRCP(UK) Part 2.ConclusionsAn emphasis upon assessing the quality of assessments primarily in terms of reliability alone can produce a paradoxical and distorted picture, particularly in the situation where a narrower range of candidate ability is an inevitable consequence of being able to take a second part examination only after passing the first part examination. Reliability also shows problems when numbers of candidates in examinations are low and sampling error affects the range of candidate ability. SEM is not subject to such problems; it is therefore a better measure of the quality of an assessment and is recommended for routine use.

The AsTex ®: clinimetric properties of a new tool for evaluating hand sensation following stroke

Objectives: To investigate the clinimetric properties and clinical utility of the AsTex®, a new clinical tool for evaluation of hand sensation following stroke. Design: The AsTex ® was administered on two occasions separated by a week to appraise test—retest reliability, and by three assessors on single occasion to establish inter-rater reliability. Pilot normative values were collected in an age-stratified sample. Clinical utility was evaluated based on ease of administration, ceiling and floor effects, and responsiveness to sensory recovery. Participants: Test—retest (n = 31) and inter-rater (n = 31) reliability and normative values (n = 95) for the AsTex® were established in neurologically normal participants aged 18—85 years. Test—retest reliability was investigated in 22 individuals a mean of 46 months (range 12—125) post stroke and clinical utility was evaluated in an additional 24 subacute stroke participants a mean of 29.4 days (range 12—41) post stroke. Main measure: The AsTex ®. Results: The AsTex ® demonstrated excellent test—retest (intraclass correlation coefficient (ICC) = 0.98, 95% confidence interval (95% CI) = 0.97—0.99) and inter-rater reliability (ICC = 0.81, 95% CI = 0.73—0.87) in neurologically normal participants. Test—retest reliability of the AsTex® in individuals following stroke was excellent (ICC = 0.86, 95% CI = 0.68—0.94). The AsTex® was simple to administer, demonstrated small standard error of measurement (0.14 mm), minimal floor and ceiling effects (12.5% and 8.3%) and excellent responsiveness (standardized response mean = 0.57) in subacute stroke participants. Conclusion: The AsTex ® is a reliable, clinically useful and responsive tool for evaluating hand sensation following stroke.