External Validity Measures Research Articles

B - 42 Evaluating the MMPI-3 Somatic/Cognitive Scales in Service Members and Veterans with Mild Traumatic Brain Injury (mTBI)

Abstract Objective Research on the Minnesota Multiphasic Personality Inventory, 3 (MMPI-3) Somatic/Cognitive scales is limited, but primarily focused on their relationships with symptom and performance validity tests. This study expands prior literature by providing extensive Somatic/Cognitive Scale correlates with symptom validity tests and neurocognitive measures in a sample of active-duty personnel and veterans. Method The study included Active-duty Service Members (n = 22) and Veterans (n = 31). Various measures of cognitive abilities and symptom validity were examined as a function of the Cognitive Complaints (COG) scale. Results Moderate to large differences were observed between COG groups across all MMPI-3 overreporting scales and all somatic/cognitive scales, save the NUC scale (g = 0.66–1.31 [excluding NUC]). Groups differed on three Memory Complaints Inventory memory scales: General (g = 0.89), Visuo-spatial (g = 0.62), and Verbal (g = 0.97). No significant differences were observed across cognitive tests (e.g., WAIS, CVLT). COG was moderately associated with all MMPI-3 overreporting and most somatic/cognitive scales (r = 0.30–0.72). Small to moderate correlations were observed between COG and most external validity and neurocognitive measures (r = 0.05–0.60). Conclusions Results suggest the COG scale primarily captures subjective distress rather than objective neurocognitive performance, consistent with prior work (Gervais et al., 2009). These patterns are especially pronounced in emotional, social, and somatic symptom domains. Additionally, COG scores were associated with MMPI-3 overreporting scores, but COG had weaker associations with external neurocognitive and symptom validity measures. These results suggest that COG has weak convergent validity with other self-report measures of cognition (e.g., Neurobehavioral Symptom Inventory).

From A-to-Z review of clustering validation indices

Data clustering involves identifying latent similarities within a dataset and organizing them into clusters or groups. The outcomes of various clustering algorithms differ as they are susceptible to the intrinsic characteristics of the original dataset, including noise and dimensionality. The effectiveness of such clustering procedures directly impacts the homogeneity of clusters, underscoring the significance of evaluating algorithmic outcomes. Consequently, the assessment of clustering quality presents a significant and complex endeavor. A pivotal aspect affecting clustering validation is the cluster validity metric, which aids in determining the optimal number of clusters. The main goal of this study is to comprehensively review and explain the mathematical operation of internal and external cluster validity indices, but not all, to categorize these indices and to brainstorm suggestions for future advancement of clustering validation research. In addition, we review and evaluate the performance of internal and external clustering validation indices on the most common clustering algorithms, such as the evolutionary clustering algorithm star (ECA*). Finally, we suggest a classification framework for examining the functionality of both internal and external clustering validation measures regarding their ideal values, user-friendliness, responsiveness to input data, and appropriateness across various fields. This classification aids researchers in selecting the appropriate clustering validation measure to suit their specific requirements.

Prediction of acute toxicity for Chlorella vulgaris caused by tire wear particle-derived compounds using quantitative structure-activity relationship models

Tire wear particles (TWPs) enter aquatic ecosystems through various pathways, such as rainwater and urban runoff. Additives in TWPs can harm aquatic organisms in these ecosystems. Therefore, it is essential to investigate their toxicity to aquatic organisms. In our study, we initially recorded the median effective concentrations of 21 TWP-derived compounds on Chlorella vulgaris growth, ranging from 0.04 to 8.60 mg/L. Subsequently, through an extensive review of the literature, we incorporated 112 compounds with specific toxicity endpoints to construct the QSAR model using genetic algorithm and multiple linear regression techniques, followed by the construction of the consensus model and the quantitative read-across structure-activity relationship (q-RASAR) model. Meanwhile, we employed rigorous internal and external validation measures to assess the performance of the model. The results indicated that the developed q-RASAR model exhibited strong adaptation, robustness, and reliable prediction, with q-RASAR indicators of Q2LOO = 0.7673, R2tr = 0.8079, R2test = 0.8610, Q2Fn = 0.8285−0.8614, and CCCtest = 0.9222. Based on an external dataset containing 128 emerging TWP-derived compounds, the model's applicability domain coverage was 90.6 %. The q-RASAR model predicted that the structure of diphenylamine was associated with higher toxicity, possibly liked to the SpMax2_Bhm and LogBCF descriptors. The established model reliably provides prediction and fills a critical data gap. These findings highlight the potential risks posed by emerging TWP-derived compounds to aquatic organisms.

DNA methylation-based estimators of telomere length show low correspondence with paternal age at conception and other measures of external validity of telomere length.

In humans, DNA methylation (DNAm) based estimators of telomere length (TL) have been shown to better predict TL-associated variables (e.g., age, sex, and mortality) than TL itself. The biological significance of DNAm-based estimators of TL (DNAmTL) is unclear. In vitro DNAmTL shortens with cell replications, even when telomerase is maintaining TL. Telomerase is typically suppressed in humans, except in testes. Accordingly, sperm TL increases with age, and offspring with greater paternal age at conception (PAC) have longer TL. Thus, we expect that PAC associations with DNAmTL can shed light on whether in vivo cell replications in the presence of high telomerase activity (production of sperm) shorten DNAmTL or if PAC-lengthened TL causes lengthened DNAmTL. In a pre-registered analysis, using data from 1733 blood samples from the Philippines, we examined the association between paternal age at conception (PAC) and offspring DNAmTL. We did not find an association between PAC and DNAmTL but found a positive association of paternal grandfather's age at father's conception predicting grandchild's DNAmTL. In post hoc analyses, we examined how DNAmTL versus qPCR-measured TL (qPCR-TL) correlated with measures typically associated with TL. Contrary to previous findings, on almost all measures of external validity (correlations with parental TLs, southern blot TL, and age), qPCR-TL outperformed DNAmTL. The "kilobase" units of DNAm-based estimators of TL showed considerable deviations from southern blot-derived kilobase measures. Our findings suggest that DNAmTL is not a reliable index of inherited aspects of TL and underscores uncertainty about the biological meaning of DNAmTL.

Application of clustering strategy for automatic segmentation of tissue regions in mass spectrometry imaging.

Mass spectrometry imaging (MSI) has been widely used in biomedical research fields. Each pixel in MSI consists of a mass spectrum that reflects the molecule feature of the tissue spot. Because MSI contains high-dimensional datasets, it is highly desired to develop computational methods for data mining and constructing tissue segmentation maps. To visualize different tissue regions based on mass spectrum features and improve the efficiency in processing enormous data, we proposed a computational strategy that consists of four procedures including preprocessing, data reduction, clustering, and quantitative validation. In this study, we examined the combination of t-distributed stochastic neighbor embedding (t-SNE) and hierarchical clustering (HC) for MSI data analysis. Using publicly available MSI datasets, one dataset of mouse urinary bladder, and one dataset of human colorectal cancer, we demonstrated that the generated tissue segmentation maps from this combination were superior to other data reduction and clustering algorithms. Using the staining image as a reference, we assessed the performance of clustering algorithms with external and internal clustering validation measures, including purity, adjusted Rand index (ARI), Davies-Bouldin index (DBI), and spatial aggregation index (SAI). The result indicated that SAI delivered excellent performance for automatic segmentation of tissue regions in MSI. We used a clustering algorithm to construct tissue automatic segmentation in MSI datasets. The performance was evaluated by comparing it with the stained image and calculating clustering validation indexes. The results indicated that SAI is important for automatic tissue segmentation in MSI, different from traditional clustering validation measures. Compared to the reports that used internal clustering validation measures such as DBI, our method offers more effective evaluation of clustering results for MSI segmentation. We envision that the proposed automatic image segmentation strategy can facilitate deep learning in molecular feature extraction and biomarker discovery for the biomedical applications of MSI.

한국판 정서·행동평가시스템(K·BASC-3), 교사보고형 검사-아동용(TRS-C)의 타당화 연구

The purpose of this study was to evaluate the validity of the Korean Behavior Assessment System for Children 3rd Edition, Teacher Rating Scales-Child Form (K·BASC-3, TRS-C). The 156-item K·BASC-3, TRS-C was administered to 318 teachers of children. Equating using common item non-equivalent group designs, a general norm was made for a total of 954. The generalized partial credit model based on item response theory analysis was used to conduct internal validity. Additionally, as a measure of external validity, concurrent validity was tested using the Teacher’s Report Form for Ages 6-18 (TRF 6-18). Results from the analysis indicated that items included in clinical subscale provided appropriate information at the ability levels higher than the mean. Also, the results revealed that the most item difficulty and discrimination were appropriate. The items that yielded less than desirable discrimination scores correspond to those measuring symptoms related to the DSM diagnostic categories or protective and risk factors for childhood emotional/behavioral disorders. The anxiety, hyperactivity, and attention problems subscales of the K·BASC-3, TRS-C demonstrated strong concurrent validity statistics with the related subscales of the TRF 6-18. Other relevant findings, practical implications regarding the use of the K·BASC-3, TRS-C, strengths and weaknesses of the current study and suggestions for future research are included in the discussion.

Assessment of the global Copernicus, NASADEM, ASTER and AW3D digital elevation models in Central and Southern Africa

ABSTRACT Validation studies of global Digital Elevation Models (DEMs) in the existing literature are limited by the diversity and spread of landscapes, terrain types considered and sparseness of groundtruth. Moreover, there are knowledge gaps on the accuracy variations in rugged and complex landscapes, and previous studies have often not relied on robust internal and external validation measures. Thus, there is still only partial understanding and limited perspective of the reliability and adequacy of global DEMs for several applications. In this study, we utilize a dense spread of LiDAR groundtruth to assess the vertical accuracies of four medium-resolution, readily available, free-access and global coverage 1 arc-second (30 m) DEMs: NASADEM, ASTER GDEM, Copernicus GLO-30, and ALOS World 3D (AW3D). The assessment is carried out at landscapes spread across Cape Town, Southern Africa (urban/industrial, agricultural, mountain, peninsula and grassland/shrubland) and forested national parks in Gabon, Central Africa (low-relief tropical rainforest and high-relief tropical rainforest). The statistical analysis is based on robust accuracy metrics that cater for normal and non-normal elevation error distribution, and error ranking. In Cape Town, Copernicus DEM generally had the least vertical error with an overall Mean Error (ME) of 0.82 m and Root Mean Square Error (RMSE) of 2.34 m while ASTER DEM had the poorest performance. However, ASTER GDEM and NASADEM performed better in the low-relief and high-relief tropical forests of Gabon. Generally, the DEM errors have a moderate to high positive correlation in forests, and a low to moderate positive correlation in mountains and urban areas. Copernicus DEM showed superior vertical accuracy in forests with less than 40% tree cover, while ASTER and NASADEM performed better in denser forests with tree cover greater than 70%. This study is a robust regional assessment of these global DEMs.

Characteristics of the quick repeat-recall test (Q-RRT)

Objective To determine the reliability and validity of the Quick Repeat-Recall Test (Q-RRT). Design Within-subject repeated measures. Participants completed the Q-RRT at a speech level of 75 dB SPL in 2 noise configurations at signal-to-noise ratios of 5, 10, and 15 dB in a counterbalanced order, along with the full-RRT. The Q-RRT was repeated after 1–3 weeks to estimate within-session and between-session reliability. Participants also completed external validation measures relating speech-in-noise ability (Hearing-In-Noise Test, HINT), memory (Reading Span Test, RST; Forward Digit Span Test, DST-F), and noise acceptance (Tracking of Noise Tolerance, TNT). Study sample Twenty-five normal-hearing listeners (mean age = 63.9 years). Results Group Repeat and Recall scores were similar between the full-RRT and the Q-RRT. However, Listening Effort and Tolerable Time ratings were significantly poorer for the full-RRT than the Q-RRT. The average repeat scores correlated with the HINT scores, while the recall scores correlated with the DST-F and RST scores. Tolerable Time ratings also correlated with the TNT scores. Within-session and between-session test-retest reliability using intraclass correlation coefficients were fair-to-excellent (0.41–0.91) depending on the outcome measures. Conclusions The Q-RRT is a reliable and valid measure of speech-in-noise ability, working memory, listening effort, and noise acceptance.

External clustering validity index based on extended similarity measures

Clustering is one of the most important unsupervised approaches to organizing data in many practical applications. The application of different methods to clustering one dataset generally results in different clusterings of that dataset; for that reason, there is a need for evaluating the similarities of various clustering methods. This can be accomplished with the help of external validity indices. Although many external validity measures exist in the literature, most of them suffer from some drawbacks: they apply very strong null hypotheses, which lead to the independence of the clusterings, fixed number of clusters, and fixed cluster sizes. In addition, an acceptable measure should have some properties to be a metric. If a measure does not qualify for being a metric, the interpretation will not be easy. In this study, a novel Set Matching Index (SMI) is proposed based extended derivation of Jaccard, Dice, and Cosine measures. Thus, three new indices are introduced in this paper, i.e., Set Matching Index based on Extended Jaccard similarity measure (SMI-EJ), Set Matching Index based on Extended Dice similarity measure (SMI-ED) and Set Matching Index based on Extended Cosine similarity measure (SMI-EC) as a set matching metric. All of the metric properties are proved in the present research. For the assessment of the proposed indices performance quality, they are experimented on ten real-world and Synthetic datasets, and then the obtained results are compared with those of four popular external indices (NMI, RI, ARI and Purity) and a new index (PSI). Also to verify the performance of three proposed similarity indices, the experiments develop. We apply the proposed indices to the generated clustering solutions and compare the results. We show that the proposed indices achieve desirable results based on the external validity index properties.Moreover, SMI has simple formula with moderate and appropriate results that can be easily interpreted.

Probing apathy in children and adolescents with the Apathy Motivation Index–Child version

Apathy is linked to mental health and altered neurocognitive functions such as learning and decision-making in healthy adults. Mental health problems typically begin to emerge during adolescence, yet little is known about how apathy develops due to an absence of quantitative measurements specific to young people. Here, we present and evaluate the Apathy Motivation Index–Child Version (AMI-CV) for children and adolescents. We show across two samples of young people (aged 8 to 17 years, total N = 191) tested in schools in the UK and on a smartphone app, that the AMI-CV is a short, psychometrically sound measure to assess levels of apathy and motivation in young people. Similar to adult versions, the AMI-CV captures three distinct apathy domains: Behavioural Activation, Social Motivation and Emotional Sensitivity. The AMI-CV showed excellent construct validity with an alternative measure of apathy and external validity replicating specific links with related mental health traits shown in adults. Our results provide a short measure of self-reported apathy in young people that enables research into apathy development. The AMI-CV can be used in conjunction with the adult version to investigate the impact of levels of apathy across the lifespan.

97 Constant Emitting Reference Material for Material Emissions Test Procedures

Abstract In industrialised countries more than 80 % of the time is spent indoors. Products, such as building materials and furniture, emit volatile organic compounds (VOCs), which are therefore ubiquitous in indoor air. VOC in combination may, under certain environmental and occupational conditions, result in reported sensory irritation and health complaints. Emission concentrations can become further elevated in new or refurbished buildings where the rate of air exchange with fresh ambient air may be limited due to improved energy saving aspects. A healthy indoor environment can be achieved by controlling the sources and by eliminating or limiting the release of harmful substances into the air. One way is to use (building) materials proved to be low emitting. Meanwhile, a worldwide network of professional commercial and non-commercial laboratories performing emission tests for the evaluation of products for interior use has been established. Therefore, comparability of test results must be ensured. A laboratory’s proficiency can be proven by internal and external validation measures that both include the application of suitable emission reference materials (ERM). For the emission test chamber procedure according to EN 16516, no artificial ERM is commercially available. The EU-funded EMPIR project MetrIAQ aims to fill this gap by developing new and improved ERMs. The goal is to obtain a material with a reproducible and temporally constant compound release (less than 10 % variability over 14 days). Different approaches, such as the impregnation of porous materials, are being tested. The generation as well as results of the most promising materials will be presented.

“Trustworthiness,” confidence in estimated effects, and confidently translating research into clinical practice

Trustworthy, preprocessed sources of evidence, such as systematic reviews and clinical practice guidelines, are crucial for practicing clinicians. Confidence in estimated effects is related to how different the outcome data were between the two groups. Factors including the effect size, variability of the effect, research integrity, research methods, and selected outcome measures impact confidence in the estimated effect. The current evidence suggests that post-randomization biases cannot be ruled out with a high degree of certainty in published research, limiting the utility of preprocessed sources for clinicians. Research should be prospectively registered to improve this situation, and fidelity with prospective intent should be verified to minimize biases and strengthen confidence in estimated effects. Otherwise, discussions related to preprocessed literature, including P-values, point estimates of effect, confidence intervals, post-randomization biases, external and internal validity measures, and the confidence in estimated effects required to translate research into practice confidently, are all moot points.

Psychometric properties of the Arabic version of the Trait Emotional Intelligence Questionnaire Short Form (TEIQue-SF)

The Trait Emotional Intelligence Questionnaire Short Form (TEIQue-SF), a brief and effective tool used for assessing emotional intelligence, originated in the Western culture and needs to be adapted to other cultures. The current study aimed to evaluate the psychometric properties of an Arabic-adapted version of the TEIQue-SF. A cross-sectional study was conducted on a sample of 487 Arabic-speaking participants, including students and laypeople. Confirmatory factor analysis (CFA) was used to assess the conceptual factorial structure. Measurement invariance analyses were performed across genders. The big five were employed as external validation measures. The CFA findings validated the proposed TEIQue-SF structure, which included one latent factor (global trait EI) and four indicator factors. Analysis of measurement invariance revealed that the TEIQue-SF Arabic version was factorially equivalent across genders. Furthermore, TEIQue-SF demonstrated good internal consistency. No significant differences are observed between women and men in terms of trait emotional intelligence. The correlations between TEIQue-SF and the big five dimensions demonstrated the theoretically predicted relationships with these variables. We concluded that the Arabic version of the TEIQue-SF is psychologically sound and appropriate for research and practice in Jordan.

Validating the representation of distance between infarct diseases using word embedding

BackgroundThe pivot and cluster strategy (PCS) is a diagnostic reasoning strategy that automatically elicits disease clusters similar to a differential diagnosis in a batch. Although physicians know empirically which disease clusters are similar, there has been no quantitative evaluation. This study aimed to determine whether inter-disease distances between word embedding vectors using the PCS are a valid quantitative representation of similar disease groups in a limited domain.Methodss were extracted from the Ichushi Web database and subjected to morphological analysis and training using Word2Vec, FastText, and GloVe. Consequently, word embedding vectors were obtained. For words including “infarction,” we calculated the cophenetic correlation coefficient (CCC) as an internal validity measure and the adjusted rand index (ARI), normalized mutual information (NMI), and adjusted mutual information (AMI) with ICD-10 codes as the external validity measures. This was performed for each combination of metric and hierarchical clustering method.ResultsSeventy-one words included “infarction,” of which 38 diseases matched the ICD-10 standard with the appearance of 21 unique ICD-10 codes. When using Word2Vec, the CCC was most significant at 0.8690 (metric and method: euclidean and centroid), whereas the AMI was maximal at 0.4109 (metric and method: cosine and correlation, and average and weighted). The NMI and ARI were maximal at 0.8463 and 0.3593, respectively (metric and method: cosine and complete). FastText and GloVe generally resulted in the same trend as Word2Vec, and the metric and method that maximized CCC differed from the ones that maximized the external validity measures.ConclusionsThe metric and method that maximized the internal validity measure differed from those that maximized the external validity measures; both produced different results. The cosine distance should be used when considering ICD-10, and the Euclidean distance when considering the frequency of word occurrence. The distributed representation, when trained by Word2Vec on the “infarction” domain from a Japanese academic corpus, provides an objective inter-disease distance used in PCS.

An adaptive clustering algorithm by neighbourhood search for large-scale data

Adaptive cluster sampling (ACS) is a sampling method relies on the neighbourhood search on a grid structure. It has an adaptive selection process of units and recursively added units reveal the batched individuals easily and quickly. In this paper, we propose a new clustering method called spatial adaptive clustering (SAC) based on the idea of ACS design. The SAC algorithm forms clusters based on neighbourhood search using grid structures and is able to detect noise points. The performance of the proposed algorithm is evaluated through comparison with the results from well-known density-based clustering approaches in the literature using real and artificial data sets. Computational results indicate that the proposed algorithm is effective in terms of external validation measures for clustering of arbitrary shaped data with noise. Additionally, the SAC algorithm is tested on artificial data sets of varying sizes for the runtime criterion. The results reveal that it also performs superbly for the objective of reducing the runtime.

Comparing qPCR and DNA methylation-based measurements of telomere length in a high-risk pediatric cohort.

Various approaches exist to assess population differences in biological aging. Telomere length (TL) is one such measure, and is associated with disease, disability and early mortality. Yet, issues surrounding precision and reproducibility are a concern for TL measurement. An alternative method to estimate TL using DNA methylation (DNAmTL) was recently developed. Although DNAmTL has been characterized in adult and elderly cohorts, its utility in pediatric populations remains unknown. We examined the comparability of leukocyte TL measurements generated using qPCR (absolute TL; aTL) to those estimated using DNAmTL in a high-risk pediatric cohort (N = 269; age: 8–13 years, 83% investigated for maltreatment). aTL and DNAmTL measurements were correlated with one another (r = 0.20, p = 0.001), but exhibited poor measurement agreement and were significantly different in paired-sample t-tests (Cohen’s d = 0.77, p < 0.001). Shorter DNAmTL was associated with older age (r = −0.25, p < 0.001), male sex (β = −0.27, p = 0.029), and White race (β = −0.74, p = 0.008). By contrast, aTL was less strongly associated with age (r = −0.13, p = 0.040), was longer in males (β = 0.31, p = 0.012), and was not associated with race (p = 0.820). These findings highlight strengths and limitations of high-throughput measures of TL; although DNAmTL replicated hypothesized associations, aTL measurements were positively skewed and did not replicate associations with external validity measures. These results also extend previous research in adults and suggest that DNAmTL is a sensitive TL measure for use in pediatric populations.

Brazilian version of Positive Oral Health and Well-Being: cross-cultural adaptation and psychometric analysis.

Specific measures to evaluate positive oral health have been in a nascent stage in Dentistry, especially in developing countries. The present study aimed to translate, cross-culturally adapt and evaluate the psychometric properties of the Brazilian version of Positive Oral Health and Well-Being (B-POHW). After forward-backwards translation to Brazilian Portuguese language, the cross-cultural adaptation of B-POHW was pretested, followed by the main study to perform psychometric analysis. We tested the model fit by Confirmatory Factor Analysis with categorical factor indicators in bifactor and simple structure models on a sample of 209 participants (mean age: 39.36 ± 12.26. Questionnaires about sociodemographic status, self-reported oral health-related outcomes, and general well-being were administered and used as external validation measures. Moreover, dental caries experience was clinically diagnosed. For test-retest reliability, 53 participants completed the B-POHW a fortnight later. The following results were found: a) the bifactor model presented the best model fit; b) the B-POHW demonstrated satisfactory internal consistency (Cronbach's α and McDonald's ω > 0.8); c) the intraclass correlation coefficient suggested good reliability for the Global Factor of B-POHW in the test-retest (ICC = 0.84); d) evidence based on other variables and construct representation was in line with the positive oral health framework. The B-POHW is psychometrically sound to be used in a Brazilian context, and evidence of its internal structure confirmed its theoretical framework for measuring positive oral health. These findings advance in holistic approaches, enabling to assess positive oral health in Dental practice in Brazil.

Standardized database of 400 complex abstract fractals.

In experimental settings, characteristics of presented stimuli influence cognitive processes. Knowledge about stimulus features is important to manipulate or control the influence of stimuli. To date, there are a lack of standardized data incorporating such information for complex abstract stimuli. Thus, we provide norms for a database of 400 abstract and complex stimuli. Grey-scaled fractals were rated by 512 participants on the stimulus features of abstractness, animacy, verbalizability, complexity, familiarity, favorableness, and memorability. Moreover, 111 participants labeled the fractals, enabling us to calculate indices of naming agreement and modal names. Overall, the results confirmed high abstractness and low verbalizability of the provided stimuli. To establish external validation for selected stimulus features, we evaluated (a) classifier probability of a deep neural network labeling the fractals, negatively correlated with ratings of abstractness and positively with verbalizability and naming agreement; (b) data compression rate of fractal image files, positively correlated with the rating of complexity; and (c) performance of 212 participants in a recognition-memory task, positively correlated with the rating of memorability. The present work fills the gap of a standardized database for abstract stimuli and provides a database with valid norms for abstract and complex stimuli based on ratings and external validation measures. This database can be used to control and manipulate these stimulus features in experimental settings using abstract stimuli. Such a database is essential in experimental research using abstract stimuli for instance to control for verbal influence and strategy or to control for novelty and familiarity.

The impact of inconsistent responders to mixed-worded scales on inferences in international large-scale assessments

ABSTRACT Questionnaire scales that are mixed-worded, i.e. include both positively and negatively worded items, often suffer from issues like low reliability and more complex latent structures than intended. Part of the problem might be that some responders fail to respond consistently to the mixed-worded items. We investigated the prevalence and impact of inconsistent responders in 37 primary education systems participating in the joint PIRLS/TIMSS 2011 assessment. Using the mean absolute difference method and three mixed-worded self-concept scales, we identified between 2%‒36% of students as inconsistent responders across education systems. Consistent with expectations, these students showed lower average achievement scores and had a higher risk of being identified as inconsistent on more than one scale. We also found that the inconsistent responders biased the estimated dimensionality and reliability of the scales. The impact on external validity measures was limited and unsystematic. We discuss implications for the use and development of questionnaire scales.

What can we learn from the relationship between gait and clinical impairments in children with cerebral palsy when comparing databases?

Spastic cerebral palsy (CP) is characterized by increased joint resistance, caused by a mix of increased tissue stiffness, as well as involuntary reflex and background muscle activity. These properties can be quantified using a neuromechanical model of the musculoskeletal complex and instrumented assessment. The construct validity of the neuromechanical parameters was examined (i.e. the internal model validity, effect of knee angle, speed and age, sensitivity to patients versus controls, spasticity severity and treatment), together with the repeatability. We included 38 children with CP and 35 controls. A motor driven footplate applied two slow (15°/s) and two fast (100°/s) rotations around the ankle joint, at two different knee angles. Ankle angle, torque and EMG of the gastrocnemius (GA), soleus (SO) and tibialis anterior (TA) muscle were used to optimize a nonlinear neuromuscular model. Outcome measures were tissue stiffness, reflex and background activity for GA, SO and TA. The internal model validity showed medium to high parameter confidence and good model fits. All parameter could discriminate between patients with CP and controls according to CP pathology. Other measures of external model validity (effect of test position, speed and age) showed behaviour along the lines of current knowledge of physiology. GA/SO background activity was sensitive to spasticity severity, but reflex activity was not. Preliminary data indicated that reflex activity was reduced after spasticity treatment. The between-trial and -day repeatability was moderate to good. The large variance between patients in the ratio of stiffness and neural resistance indicates that the method could potentially contribute to patient-specific treatment selection.