The validity of the assessment for learning measurement instrument for Ethiopian middle school mathematics teachers

In this case study the aim is to examine the self-reflection processes (self-evaluation, causal attribution, emotional reactions and inferences) of novice mathematics teachers. The participants were six middle school (grades of 5-8) mathematics teachers who have less than five years teaching experience working in the Western and Central Anatolia Regions of Turkey. The data of the study were collected through the semi-structured interviews developed based on classroom observations. First, the instructional situations were determined by observing the mathematics lessons of the participants (approximately 10 hours for each participant). Afterwards, through semi-structured interviews, the participants were asked to reflect on these instructional situations. During the data analysis, first within case analyses were made, and the self-reflection of each teacher was described. Then by comparing the data obtained from six teachers, it was defined which self-reflection processes the novice mathematics teachers performed and to what extent they performed them. The findings suggest that the participants did not consider their teaching behaviors when they were asked to evaluate the lesson. They attributed the cause of the failure in their lessons to factors beyond their control (e.g., low involvement of parents, insufficient physical facilities of the school). In addition, it is observed that they were less likely to develop positive emotional responses for the observed lessons and were insufficient in taking adaptive decisions for their future lessons. These findings indicate the necessity of pre-service and in-service practices that will support newly recruited middle school mathematics teachers to gain much more comprehensive self-reflection skills.

Confirmatory factor analysis (CFA) and its bifactor models are popular in empirical investigations of the factor structure of psychological constructs. CFA offers straightforward hypothesis testing but has notable pitfalls, such as the imposition of strict assumptions (i.e., simple structure) that obscure unmodeled complexity. Due to the limitations of bifactor CFAs, they have yielded anomalous results across samples and studies that suggest model misspecification (e.g., evaporating specific factors and unexpected loadings). We propose the use of exploratory factor analysis (EFA) to evaluate the structural validity of CFA solutions-either before or after the estimation of more restrictive CFA models-to (a) identify model misspecifications that may drive anomalous estimates and (b) confirm CFA models by examining whether hypothesized structures emerge with limited researcher input. We evaluated the degree to which predominant factor structures were invariant across contexts along the exploratory-confirmatory continuum and demonstrate how poor methodological choices can distort results and impede theory development. All CFA models fit well, but there were numerous differences in replicability and substantive interpretability. Several similarities emerged between bifactor CFA and EFA models, including evidence of overextraction, the collapse of specific factors onto the general factor, and subsequent shifts in how the general factor was defined. We situate these methodological shortcomings within the broader literature on structural models of psychopathology, articulate implications for theories (such as the p-factor) that are borne out of factor analysis, outline several remedies for problems encountered when performing exploratory bifactor analysis, and propose alternative specifications for confirmatory bifactor models. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Management researchers often use structural equation modeling to analyze data from questionnaire-based instruments. Usually, confirmatory factor analysis (CFA) is applied to confirm the hypothesized or theorized factor structure of the instrument. Most authors adopt a single CFA model without comparing it against other potentially valid models (general factor, correlated factor model, second-order hierarchical model, and bifactor model). Hence, the dimensionality and reliability of constructs using bifactor modeling to validate latent scores are often ignored. Also, this gap is widened by no unanimous agreement on the use of post hoc modification of CFA models to support fit to the data in covariance-based structural equation modeling (CB-SEM). The objective of the study was to explore model fit, dimensionality, and reliability of the Employee Work Assessment Tool (EWAT) using competing CFA models. The study used a published dataset on the EWAT instrument to illustrate the assessment of the dimensionality and model-based reliability of the tool using CB-SEM. Results showed that CFA statistics of the bifactor model were most adequate for the instrument (χ2=70.053, df=19, RMSEA=0.082 [90% confidence interval; 0.062, 0.103], SRMR=0.036, CFI=0.963). The bifactor model ancillary measures supported the unidimensional structure of EWAT with justification for the use of total scores. The study concludes that the instrument is best described and applied as a unidimensional construct, and therefore, a single score can be used to rate employees’ perceptions of their work conditions. The study presents both practical implications for management researchers and simplified reporting for bifactor modelling.

To (a) assess the validity and reliability of the 9-item Positive Aspects of Caregiving (PAC) scale among a national sample of caregivers for older adults with functional limitations, (b) develop a shorter version (short-PAC [S-PAC] scale) and assess its psychometric properties, and (c) investigate both scales' measurement equivalence/invariance (ME/I) across language of administration (Chinese/English/Malay). Scale/item measurement property assessment, confirmatory factor analysis (CFA), testing the "original" 2-factor model (6 items: first factor; 3 items: second factor), and exploratory FA (EFA) of the 9-item PAC scale was done. Consequently, alternate CFA models were tested. The S-PAC was developed and subjected to CFA. For both scales, convergent (correlation with caregiver esteem) and divergent (correlation with caregiver depressive symptoms) validity, and language ME/I was assessed. For the 9-item PAC scale, the "original" 2-factor CFA model had a poor fit; its EFA and scale/item measurement properties supported a single factor. Among alternate CFA models, a bi-factor model (all nine items: first factor [overall PAC]; six items: second factor [self-affirmation]; three items: third factor [outlook-on-life]) had the best fit. The bi-factor CFA model also had a good fit for the S-PAC scale, developed after eliminating 2 items from the 9-item PAC scale. Both scales demonstrated convergent and divergent validity, and partial ME/I across language of administration. Both the 9-item PAC and 7-item S-PAC scales can be used to assess positive feelings resulting from care provision among family caregivers of older adults with functional limitations.

The study aims at examining middle school mathematics teachers' conceptions of open-ended questions. A questionnaire consisting of open-ended items was applied to 40 mathematics teachers. The teachers were asked to define the open-ended question in general and the mathematical open-ended question in particular and provide examples to exemplify their definitions. This study employs phenomenographic study aiming at revealing middle school mathematics teachers' conceptions and experience regarding open-ended questions in an exploratory manner. The findings show that the teachers explained the open-ended question through its form (appearance), the number of outputs, the process/method required, and its functionality. In addition, teachers defined the open-ended question mostly using non-mathematical terms, and they had particular difficulties defining the mathematical open-ended question. The teachers regarded questions with variable correct answers as open-ended, could not give examples of open-ended questions with infinitely correct answers, and some deemed closed-ended questions as open-ended. Although the participants were mathematics teachers, the examples they presented for the open-ended question were mostly from outside the field of mathematics. This study points out the fact that teachers need a guiding conceptualization of open-ended questions.

ObjectivesPatient-Reported Outcomes Measurement Information System–Global Health (PROMIS-GH) is a widely used generic measure of health status. This study aimed to (1) assess the psychometric properties of the Hungarian PROMIS-GH and to (2) develop general population reference values in Hungary.MethodsAn online cross-sectional survey was conducted among the Hungarian adult general population (n = 1700). Respondents completed the PROMIS-GH v1.2. Unidimensionality (confirmatory factor analysis and bifactor model), local independence, monotonicity (Mokken scaling), graded response model fit, item characteristic curves and measurement invariance were examined. Spearman’s correlations were used to analyse convergent validity of PROMIS-GH subscales with SF-36v1 composites and subscales. Age- and gender-weighted T-scores were computed for the Global Physical Health (GPH) and Global Mental Health (GMH) subscales using the US item calibrations.ResultsThe item response theory assumptions of unidimensionality, local independence and monotonicity were met for both subscales. The graded response model showed acceptable fit indices for both subscales. No differential item functioning was detected for any sociodemographic characteristics. GMH T-scores showed a strong correlation with SF-36 mental health composite score (rs = 0.71) and GPH T-scores with SF-36 physical health composite score (rs = 0.83). Mean GPH and GMH T-scores of females were lower (47.8 and 46.4) compared to males (50.5 and 49.3) (p < 0.001), and both mean GPH and GMH T-scores decreased with age, suggesting worse health status (p < 0.05).ConclusionThis study established the validity and developed general population reference values for the PROMIS-GH in Hungary. Population reference values facilitate the interpretation of patients’ scores and allow inter-country comparisons.

IntroductionThe Defining Issues Test (DIT) aimed to measure one’s moral judgment development in terms of moral reasoning. The Neo-Kohlbergian approach, which is an elaboration of Kohlbergian theory, focuses on the continuous development of postconventional moral reasoning, which constitutes the theoretical basis of the DIT. However, very few studies have directly tested the internal structure of the DIT, which would indicate its construct validity.ObjectivesUsing the DIT-2, a later revision of the DIT, we examined whether a bi-factor model or 3-factor CFA model showed a better model fit. The Neo-Kohlbergian theory of moral judgment development, which constitutes the theoretical basis for the DIT-2, proposes that moral judgment development occurs continuously and that it can be better explained with a soft-stage model. Given these assertions, we assumed that the bi-factor model, which considers the Schema-General Moral Judgment (SGMJ), might be more consistent with Neo-Kohlbergian theory.MethodsWe analyzed a large dataset collected from undergraduate students. We performed confirmatory factor analysis (CFA) via weighted least squares. A 3-factor CFA based on the DIT-2 manual and a bi-factor model were compared for model fit. The three factors in the 3-factor CFA were labeled as moral development schemas in Neo-Kohlbergian theory (i.e., personal interests, maintaining norms, and postconventional schemas). The bi-factor model included the SGMJ in addition to the three factors.ResultsIn general, the bi-factor model showed a better model fit compared with the 3-factor CFA model although both models reported acceptable model fit indices.ConclusionWe found that the DIT-2 scale is a valid measure of the internal structure of moral reasoning development using both CFA and bi-factor models. In addition, we conclude that the soft-stage model, posited by the Neo-Kohlbergian approach to moral judgment development, can be better supported with the bi-factor model that was tested in the present study.

The Defining Issues Test (DIT) aimed to measure one's moral judgment development in terms of moral reasoning. The Neo-Kohlbergian approach, which is an elaboration of Kohlbergian theory, focuses on the continuous development of postconventional moral reasoning, which constitutes the theoretical basis of the DIT. However, very few studies have directly tested the internal structure of the DIT, which would indicate its construct validity. Using the DIT-2, a later revision of the DIT, we examined whether a bi-factor model or 3-factor CFA model showed a better model fit. The Neo-Kohlbergian theory of moral judgment development, which constitutes the theoretical basis for the DIT-2, proposes that moral judgment development occurs continuously and that it can be better explained with a soft-stage model. Given these assertions, we assumed that the bi-factor model, which considers the Schema-General Moral Judgment (SGMJ), might be more consistent with Neo-Kohlbergian theory. We analyzed a large dataset collected from undergraduate students. We performed confirmatory factor analysis (CFA) via weighted least squares. A 3-factor CFA based on the DIT-2 manual and a bi-factor model were compared for model fit. The three factors in the 3-factor CFA were labeled as moral development schemas in Neo-Kohlbergian theory (i.e., personal interests, maintaining norms, and postconventional schemas). The bi-factor model included the SGMJ in addition to the three factors. In general, the bi-factor model showed a better model fit compared with the 3-factor CFA model although both models reported acceptable model fit indices. We found that the DIT-2 scale is a valid measure of the internal structure of moral reasoning development using both CFA and bi-factor models. In addition, we conclude that the soft-stage model, posited by the Neo-Kohlbergian approach to moral judgment development, can be better supported with the bi-factor model that was tested in the present study.

Many psychological constructs are conceived to be hierarchically structured and thus to operate at various levels of generality. Alternative confirmatory factor analytic (CFA) models can be used to study various aspects of this proposition: (a) The one-factor model focuses on the top of the hierarchy and contains only a general construct, (b) the first-order factor model focuses on the intermediate level of the hierarchy and contains only specific constructs, and both (c) the higher order factor model and (d) the nested-factor model consider the hierarchy in its entirety and contain both general and specific constructs (e.g., bifactor model). This tutorial considers these CFA models in depth, addressing their psychometric properties, interpretation of general and specific constructs, and implications for model-based score reliabilities. The authors illustrate their arguments with normative data obtained for the Wechsler Adult Intelligence Scale and conclude with recommendations on which CFA model is most appropriate for which research and diagnostic purposes.

This paper goal was to validate the Portuguese version (Brazil/Portugal) of the Anger, Anxiety, and Depressive Symptoms item banks of the Pediatric PROMIS® Emotional Distress domain (version 1.0) for the Brazilian and Portuguese pediatric population. The total of 1216 participants answered a self-applied version of the Portuguese Anger, Anxiety, and Depressive Symptoms item banks. Reliability was assessed through internal consistency, test-retest reliability, and total information curve (TIC). Confirmatory Factor Analysis (CFA) with a bifactor model was used to confirm construct validity and IRT assumptions. Item calibration was performed according to Graded Response Model (GRM). Differential Item Functioning (DIF) was analyzed according the participants' age, gender, health condition (healthy versus chronic disease), and language. Internal consistency reliability (Cronbach's alpha coefficient = 0.84) and test-retest reliability (intraclass correlation = 0.93) were accurate. Unidimensionality, Local Independency, and construct validity were verified by CFA (CFI = 0.93; TLI = 0.93; RMSEA = 0.05; χ2 = 3052.4 with DIF = 557 and P value = 0.595). GRM was adjusted, and Emotional Distress had a satisfactory coverage. DIF was not significant. The results obtained indicate the adequacy of the psychometric properties of the Portuguese version (Brazil/Portugal) of the Anger, Anxiety, and Depressive Symptoms item banks of the Pediatric PROMIS® Emotional Distress domain.

The importance of providing structural validity evidence for test score(s) derived from psychometric test instruments is highlighted by several institutions; for example, the American Psychological Association (2014) demands that evidence for the validity of an instruments' internal structure and its underlying measurement model must be provided before it is applied in psychological assessment. The knowledge about the latent structure of data obtained with tests addressing the major question "What is/are the construct[s] being measured" by psychological tests under investigation (Ziegler, 2014 (Ziegler, , 2020)) . The study of structural validity is typically addressed with factor analyses when the test scores reflect continuous latent traits. As most submissions to Psychological Test Adaptation and Development (PTAD) deal with the adaptation and further development of existing measures, authors typically test a measurement model that is based on theoretical considerations and prior findings on original versions (or adaptations) of the test under investigation. Our literature review of PTAD's publications showed that more than 90% of the articles contain at least one confirmatory factor analysis (CFA). As editor and reviewers of PTAD, we appreciate that authors are rigorous in providing evidence on the structural validity of their tests' data. However, since PTAD's inception in 2019, we experience that one comment is frequently communicated to authors during the review process, namely, the request to adjust the analytic approach in CFA from maximum likelihood (ML) estimation toward using the mean-and variance-adjusted weighted least squares (WLSMV; Muthén et al., 1997) estimator to account for the ordinal nature of the data that psychological instruments typically generate on the item level. In this editorial, we discuss the rationale behind choosing the WLSMV estimator when analyzing test adaptations and developments that are based on ordinal categorical data and concisely illustrate the problems associated with using the ML estimator (potentially in combination with robust tests of model fit) for such data.

Abstract. Mindfulness is conceived as a state in which the individual pays full attention to everything that is happening around him or her. The Mindful Attention Awareness Scale (MAAS) is the most popular instrument for assessing mindfulness. Studies on its structure have shown some conflicting results. This study aims to offer new evidence on the dimensionality and reliability of the MAAS, handling both SEM and IRT procedures, in palliative professionals. The sample was composed of 385 professionals from a national online survey. First, two Confirmatory Factor Analyses (CFAs) were specified, estimated, and tested, with one- and two-factor structures, respectively. Second, the Graded Response Model (GRM) was used and accuracy of the MAAS using information functions was estimated. Results showed appropriate fit for the two CFA models. As the correlation between the two factors in the two-factor model was extremely high and the original authors posited a one-factor solution, this structure was retained for parsimony. The GRM also supported this structure, but found that the scale offered more information on professionals with lower levels of mindfulness, pointing at items 1, 2, 6, and 15 as the less discriminative, in line with the CFA lower factor loadings for these very same items.

Middle school mathematics teacher quality is questionable because the number of certified mathematics teachers considered highly qualified is low (Birman et al., 2009). The author examined Grade 8 data from the 2005 National Association of Educational Progress mathematics assessment. The purposes of the study were to (a) determine the impact of middle school mathematics teachers’ content knowledge and teachers’ mathematics pedagogical knowledge on student achievement and (b) compare the effect of the degree to which teachers received reform-oriented professional development activities on student achievement. The results indicated that mathematics content knowledge has a larger role in predicting student achievement than mathematics pedagogical knowledge. Also, teachers who reported participating in fewer professional development activities had students with higher scores than those students whose teachers reported either participating in more professional development. Results for various professional development activities are also presented.

This study examines middle school mathematics teachers’ pedagogical content knowledge (PCK) regarding prime numbers. The study focuses on three subcomponents of PCK: Subject matter knowledge, knowledge of students’ understanding, and knowledge of instructional strategies. A case study method, one of the qualitative research approaches, was employed. The participants consisted of 12 middle school mathematics teachers. Data were collected through the Prime Numbers Knowledge Test (PNKT) and semi-structured interviews developed by the researchers. The data were analyzed using content analysis. The findings revealed that mathematics teachers demonstrated limited PCK, particularly in the components of subject matter knowledge and knowledge of instructional strategies. Specifically, it was observed that teachers provided definitions and responded to questions related to prime numbers without including negative integers, indicating gaps in their subject matter knowledge. Regarding the knowledge of students’ understanding component, participants showed satisfactory competence and performed better than in the other two components. Findings revealed that the teachers had a high level of awareness regarding students' learning difficulties, errors, and misconceptions, and they were able to correctly answer the questions posed by the researchers in this regard. Despite this relative strength, teachers’ knowledge of instructional strategies was notably weak. Although teachers confused the concepts of strategy, method, and technique, they still applied these in their teaching practices. The presentation teaching strategy, the discovery-based teaching strategy, and the question-and-answer technique were identified as the most frequently implemented by the participants.

BackgroundAnxiety disorders can cause serious physical and psychological damage, so many anxiety scales have been developed internationally to measure anxiety disorders, but due to the cultural differences and cultural dependence of quality of life between Chinese and Western cultures, it is difficult to reflect the main characteristics of Chinese patients. Therefore, we developed a scale suitable for Chinese patients with anxiety disorders: the Anxiety Disorders Scale of the Quality of Life Instruments for Chronic Diseases (QLICD-AD), hoping to achieve satisfactory QOL assessments for anxiety disorders.ObjectivesItems from the Anxiety Disorders Scale of the Quality of Life in Chronic Disease Instrument QLICD-AD system were analyzed using CTT and IRT to lay the groundwork for further refinement of the scale to accurately measure anxiety disorders.Methods120 patients with anxiety disorder were assessed using the QLICD-AD (V2.0). Descriptive statistics, variability method, correlation coefficient method, factor analysis and Cronbach’s coefficient of CTT, and graded response model (GRM) of item response theory were used to analyze the items of the scale.ResultCTT analysis showed that the standard deviation of each item was between 0.928 and 1.466; Pearson correlation coefficients of item-to-domain were generally greater than 0.5 and also greater than that of item-to-other domain; the Cronbach ‘s of the total scale was 0.931, α of each domain was between 0.706 and 0.865. IRT analysis showed that the discrimination was between 1.14 and 1.44. The difficulty parameter of all items increased with the increase of grade. But some items (GPH6,GPH8,GPS3,GSO2-GSO4,AD2,AD5) difficulty parameters were less than 4 or greater than 4. The average of information amount was between 0.022 and 0.910.ConclusionBased on CTT and IRT analysis, most items of the QLICD-AD (V2.0) scale have good performance and good differentiation, but a few items still need further revision. Suggests that the QLICD-AD (V2.0) appears to be a valid measure of anxiety disorders. It may effectively improve the diagnosticity of anxiety disorders, but due to the limitations of the current sample, further validation is needed in a broader population extrapolation trial.