Response Theory Methods Research Articles

Theoretical Insights on the Sensing Performance for Newly-synthesized Two-photon Fluorescent N2H4 Probes Based on Spirobifluorence.

The development of fluorescent probe for hydrazine (N2H4) detection has attracted much attention due to the important role of N2H4 plays in the fields of medicine, agriculture, biology and environments. In this paper, the optical properties and water solubility of two novel two-photon fluorescent molecular probes (Probe1 and Probe2) before and after the reaction with N2H4 are studied by using the density function theory. The results show that electronic distribution and transition dipole moment of the probes are obviously changed after the reaction with N2H4, thus the optical properties of the molecules are influenced and the detection of N2H4 are realized. In addition, photoinduced electron transfer processes for Probe1 and Probe2 in the presence of N2H4 are theoretically characterized, which explains the experimental observations from the microscopic mechanism. Special attention has been paid on the analysis of the two-photon absorption for the probes with the absence and presence of N2H4 by the response theory method. Both probes with good water solubility show large variation on the two-photon absorption cross section when reacts with N2H4. In particular, the two-photon absorption response of Probe2 is more obvious, so it possesses preferable two-photon fluorescence microscopic imaging ability. More importantly, the receptor effect on the sensing performances of the probes are demonstrated, providing a theoretical reference for the design and synthesis on more efficient two-photon fluorescence N2H4 probes. Our study provides necessary information on the response mechanism of the studied chemosensors and helps to establish the relationship between the structure and optical properties of two-photon fluorescence N2H4 probes.

Factor Structure, Measurement Invariance, and Abbreviated Versions of The Adolescent Dissociative Experiences Scale (A-DES)

ABSTRACT The purpose of this study was to examine the factor structure of the Adolescent Dissociative Experiences Scale (A-DES) questionnaire with item response theory (IRT) methods, including an assessment of measurement invariance with differential item functioning (DIF) analysis. Three abbreviated versions of the A-DES (with 20, 10, and 5 items) were constructed based on the IRT and DIF statistics. The respondents in this population-based study (N = 4,072) were 12- to 19-year-old Finnish junior and senior high school students. A one-factor model of the A-DES was best supported, and the original theoretical four-factor model showed poor fit. The A-DES turned out to have high measurement invariance with respect to age, gender, transgender tendencies, having multiple friends, the use of illegal substances, and experience of being bullied. Compared to the full 30-item A-DES, abbreviated versions of the questionnaire retained acceptable information value and empirical reliability in the clinically relevant range of symptomatology. Further psychometric studies are needed especially with regards to clinical use.

The effect of missing data and imputation on the detection of bias in cognitive testing using differential item functioning methods

BackgroundItem response theory (IRT) methods for addressing differential item functioning (DIF) can detect group differences in responses to individual items (e.g., bias). IRT and DIF-detection methods have been used increasingly often to identify bias in cognitive test performance by characteristics (DIF grouping variables) such as hearing impairment, race, and educational attainment. Previous analyses have not considered the effect of missing data on inferences, although levels of missing cognitive data can be substantial in epidemiologic studies.MethodsWe used data from Visit 6 (2016–2017) of the Atherosclerosis Risk in Communities Neurocognitive Study (N = 3,580) to explicate the effect of artificially imposed missing data patterns and imputation on DIF detection.ResultsWhen missing data was imposed among individuals in a specific DIF group but was unrelated to cognitive test performance, there was no systematic error. However, when missing data was related to cognitive test performance and DIF group membership, there was systematic error in DIF detection. Given this missing data pattern, the median DIF detection error associated with 10%, 30%, and 50% missingness was -0.03, -0.08, and -0.14 standard deviation (SD) units without imputation, but this decreased to -0.02, -0.04, and -0.08 SD units with multiple imputation.ConclusionsIncorrect inferences in DIF testing have downstream consequences for the use of cognitive tests in research. It is therefore crucial to consider the effect and reasons behind missing data when evaluating bias in cognitive testing.

A typical stochastic resonance in Landau-type potential field

Considering the significant influence of the asymmetric structure of the one-dimensional potential field on the stochastic resonance (SR), the research is extended to the two-dimensional asymmetric potential field. The Brownian particles moving in the Landau-type potential field served as the research object. According to the linear response theory and probability flow method, the analytical formula for the spectrum power amplification (SPA) of the physical system concerning the driving frequency is derived. When the SPA is used as a measurement index, the theoretical analysis shows that an atypical SR phenomenon occurs in the system. The reason is that as the noise intensity increases, the response mode changes due to the asymmetric structure of the Landau-type potential field. Moreover, the linear responses and the SPA are inversely proportional to the driving frequency. The conclusions of the theoretical analysis are verified by numerical simulation. Finally, the Landau-type potential system is applied to the bearing fault signal detection. The experimental results show that the Landau-type potential system can effectively diagnose the faults of the inner and outer ring of the bearings, which proves the application value of the Landau-type potential system.

Translating Item Response Theory Findings for Clinical Practice

This issue contains two articles that use item response theory (IRT) methods to draw conclusions about the psychometric properties of assessment instruments. Specifically Black (2019) uses IRT anal...

Classification Consistency and Accuracy for Mixed-Format Tests

ABSTRACT This study explores classification consistency and accuracy for mixed-format tests using real and simulated data. In particular, the current study compares six methods of estimating classification consistency and accuracy for seven mixed-format tests. The relative performance of the estimation methods is evaluated using simulated data. Study results from real data analysis showed that the procedures exhibited similar patterns across various exams, but some tended to produce lower estimates of classification consistency and accuracy than others. As data became more multidimensional, unidimensional and multidimensional item response theory (IRT) methods tended to produce different results, with the unidimensional approach yielding lower estimates than the multidimensional approach. Results from simulated data analysis demonstrated smaller estimation error for the multidimensional IRT methods than for the unidimensional IRT method. The unidimensional approach yielded larger error as tests became more multidimensional, whereas a reverse relationship was observed for the multidimensional IRT approach. Among the non-IRT approaches, the normal approximation and Livingston-Lewis methods performed well, whereas the compound multinomial method tended to produce relatively larger error.

Psychometrics of the self-efficacy for physical activity scale among a Latina women sample

BackgroundEven though Latinos have become a priority population for the promotion of physical activity in the United States, several widely used scales in physical activity promotion research have not been validated among this population, particularly in Spanish. This study aims to assess the validity and other psychometrics of the Self-Efficacy for Physical Activity scale among a sample of Spanish-speaking Latina women who participated in the Pasos Hacia La Salud intervention. We also explored alternatives for scale simplification.MethodsData from 205 women corresponding to baseline, 6-month, and 12-month time points were analyzed. Internal consistency was assessed. A series of Spearman correlations, t-tests, linear regressions, and logistic regressions were used to assess the concurrent and predictive validity of the Self Efficacy for Physical Activity scale against both self-report and accelerometer-measured physical activity, using both continuous and categorical outcome data. Item Response Theory and factor analysis methods were used to explore alternatives to simplify the scale. Psychometric tests were repeated with the simplified scale.ResultsCronbach’s alpha for the original scale was .72, .76, and .78 for baseline, 6-month, and 12-month data respectively. All concurrent validity tests conducted with 6-month and 12-month data, but not with baseline data, were statistically significant. Self-efficacy at 6 months was also predictive of physical activity at 12 months for all tests except one. Based on plots of Option Characteristic Curves, a modified version of the scale was created. Psychometric results of the modified scale were similar to those of the original scale.ConclusionsThis study confirmed the scale’s reliability and validity, and revealed that the scale’s accuracy improves when some response items are collapsed, which is an important finding for future research among populations with low literacy levels.

Examining the Psychometric Properties of the SRSS-IE With the Nominal Response Model Within a Middle School Sample

This study examined the item and category response option functioning of items on the Student Risk Screening Scale for Internalizing and Externalizing Behaviors (SRSS-IE), using item response theory (IRT) methods with a sample of 2,122 middle school students. The SRSS-IE is a screening instrument used to identify students who are potentially at risk for emotional and behavioral disorders (EBD). Utilizing the nominal response and generalized partial credit models of IRT, items from the SRSS-IE were evaluated in terms of the degree to which the response options for each item functioned as intended and how well those response options discriminated among students who exhibited varying levels of EBD risk. Results indicate that the four-response option configuration used in this study may not adequately discriminate regarding the frequency of externalizing and internalizing behaviors demonstrated by middle school students. Recommendations for revising the response options and scoring procedures are discussed.

Assessing Learning Behaviors in Early Childhood Special Education Classrooms

ABSTRACTResearch Findings: This study reports on the dimensionality and validation of the Preschool Learning Behaviors Scale (PLBS) for use in a sample of preschool children in early childhood special education classrooms. The PLBS is a teacher-report instrument that characterizes children’s approaches to learning. The sample included 600 preschool-age students in 97 classrooms. Exploratory and confirmatory analyses uncovered 2 reliable and valid dimensions: Competence Motivation and Learning Strategy. Scale scores were produced using item response theory and Bayesian scoring methods. Competence Motivation and Learning Strategy scores were moderately related to children’s performance on language and literacy measures. Practice or Policy: The potential use of this tool in early childhood special education classrooms is explored.

A Criterion-Referenced Approach to Student Ratings of Instruction

We developed a criterion-referenced student rating of instruction (SRI) to facilitate formative assessment of teaching. It involves four dimensions of teaching quality that are grounded in current instructional design principles: Organization and structure, Assessment and feedback, Personal interactions, and Academic rigor. Using item response theory and Wright mapping methods, we describe teaching characteristics at various points along the latent continuum for each scale. These maps enable criterion-referenced score interpretation by making an explicit connection between test performance and the theoretical framework. We explain the way our Wright maps can be used to enhance an instructor’s ability to interpret scores and identify ways to refine teaching. Although our work is aimed at improving score interpretation, a criterion-referenced test is not immune to factors that may bias test scores. The literature on SRIs is filled with research on factors unrelated to teaching that may bias scores. Therefore, we also used multilevel models to evaluate the extent to which student and course characteristic may affect scores and compromise score interpretation. Results indicated that student anger and the interaction between student gender and instructor gender are significant effects that account for a small amount of variance in SRI scores. All things considered, our criterion-referenced approach to SRIs is a viable way to describe teaching quality and help instructors refine pedagogy and facilitate course development.

Psychometric Properties of the Shipley Block Design Task: A Study With Jamaican Young Adults

We examined the structure of the new Block Patterns (BP) test from the Shipley Institute of Living Scale–Second Edition in a sample of Jamaican young adults. To date, very little has been published on the properties of this subtest’s items and scores. The BP test is similar in design to the Block Design subtest found in many cognitive ability assessments but uses a matching format that minimizes the need for excess materials and time. We analyzed the BP items using item response theory (IRT) methods. Although designed to measure a single construct, the analyses from this study found that the BP subtest is likely measuring more than a single construct, which confounds the interpretation of the instrument’s scores. Before the subtest is used clinically, more research should be done to purposefully investigate the effects of ancillary variables on its scores.

Quantitative studies of validity: expanding our understanding of disability

Purpose: This manuscript seeks to explain why applications of item response theory (IRT) and factor analytic methods provide evidence of validity, and why for the same reasons that they provide evidence of validity, they can advance substantive knowledge. Methods: A narrative review of the psychometrics literature and disability literature is presented explaining the rationale for the use of quantitative validation methods. Results: Both the field of psychometrics and the application of psychometric methods in rehabilitation science are expanding rapidly. Logistic IRT models and factor analytic methods are the most commonly used validation tools in rehabilitation. Conclusions: Many of the available psychometric tools provide evidence of validity, because they are powerful tests of formally specified hypotheses regarding how specific observable traits relate to underlying latent construct. Thus, while the methods for studying psychometric validity have largely been focused on the question of whether or not rehabilitation outcome measures can be used with legitimacy, they also offer a potential research tool for explaining the mechanisms of disability within the framework of the International Classification of Functioning, Disability, and Health.Implications for RehabilitationQuantitative validation of a functional status instrument using latent trait modeling techniques (i.e. IRT and CFA) study the measurement value of an instrument and help to ensure that the instrument provides meaningful information to clinical providers.Latent trait modeling has demonstrated that consistent with the ICF, participation restrictions in those with chronic knee osteoarthritis is driven by activity limitations caused by the knee impairments rather than directly by the knee impairments themselves.

Using modern test theory to maintain standards in public qualifications in England

This paper describes how item response theory (IRT) methods of test-equating could be applied to the maintenance of public examination standards in England. IRT methods of test-equating have been sparingly applied to the main public examinations in England, namely the General Certificate of Secondary Education (GCSE), the equivalent of a school leaving examination, taken at age 16, and A-levels, taken at age 18 prior to university entrance. The lack of application of test-equating may be because such methods are thought to be irrelevant or surplus to requirements or because the IRT models that were originally considered in this context were simple and based on rigid assumptions that would not hold in the case of the GCSE and A-level. This paper illustrates that current methods used for the maintenance of standards lack any reliable measure of performance standard, and explores some developments in modern IRT methods that seek to overcome the restrictions of early IRT models and equating designs. Specifically, it reports on a post-equating study that attempts to link performance standards between a January and a June GCSE test session. The linking is done in a Bayesian IRT framework as well as a marginal maximal likelihood framework using the one parameter logistic model. Results from various equating methods and IRT models are discussed and compared to the results derived from the standard methods used in England. It concludes that test-equating studies can be used to derive fairly accurate estimations of performance standards that are only crudely available from other indicators currently used in the system.

Role of non-Condon vibronic coupling and conformation change on two-photon absorption spectra of green fluorescent protein

Two-photon absorption spectra of green fluorescent proteins (GFPs) often show a blue-shift band compared to their conventional one-photon absorption spectra, which is an intriguing feature that has not been well understood. We present here a systematic study on one- and two-photon spectra of GFP chromophore by means of the density functional response theory and complete active space self-consistent field (CASSCF) methods. It shows that the popular density functional fails to provide correct vibrational progression for the spectra. The non-Condon vibronic coupling, through the localised intrinsic vibrational modes of the chromophore, is responsible for the blue-shift in the TPA spectra. The cis to trans isomerisation can be identified in high-resolution TPA spectra. Our calculations demonstrate that the high level ab initio multiconfigurational CASSCF method, rather than the conventional density functional theory is required for investigating the essential excited-state properties of the GFP chromophore.

Computer Adaptive Tests in der Medizin

Measurement of Patient-reported Outcomes (PRO) still lacks behind clinical standards. Most established tools are also either too burdensome or too imprecise to be used in clinical practice. Item Response Theory (IRT) methods and Computer Adaptive Tests (CAT) promise to overcome these shortcomings. Simulation studies have shown that individually tailored CATs can provide more precise and less burdensome measurements over a larger measurement range than static tools. Several studies with real CAT application have supported the psychometric superiority of CATs, but results from longitudinal studies are still scarce. IRT item banks also allow scoring different established tools measuring the same construct on one common metric, which could greatly facilitate the harmonization of PRO-assessments.

Comparison between dominance process model and ideal point response model for personality assesment

The present study investigated the utilities of two types of item response process models(dominance model and ideal point model) for personality item parameter estimation and scoring. The authors developed scales for four personality traits(achievement, fairness, cooperation and honesty) using classical test theory, dominance item response theory(IRT) method, and ideal point IRT method and compared the methods in terms of model-data fit, information and criterion validity. Results show that the fit of ideal point IRT model was better than that of dominance IRT model, but the difference between the fit of two models was very slight. The test information functions of ideal point IRT model and dominance IRT model for honesty and cooperation scales were very similar. The criterion-related validity based on individual ability estimates and grades was not significant for the three methods but the validity for the ideal point method is not better than dominant IRT model. Implications and limitations of the findings are discussed.

An IRT validation of the Affective Self Rating Scale

Background: The Affective Self Rating Scale (AS-18) is intended for the use in bipolar outpatients. It includes subscales for the rating of depressive and manic-type symptoms. It has previously been validated using methods from Classical Test Theory. Aims: The aim of this study was to evaluate the psychometric properties of the AS-18 when used at an outpatient clinic for patients with bipolar disorder at routine visits, and to analyze the potential for improvement of the scale. Methods: 231 patients with mainly bipolar I disorder doing ratings on routine visits at an affective disorder outpatient clinic were included. Ratings were analyzed using the Mokken non-parametric and the Rasch parametric model statistical methods. Results: In the Mokken analysis, both subscales of the AS-18 showed a strong ability to rank respondents according to their total score and all items contributed adequately to the measurement. In the Rasch model, there were no indications of disturbing influence from secondary dimensions in the subscales. The depression subscale had the capacity reliably to separate the sample in at most three levels and the mania subscale in two levels. The limited capacity to separate respondents can mainly be explained by a lack of items reflecting lower levels of depressive and manic symptoms. Conclusions: AS-18 has good basic psychometric properties for use of rating of symptoms in bipolar I patients at routine visits, but there is also room for improvement. Item Response Theory (IRT) methods are suitable tools for evaluation and construction of rating scales.

Development and Validation of a Disease-Specific Quality-of-Life Measure for Children with Achalasia

Achalasia is an uncommon oesophageal motility disorder occasionally affecting children. Children with achalasia can experience significant morbidity, and quality-of- life (QoL) has never been studied in this population. The aim was to develop a disease-specific quality-of-life (DS-QoL) measure for children with achalasia. Item response theory methods were used to develop the DS-QoL measure. The construct validity of this measure was assessed by comparing items and domains with the generic PedsQL questionnaire. Reliability was assessed using Cronbach's alpha coefficient for internal consistency. 17 children completed the final DS-QoL measure, which consisted of 20 items in three domains. The completion rate for items was 99%. "Floor and ceiling effects" ranged from 0-19%. Construct validity was good with significant correlation between 2 domains and 2 items on the PedsQL. Reliability was excellent, with Cronbach's alpha coefficient ranging from 0.78-0.93. This DS-QoL measure is appropriate for use in children with achalasia and has shown good results in this validation study. Further work in higher numbers is necessary to determine discriminant validity and test-retest reliability.

Stability of Rasch Scales Over Time

Item response theory (IRT) methods are generally used to create score scales for large-scale tests. Research has shown that IRT scales are stable across groups and over time. Most studies have focused on items that are dichotomously scored. Now Rasch and other IRT models are used to create scales for tests that include polytomously scored items. When tests are equated across forms, researchers check for the stability of common items before including them in equating procedures. Stability is usually examined in relation to polytomous items' central “location” on the scale without taking into account the stability of the different item scores (step difficulties). We examined the stability of score scales over a 3–5-year period, considering both stability of location values and stability of step difficulties for common item equating. We also investigated possible changes in the scale measured by the tests and systematic scale drift that might not be evident in year-to-year equating. Results across grades and content areas suggest that equating results are comparable whether or not the stability of step difficulties is taken into account. Results also suggest that there may be systematic scale drift that is not visible using year-to-year common item equating.

On “Prospective Evaluation of the AM-PAC-CAT…” Jette et al. Phys Ther. 2007;87:385–398.

To the editor: The authors are to be complimented for a strong paper describing a methodologically complex process in an understandable manner. Although others have applied computerized adaptive testing (CAT) applications in outpatient rehabilitation for several years,1 Jette et al2 are the first to publish results of a practical application of a CAT in a peer-reviewed journal. The strength of the work by Jette et al lies in the process used to develop the product. Item response theory (IRT) methods and CAT applications have the potential to be the foundation of outcomes measurement development in rehabilitation just as they were in educational measurement.3 We should not forget that IRT and CAT are not new; they are just new to rehabilitation and medicine. Jette et al discuss in the current paper and in earlier work how these methods can be used to develop a new outcomes scale, assess the strengths and weaknesses of the scale, and discuss how a scale can be improved when scale deficits are identified via practical application in busy clinics. These methods are sorely needed for many common paper-and-pencil instruments that are so popular in rehabilitation. The study is not without limitations, many of which are detailed nicely by Jette et al. One psychometric issue not discussed relates to differential item functioning (DIF).4 Differential item functioning occurs when patients from different groups—for example, patients with hip versus knee impairments—have different probabilities of endorsing item response categories. …