Item Response Research Articles

Gender and socioeconomic perspectives on students’ emotional well-being during the COVID-19 pandemic

This study examines emotional well-being variations among students ( n = 13,398) across 5 countries during the COVID-19 pandemic. Secondary analysis is conducted using data from the Responses to Educational Disruption Survey (REDS), which focused on the pandemic experiences, including well-being, of 8th-grade students. The study constructs an emotional well-being scale employing bi-factor models and multidimensional item response theory, encompassing students’ concerns, loneliness, social isolation, and negative emotions. Cross-analyses with gender and socioeconomic status (SES) reveal significant between-group disparities, measured by Cohen’s d effect sizes. It uncovers gender gaps in emotional well-being, with girls consistently scoring lower in most countries. SES effects on emotional well-being vary among countries, with SES group distinctions generally smaller than gender disparities. Thus, gender appears to have played a more significant role than SES in relation to 8th-grade students' emotional well-being during the COVID-19 pandemic.

Combining Mokken Scale Analysis with Rasch measurement theory to explore differences in measurement quality between subgroups

Mokken Scale Analysis (MSA) is a nonparametric approach that offers exploratory tools for understanding the nature of item responses while emphasizing invariance requirements. MSA is often discussed as it relates to Rasch measurement theory, which also emphasizes invariance, but uses parametric models. Researchers who have compared and combined MSA and Rasch approaches have generally concluded that these approaches offer complementary perspectives. However, researchers have not considered how the two approaches may provide insight related to examinee subgroups. Using a simulation and illustrative real data analysis, we propose an exploratory approach to integrating MSA and Rasch techniques to understand item fit as it relates to differences in item characteristics between examinee subgroups. Our results suggest that integrating complete-sample and subgroup-level MSA and Rasch model item fit analyses can provide insight into differences in item quality between subgroups. In some cases, misfit at the complete sample level corresponds to subgroup-related differences. On the other hand, fit at the complete sample level does not guarantee consistent measurement properties within subgroups. When subgroups are important for scale interpretation and use, our results suggest that conducting separate analyses within groups and considering their results with respect to those from the full sample may lend useful insight.

Проверка измерительной инвариантности для тестов когнитивных и языковых способностей

<p style="text-align: justify;">This paper discusses the measurement invariance of assessment instruments. The purpose of measurement invariance testing is to assess the extent to which a measured construct has comparable meaning across different groups or measurement points, to ensure that comparisons of these groups or measures to each other are appropriate. The paper provides an overview of recent and classic papers discussing conceptual and methodological aspects of invariance testing. Specific focus is placed on the available data on invariance testing for the assessments of cognitive and language abilities, and on the significance of invariance for the accurate assessment of individual and between-group differences. Finally, current approaches to invariance testing within structural equation modelling and item response theory are discussed. This paper may serve as an introduction to the concept of measurement invariance. It will be useful to readers interested in improving the quality of measurement instruments and making more reliable inferences about psychological phenomena.</p>

Equivalence of Alcohol Use Disorder Symptom Assessments in Routine Clinical Care When Completed Remotely via Online Patient Portals Versus In Clinic via Paper Questionnaires: Psychometric Evaluation.

The National Institute on Alcohol Abuse and Alcoholism (NIAAA) recommends the paper-based or computerized Alcohol Symptom Checklist to assess alcohol use disorder (AUD) symptoms in routine care when patients report high-risk drinking. However, it is unknown whether Alcohol Symptom Checklist response characteristics differ when it is administered online (eg, remotely via an online electronic health record [EHR] patient portal before an appointment) versus in clinic (eg, on paper after appointment check-in). This study evaluated the psychometric performance of the Alcohol Symptom Checklist when completed online versus in clinic during routine clinical care. This cross-sectional, psychometric study obtained EHR data from the Alcohol Symptom Checklist completed by adult patients from an integrated health system in Washington state. The sample included patients who had a primary care visit in 2021 at 1 of 32 primary care practices, were due for annual behavioral health screening, and reported high-risk drinking on the behavioral health screen (Alcohol Use Disorder Identification Test-Consumption score ≥7). After screening, patients with high-risk drinking were typically asked to complete the Alcohol Symptom Checklist-an 11-item questionnaire on which patients self-report whether they had experienced each of the 11 AUD criteria listed in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) over a past-year timeframe. Patients could complete the Alcohol Symptom Checklist online (eg, on a computer, smartphone, or tablet from any location) or in clinic (eg, on paper as part of the rooming process at clinical appointments). We examined sample and measurement characteristics and conducted differential item functioning analyses using item response theory to examine measurement consistency across these 2 assessment modalities. Among 3243 patients meeting eligibility criteria for this secondary analysis (2313/3243, 71% male; 2271/3243, 70% White; and 2014/3243, 62% non-Hispanic), 1640 (51%) completed the Alcohol Symptom Checklist online while 1603 (49%) completed it in clinic. Approximately 46% (752/1640) and 48% (764/1603) reported ≥2 AUD criteria (the threshold for AUD diagnosis) online and in clinic (P=.37), respectively. A small degree of differential item functioning was observed for 4 of 11 items. This differential item functioning produced only minimal impact on total scores used clinically to assess AUD severity, affecting total criteria count by a maximum of 0.13 criteria (on a scale ranging from 0 to 11). Completing the Alcohol Symptom Checklist online, typically prior to patient check-in, performed similarly to an in-clinic modality typically administered on paper by a medical assistant at the time of the appointment. Findings have implications for using online AUD symptom assessments to streamline workflows, reduce staff burden, reduce stigma, and potentially assess patients who do not receive in-person care. Whether modality of DSM-5 assessment of AUD differentially impacts treatment is unknown.

Development of Four-Item Attitudes toward People Living with Dementia Scale for population surveys.

This study developed a short version of a scale measuring attitudes toward people living with dementia, the Four-Item Attitudes toward People Living with Dementia Scale (APDS4), that could be included in a large population survey. We used three datasets from Japan: a web panel survey, a community-based mail survey, and data from a randomised controlled trial on dementia education. The original scale used was the Attitudes toward People Living with Dementia Scale developed by Kim and Kuroda. Test-retest reliability and item response theory analyses were used to reduce the number of items. The reliability, internal consistency, validity, and responsiveness of the short version were evaluated. Six items with low test-retest reliability and four items with low discrimination parameters were removed from the 14-item scale. The APDS4, consisting of four items, had test-retest reliability and internal consistency comparable to those of the original scale. Confirmatory factor analysis indicated that the APDS4 fit a unidimensional model. The validity of the APDS4 was confirmed by significant associations between the APDS4 scores and the original scale scores, knowledge of dementia, helping behaviour intentions toward people living with dementia, helping behaviour experience, attending the Dementia Supporter Training Course, and engagement in healthcare jobs. In a randomised controlled trial dataset, the APDS4 was more responsive to educational interventions than the original scale. The shortened APDS4 was established as a reliable, validated, and responsive scale. This scale can be used efficiently in population surveys to evaluate dementia-friendly initiatives at the community level.

Interpretability and Clinical Utility of the Pediatric Evaluation of Disability Inventory - Patient Reported Outcome (PEDI-PRO) Score Report

Aims Well-designed score reports can support therapists to accurately interpret assessments. We piloted a score report for the Pediatric Evaluation Disability Inventory-Patient Reported Outcome (PEDI-PRO) and evaluated: 1) To what extent can occupational and physical therapists (OT, PT) accurately interpret item-response theory (IRT)-based PEDI-PRO assessment results? 2) What is the perceived clinical utility of the pilot score report? Methods Exploratory, sequential mixed methods design. Focus groups with OT and PTs (n = 20) informed the development of the final score report; revisions were made in response to feedback. Next, OTs and PTs (n = 33) reviewed score reports from two fictional clients and answered survey questions about the interpretation of the PEDI-PRO results. Additional questions evaluated clinical utility. Results Focus groups: Visual cues supported score interpretation, but therapists requested additional explanations for advanced IRT measurement concepts. Survey: Therapists accurately interpreted foundational IRT concepts (e.g. identifying most/least difficult items, highest scores), but were less accurate when interpreting advanced concepts (e.g. fit, unexpected responses). Therapists anticipated sharing different components of the score report with family members, clinicians, and payers to support their clinical practice. Conclusions The pilot PEDI-PRO score report was highly endorsed by therapists, but therapists may need additional training to interpret advanced IRT concepts.

Assessment of Chinese primary school mathematics teachers’ knowledge of students’ misconceptions regarding space and shape

ABSTRACT The study aimed to establish an assessment model for mathematics teachers’ knowledge of students’ misconceptions in the Space and Shape domain, develop the testing tool, investigate and analyse the overall and differences in performance, and propose suggestions for improvement. The assessment model included content knowledge and performance standards. The content knowledge standard included cognition and content dimensions. The cognition dimension was subdivided into predicting the misconceptions, interpreting the misconceptions, analysing the reasons and teaching strategies to correct misconceptions. According to the division of Space and Shape in official curriculum standard, the content dimension was subdivided into Recognition of graphics, Measurement, Motion, and Location. 701 Chinese mathematics teachers from 8 provinces were involved to verify the assessment model. Item response theory was used to estimate teachers’ performance, and combinations of response categories was used to divide the teachers’ performance into three response levels: Low, Medium and High. 81% of teachers were performing at Medium and High response levels. Teachers performed the worst in Recognition of graphics and the best in Measurement. The worst performance occurred in analysing the reasons, the best in interpreting the misconceptions. These results have provided evidences for adjusting teacher activities, improving teacher education courses and teachers training.

Differential item functioning of the revised Multigroup Ethnic Identity Measure (MEIM-R) in racially and income diverse youth with type 1 diabetes.

Racially minoritized youth with T1D are made vulnerable to disproportionately adverse health outcomes compared to White peers due to enduring systems of oppression. Thus, understanding modifiable psychosocial factors associated with diabetes-related outcomes in racially minoritized youth may help to buffer deleterious effects of racism. One factor meriting exploration is racial-ethnic identity. There is currently limited research on measures fit to assess ethnic identity in youth with chronic illnesses. This study's purpose is to examine the factor structure, reliability, and validity of the revised Multigroup Ethnic Identity Measure (MEIM-R) in a racially- and income-diverse sample of youth with T1D across sociodemographic and illness-related proxies for one's positionality in oppressive systems. As part of a larger study examining resilience, 142 youth with T1D ages 12-18 (Mage = 14.66, SDage = 1.62, 55.6% Black/African-American, 44.4% White) completed the MEIM-R and various psychosocial measures. HbA1c levels and illness duration were extracted from medical records and caregivers reported income information. Confirmatory factor analyses compared the structural validity of competing MEIM-R models, and uniform and non-uniform differential item functioning (DIF) was explored across sociodemographic and illness-related factors. While a bifactor structure was supported, the MEIM-R was found to exhibit DIF by race and gender on multiple MEIM-R items and did not demonstrate linear bivariate relations with other psychosocial factors. Since different MEIM-R item response patterns were observed across racial/ethnic and gender groups, caution is warranted in using this measure in racially and gender diverse youth with T1D.

How Many Is Too Many? Using Cognitive Load Theory to Determine the Maximum Safe Number of Inpatient Consultations for Trainees.

Cognitive load, specifically extraneous load (EL) reflective of distractions, may provide evidence of a lack of focus, potentially making additional work unsafe. The assessment of trainees performing inpatient consultations provides a helpful model for examining this question. The goal of this study was to provide useful information to clinical and educational leaders to optimize inpatient consultation services and rotations and mitigate potential patient safety risk. In 2019, using the Consult Cognitive Load instrument, the authors obtained EL data from inpatient consultations performed by internal medicine fellows and psychiatry residents across 5 University of California hospitals. In 2023, the authors constructed a Wright map to compare the participants' EL data with the number of prior initial consultations performed during the shift. Of 326 trainees contacted, 139 (43%) completed the EL survey items. The Wright map shows that trainees were estimated to agree that interruptions were already distracting at the first consultation of the shift. After 4 consultations, trainees were estimated to strongly agree that interruptions were distracting and to agree that emotions, extraneous information, and technology were distracting. The authors propose a quantitative, empirically driven, mean safety limit of 4 new inpatient consultations per shift for trainees to avoid cognitive overload, thereby potentially supporting patient safety. Clinical and educational leaders can adjust this limit to fit the unique needs of their practice setting. A similar approach using cognitive load and item response theory could be used to conduct patient safety research in other domains.

Introduction and validation of the open symptom framework: a public domain modular framework for patient-reported measurement of symptoms related to cancer and its treatment

PurposeWe provide an initial description and validation of some public domain patient-reported outcome (PRO) items to assess cancer symptom burden to address immediate barriers to symptom assessment use in clinical practice and facilitate future research.MethodsWe created the Open Symptom Framework (OSF), a flexible tool for clinical cancer-related symptom assessment. The items comprise six components: recall period, concept, symptom, qualifier(s), a definition, and a 5-point Likert-type response. We recruited patients receiving cancer therapy in the United States and United Kingdom. We assessed external construct validity by comparing OSF scores to the PRO-CTCAE measure and assessed reliability, scalability, dimensionality, and item ordering within a non-parametric item response theory framework. We tested differential item functioning for country, age, gender, and level of education.ResultsWe developed a framework alongside clinical and psychometric experts and debrieifed with 10 patients. For validation, we recruited 331patients. All items correlated with the PRO-CTCAE equivalents (r = 0.55–0.96, all p < 0.01). Mokken analysis confirmed the scalability and unidimensionality of all symptom scales with multiple items at the scale (Ho = 0.61–0.75) and item level (Hi = 0.60–0.76). Items are interpreted consistently between demographic groups (Crit = 0 for all groups).ConclusionThe public domain OSF has excellent psychometric properties including face, content, and criterion validity and can facilitate the development of flexible, robust measurements to fulfil stakeholder need. The OSF was designed specifically to support clinical assessment but will function well for research. Further work is planned to increase the number of symptoms and number of questions per symptom within the framework.

Automated Scoring of Constructed Response Items in Math Assessment Using Large Language Models

Automated Scoring of Constructed Response Items in Math Assessment Using Large Language Models

Internal Structure of Dietary Habits as a Restriction on Healthy Eating Policy in Japan.

Although promoting healthy eating is a policy objective, the manageability of dietary habits remains uncertain. Personal dietary patterns reflect many factors, some of which are relatively manageable for individuals whilst others are not. In this article, assuming that some sort of information about the manageability of dietary habits is contained in the observed patterns of food consumption, we focused on dietary patterns on their own. We introduced a statistical descriptive model for data from a food frequency questionnaire, estimated the strength of pairwise linkage between foodstuffs, and grouped foodstuffs by applying community detection to the networks of the estimated inter-food linkages. Those linkages represent the co-movement of pairs of food in consumption. Furthermore, we demonstrated an analysis of the relationship between mental health and dietary habits, considering the aspect of the manageability of dietary habits. Using an observational study in Japan, we obtained the following results: 115 foodstuffs were divided into three groups for both genders, but the compositions were different by gender; in the analysis of mental and physical health, some stress response items were associated with a dependence on some of those food groupings (e.g., "extremely tired" was negatively associated with a group containing tomatoes, cucumber, mandarin, etc., for female subjects). As the grouping of foodstuffs based on our estimation depicted an internal structure of dietary habit that a healthy eating policy could regard as a constraint, it follows that we should design such a policy along the same lines as that grouping.

Methods for measuring career readiness of high school students: based on multidimensional item response theory and text mining

In contemporary society, career readiness holds paramount significance for individual life, exerting a direct influence on initial employment, job satisfaction, and the sense of career identity. Framed within multidimensional item response theory and text mining, this study embarks on exploring assessment methodologies for high school students’ career readiness by revising the “Career Readiness Questionnaire – Adolescent Version” and employing text mining techniques. Study One collected 1261 valid data points through cluster sampling. With the aid of Bayesian multivariate item response theory parameter estimation procedures and R language, the career readiness measurement tool was revised, yielding a concise scale that aligns with psychometric requirements. The research findings indicated that the concept of “career readiness” is more suitable for the multidimensional graded response model than for the bifactor model. The dataset’s discrimination parameters fell within the range of [1.59, 3.84], the difficulty parameters fell between [−2.91, 2.24], and the peak values of the maximum information functions fell within [0.24, 2.35]. After six items with the lowest peaks were removed (Items 4, 5, 6, 31, 32, and 33), the remaining 30 items composed the Chinese concise version “Career Readiness Questionnaire – Adolescent Version,” with discrimination parameters ranging from [1.45, 3.38], difficulty parameters between [−3.31, 1.76], and maximum information function peaks within [0.50, 2.64]. Building upon the effective participants from Study One, Study Two matched questionnaire data with textual information, resulting in 1012 valid participants. Leveraging text mining, a machine learning model was constructed to predict high school students’ career readiness based on essay texts. The results of Study 2 prove that the revised lexicon was more accurate in feature extraction. Building upon this, the machine learning model for essay text demonstrated excellent performance in predicting career readiness, with random forest outperforming the other algorithms. This study provides a novel approach for schools and parents to comprehend the state of career readiness among high school students, offering a convenient and effective tool for educational activities related to students’ career development.

Validation of the Maslach Burnout Inventory-General Survey 9-item short version: psychometric properties and measurement invariance across age, gender, and continent.

The Maslach Burnout Inventory-General Survey (MBI-GS) stands as the preeminent tool for assessing burnout across various professions. Although the MBI-GS9 emerged as a derivative of the MBI-GS and has seen extensive use over several years, a comprehensive examination of its psychometric properties has yet to be undertaken. This study followed the Standards for Educational and Psychological Testing guidelines to validate the MBI-GS9. Employing a combined approach of classical test theory and item response theory, particularly Rasch analysis, within an integrated framework, the study analyzed data from 16,132 participants gathered between 2005 and 2015 by the Centre for Organizational Research at Acadia University. The findings revealed that the MBI-GS9 exhibited satisfactory reliability and validity akin to its predecessor, the MBI-GS. Across its three dimensions, Cronbach's α and omega coefficients ranged from 0.84 to 0.91. Notably, the MBI-GS9 displayed no floor/ceiling effects and demonstrated good item fit, ordered threshold, acceptable person and item separation and reliability, clear item difficulty hierarchy, and a well-distributed item threshold. However, the results suggested a recommended minimum sample size of 350 to mitigate potential information loss when employing the MBI-GS9. Beyond this threshold, the observed mean difference between the MBI-GS and MBI-GS9 held minimal practical significance. Furthermore, measurement equivalence tests indicated that the MBI-GS9 maintained an equivalent three-factor structure and factor loadings across various gender, age, and continent groups, albeit with inequivalent latent values across continents. In sum, the MBI-GS9 emerges as a reliable and valid alternative to the MBI-GS, particularly when utilized within large, diverse samples across different age and gender demographics. However, to address potential information loss, a substantial sample size is recommended when employing the MBI-GS9. In addition, for cross-cultural comparisons, it is imperative to initially assess equivalence across different language versions at both the item and scale levels.

Improved Scoring of the Center for Epidemiologic Studies Depression Scale – Revised: An Item Response Theory Analysis

The Center for Epidemiologic Studies Depression Scale – Revised (CESD-R) is a popular self-report screening measure for depression. A 20-item questionnaire with scores ranging from 0 to 4 for each item, the CESD-R can produce total scores ranging from 0 to 80. However, the typical scoring protocol for the CESD-R restricts the range of possible scores to between 0 and 60 to retain the same range and clinical cutoff scores as the original CES-D. Despite the widespread adoption of this scoring approach, the psychometric impact has never been systematically examined. In an undergraduate and community adult sample (n = 869), item response theory analyses indicated that scoring the CESD-R with all 5 response options (CESD-R5opt) provided nearly twice as much information about a person’s latent depression for individuals with high levels of depression than did scoring the CESD-R with 4 response options per item (CESD-R4opt). The CESD-R5opt retained the strong reliability and factor structure of the CESD-R4opt and was more sensitive to individual differences for participants at high levels of depression compared to the CESD-R4opt. Results provide preliminary evidence that researchers and clinicians should score the CESD-R using the full 0-to-80 scale and a clinical cutoff score of 29.

Evaluating Item Response Format and Content Using Partial Credit Trees in Scale Development

Abstract The type of response options selected for items on a survey, along with how many response options to include and whether to allow neutral midpoints, impacts data obtained from survey collections and the interpretations made using the results. Further, if subgroups within a population (e.g., racial/ethnic, gender, age) interpret response options differently, this variance can artificially inflate non-significant differences or mask true differences between groups. In this study, we apply two recursive partitioning procedures for investigating differential item functioning (DIF) in an experiment evaluating seven item response formats (five levels of an agree–disagree [AD] format and two levels of an item-specific [IS] format). Partial credit tree procedures allow for the evaluation of multiple covariates without prespecifying subgroups to be compared. We applied the procedures to items measuring adults’ attitudes toward legal abortion and all response formats functioned without DIF for age, gender, race, education, and religion when evaluated using global DIF screening approaches. Item-focused analyses indicated that odd-numbered response formats were less susceptible to content-based DIF. The combination of psychometric properties indicated that five-point AD and IS formats may be preferable for abortion attitude measurement based on the screening procedures conducted in this study.

Harmonizing the CBCL and SDQ ADHD scores by using linear equating, kernel equating, item response theory and machine learning methods.

A problem that applied researchers and practitioners often face is the fact that different institutions within research consortia use different scales to evaluate the same construct which makes comparison of the results and pooling challenging. In order to meaningfully pool and compare the scores, the scales should be harmonized. The aim of this paper is to use different test equating methods to harmonize the ADHD scores from Child Behavior Checklist (CBCL) and Strengths and Difficulties Questionnaire (SDQ) and to see which method leads to the result. Sample consists of 1551 parent reports of children aged 10-11.5 years from Raine study on both CBCL and SDQ (common persons design). We used linear equating, kernel equating, Item Response Theory (IRT), and the following machine learning methods: regression (linear and ordinal), random forest (regression and classification) and Support Vector Machine (regression and classification). Efficacy of the methods is operationalized in terms of the root-mean-square error (RMSE) of differences between predicted and observed scores in cross-validation. Results showed that with single group design, it is the best to use the methods that use item level information and that treat the outcome as interval measurement level (regression approach).

Heterogeneity of Item-Treatment Interactions Masks Complexity and Generalizability in Randomized Controlled Trials

Researchers compare aggregate test data across treatment and control group in randomized controlled trials to determine intervention impact on student achievement. We show that item-level data and analyses can provide information about item-level treatment effect heterogeneity; such findings may be useful for improving interventions and making decisions about outcomes in subsequent studies. We apply Differential Item Functioning techniques to examine variation in the degree to which items show treatment effects. Based on our analysis of 7,244,566 item responses (265,732 students responding to 2,119 items) from 15 RCTs, we find evidence for variation in gains across items. These analyses identify items that are highly sensitive to the interventions—in one case, a single item drives nearly 40% of the observed treatment effect. Our analyses also identifies items that are insensitive and cases wherein treatment effects are consistent across items. We also show that the variation of item-level treatment sensitivity has implications for effect estimate precision. Of the treatments with significant effect estimates, 31% have patterns of item-level treatment sensitivity that may lead to a null effect when considering this source of uncertainty.

Development and validation of the Health Literacy Index for the Community (HLIC) for the Korean National Health and Nutrition and Examination Survey.

We developed and validated the Health Literacy Index for the Community (HLIC) to assess the health literacy of the Korean population within the framework of the Korean National Health and Nutrition and Examination Survey. The HLIC was developed through 1) defining the conceptual framework and generating the item pool and 2) finalizing the items and identifying the cutoff value. Interviews were conducted to examine items' face validity, and a cross-sectional survey was performed to analyze the item-response theory and Rasch models to investigate the instrument's psychometric properties. In this study of 1,041 participants, most had no difficulty understanding health information; however, 67.9% struggled to assess the reliability of health information from the internet or media. A 4-factor structure was identified through factor analysis, leading to the exclusion of some items. This resulted in 10 items across 4 domains: 1) disease prevention, 2) health promotion, 3) health care, and 4) technology and resources. The HLIC demonstrated good internal consistency, with a Cronbach's ɑ of 0.87. It also showed high test-retest reliability and correlations with other health literacy instruments. A sociodemographic analysis of the HLIC revealed disparities in health literacy across various age groups, education levels, and income brackets. The HLIC was developed to systematically measure health literacy in Korea's general population. Its simplicity and conciseness ensure reliability and validity and improve its accessibility, making it particularly suitable for the broader Korean population, including those with lower literacy levels.

Initial development and validation of item banks to measure problematic hypersexuality.

Problematic Hypersexuality (PH) is defined as a distress caused by hypersexuality, to the extent that seeking treatment is considered. PH was previously measured with instruments stemming from different perspectives on problems related to hypersexuality. These instruments might best be analyzed in unison to discover the most optimal set of characteristics to measure PH. A total of 58 items were investigated with Item Response Theory (IRT). We included 1211 participants (592 women, 618 men, 1 other) from a representative Dutch general population sample of 18 years or older. In addition, 371 participants (116 women, 253 men, 2 other) in a web-based survey who sought information on their current level of PH were included. This latter group was divided into those that did or did not consider treatment and group differences in item averages were assessed. After item selection, 26 out of 58 items were retained and divided in two scales: Emotion Dysregulation-PH - 9 items representing the distressing emotional patterns coinciding with hypersexual preoccupation - and Negative Effects-PH - 17 items representing the negative consequences of patterns of hypersexual thoughts and behavior. Assumptions for IRT analyses were met (unidimensionality, local independence and monotonicity). After an IRT graded response model was fit, the scales showed sufficient reliability for the target population of hypersexual individuals. In the general population the scales showed large floor effects and were less reliable. With this study a first step is taken in validating two complementary item banks to measure PH. Further development of the item banks should include the investigation of responsiveness. New items should be constructed to assess less-explored areas of PH and improve differentiating power of the scales. This study showed that diagnostic accuracy for PH is currently difficult to attain with a survey, even when using an extended item set representing the most unique characteristics of PH.