Comprehension Items Research Articles

Information-seeking and decision-making preferences among adult orthodontic patients: an elective health care model

When it comes to their own health care, adult patients traditionally demonstrate strong information-seeking desire but a somewhat lower desire to make their own treatment decisions in nonelective situations. Little is known about these desires in patients facing elective health care situations. We used the well-tested Autonomy Preferences Index (API) as a base to construct and test our elective Autonomy Preferences Index (eAPI) for both information-seeking and decision-making and analyzed demographic variables on both. The eAPI was constructed to mirror the API but uses elective scenarios rather than the API's nonelective scenarios. It was validated using cognitive interviews to determine item intent and comprehension and by Cronbach's alpha. Both the API and eAPI were distributed to 188 active-treatment patients at the Division of Orthodontics, University of Minnesota. API and eAPI items were scored using a 1 (low) to 5 (high) Likert scale of desire. Mean information-seeking desire was universally high (>4, P < 0.001) for both API and eAPI instruments. Mean decision-making (DM) desire was universally low to moderate: API-DM = 2.84 and eAPI-DM = 2.6. Decision-making preferences for nonelective items (API-DM) decreased as the condition severity presented in the vignettes increased: mild = 2.88, moderate = 2.67 and severe = 2.21. Conversely, elective decision-making preferences (eAPI-DM) increased with increasing condition severity: mild = 2.51, moderate = 2.79 and severe = 3.18 (P < 0.001). Adult patients have universally high information-seeking preferences and moderate to low decision-making preferences regardless of the elective or nonelective nature of their condition. However, as vignette condition severity increases, patients facing nonelective scenarios display progressively less desire for decision-making, whereas patients facing elective scenarios show progressively more decision-making desire.

A Comparative Analysis of MEXT English Reading Textbooks and Japan’s National Center Test

Despite the influence of changing demographics in Japan, the National Center Test for University Entrance Exams continues to assert an ever increasing role in the process of university admissions. In preparation for this examination, the majority of senior high school students learn from textbooks approved by the Japanese Ministry of Education, Culture, Sports, Science, and Technology (MEXT). This study compares Section Six of the 2003-2007 and 2008-2009 National Center Test English reading examination with three widely used MEXT senior high school reading textbooks. By triangulating analyses of vocabulary, reading ease/grade level indices, reading skills, and comprehension item formats, thereby expanding the methodologies of previous research in this area, the study poses the question:To what degree do the text and comprehension tasks of MEXT textbooks compare with those which students will be expected to comprehend in the National Center Test reading examination? Updating the results of earlier studies, the descriptive and inferential statistics reveal that while the vocabulary levels of the textbook now appear to correspond more closely with recent versions of the examination, further alignment of readability, grade levels, and comprehension tasks is necessary if MEXT textbooks are to support the development of reading proficiency now required.

Item Bank Development for a Revised Pediatric Evaluation of Disability Inventory (PEDI)

ABSTRACTThe Pediatric Evaluation of Disability Inventory (PEDI) is a useful clinical and research assessment, but it has limitations in content, age range, and efficiency. The purpose of this article is to describe the development of the item bank for a new computer adaptive testing version of the PEDI (PEDI-CAT). An expanded item set and response options were reviewed by clinician experts and examined at parent and clinician focus groups. Eleven parents participated in 32 cognitive interviews to examine content, format, and comprehension of items and responses. A set of 76 self-care, 78 mobility, and 64 social function items with pictures and a four-point “Difficulty” scale were developed. The PEDI's Caregiver Assistance scale was replaced by a “Responsibility Scale” with 53 items. Content validity was established incorporating input from clinicians and parents. The new item bank covers a broad range of functional activities for children of all ages and abilities.

Development of a functional assessment of chronic illness therapy questionnaire to assess dermatology-related quality of life in patients treated with EGFR inhibitors: FACT-EGFRI.

9046 Background: Dermatologic toxicities (dT) to epidermal growth factor receptor inhibitors (EGFRI) are common and impact quality of life (QoL), which may lead to dose reduction or discontinuation of anticancer therapy. Currently, there are no patient reported outcomes measures (PRO) that assess QoL related to dT from EGFRI treatment. This study describes the development of a PRO questionnaire for dT to EGFRI. Methods: A three-phase process was utilized for item generation, reduction and scale construction. First, a comprehensive literature review was performed and all questions with potential relevance for patients with dT were selected for further evaluation in the form of 62 EGFRI-dT specific items, including some from dermatologic QoL instruments such as the Skindex. Next, a survey and evaluation of these potential items by 12 dT expert physicians and nurses and 20 patients was completed. Finally, the pilot questionnaire was administered to 24 patients with various stages of dT. Item comprehension, relevance and overall content was assessed in semi- structured interviews. Results: In qualitative interviews, patients and experts evaluated the importance of 62 items from the literature and the Skindex. Comments and response frequencies were summarized. Items identified as “quite relevant” or “very relevant” by the majority of experts were retained. Item review and reduction resulted in a 38-item pilot questionnaire. A majority of patients determined the content of the pilot questionnaire to be relevant, comprehensive and easy to understand. Patient input resulted in the omission of 3 items, modification of 6 items, and the addition of 4 items. Final revisions resulted in the FACT-EGFRI, consisting of 18 items encompassing well-being in 3 QoL domains: physical (7 items), social/emotional (6 items), and functional (5 items). Conclusions: The face and content validity of the FACT-EGFRI has been confirmed by EGFRI-dT patients and professionals. Formal validation and reliability testing is planned in a prospective cohort of EGFRI-induced patients with all dT severity grades. The FACT-EGFRI is available at www.facit.org. Author Disclosure Employment or Leadership Position Consultant or Advisory Role Stock Ownership Honoraria Research Funding Expert Testimony Other Remuneration Amgen, Genentech, ImClone Systems, Onyx Amgen, Bayer, Bristol-Myers Squibb, ImClone Systems, Onyx Hana Biosciences

Development of a Japanese Version of the Care Planning Assessment Tool

To develop a Japanese version of the Care Planning Assessment Tool (J-CPAT), originally developed in Australia as a comprehensive assessment of people with dementia. The process of adapting the CPAT into Japanese included translation into Japanese, assessment of item comprehension, and a validity and reliability study. The J-CPAT is composed of eight domains: Communication, physical problems, self-help skills, confusion, behaviour, social interaction, psychiatric observations and carer dependency. The participants were 199 aged care clients. Measures were the J-CPAT, Mini-Mental State Examination (MMSE) and Care Levels used in the Long-term Care Insurance scheme. Cronbach's alpha values in each J-CPAT domain were 0.74-0.95. The correlation coefficient between the score of Confusion and MMSE was -0.90, and those between physical problems, self-help skills, carer dependency in the J-CPAT, and care level were 0.70, 0.75 and 0.67. The J-CPAT appears to be a reliable and valid tool for care planning in Japan.

UMA ANÁLISE DA VALIDAÇÃO E CONFIABILIDADE DA ESCALA DE OPINIÕES DA SELEÇÃO DE EXPERIMENTOS DE QUÍMICA (EOSEQ)

O propósito desta investigação foi de explorar, interpretar e descrever o processo de validação e confiabilidade da escala de opiniões da seleção de experimentos, que proporciona medidas das convicções dos professores quando escolhem os experimentos. A escala do tipo Likert foi administrada a 113 participantes. Para a validação do conteúdo, utilizamos as técnicas do pré-teste e teste-piloto até a versão final que pelo nosso consenso com peritos da área, alcançamos bons resultados. As técnicas da forma paralela e Spearman verificaram a estabilidade no tempo, indicando resultados positivos de confiabilidade. O alfa de Cronbach item total de 0,80, 0,85 e 0,7 para geral, os professores e licenciandos, respectivamente, considerando a compreensão e variabilidade dos itens pertinentes ao serem interpretáveis.

Long term understanding of study information in research participants with Parkinson's disease

Context Little is known about research participants' understanding of consent information over the course of a clinical study and the relationship of this information with participant behavior. Methods We conducted a cross sectional patient completed questionnaire of comprehension and satisfaction administered at the end of a Parkinson's disease clinical trial. Main outcome Scores on 9 comprehension items in a 30 item questionnaire covering the key elements of informed consent. Results 78% of eligible trial participants completed this substudy. Greater than 90% of respondents showed good comprehension of the study purpose, method of treatment assignment, experimental nature of drugs, voluntary participation, and expected effect of the trial on their PD. However, 42.3% of subjects incorrectly endorsed that participating in the study was part of the “usual treatment” for their PD. We found no relationship between comprehension, compliance, and satisfaction with whether or not one's own neurologist was also the study doctor. Years of education and cognitive function at baseline were correlated with comprehension of study information. Conclusions Overall comprehension of key study information presented in the consent was high after 12 months of trial participation, although there were inconsistencies in responses that need further study.

They Read, but How Well Do They Understand?

Much of the literature on reading development focuses on measures of early literacy skills (e.g., phonological awareness, phonics, fluency). Elementary educators interested in improving students’ skills in these areas can draw on a wealth of research studies. However, many studies of early literacy skills have not addressed comprehension, obviously an important feature of literacy. The authors used a one-parameter Rasch model to examine the relative difficulty of different multiple-choice reading comprehension items assessing students’ literal, inferential, and evaluative comprehension of fictional narratives. They also examined the difficulty of questions derived from concepts identified by state content standards as important components of reading comprehension, such as character, prediction, and plot sequence. The findings suggest a curvilinear relationship between literal, inferential, and evaluative comprehension, with literal comprehension being the easiest and inferential and evaluative comprehension more challenging. The findings also indicate that assessment objectives differ on the basis of difficulty. Implications of these findings for practice are discussed.

Differential Item Functioning Analysis of the Preschool Language Scale—4 Between English-Speaking Hispanic and European American Children From Low-Income Families

The study examined whether item bias is present in the Preschool Language Scale—4 (PLS-4). Participants were 440 children (3—5 years old; 86% English-speaking Hispanic and 14% European American) who were enrolled in Head Start programs. The PLS-4 items were analyzed for differential item functioning (DIF) using logistic regression and a one-parameter item response theory (IRT) model. The logistic regression analysis identified Auditory Comprehension (AC) Item 35 as having non-uniform DIF. Using IRT, AC Item 55 and Expressive Communication (EC) Items 30 and 31 displayed statistical evidence of DIF. AC Item 55 and EC Item 30 were more difficult for Hispanic children than for European American children, whereas the opposite was true for EC Item 31. These findings did not find strong evidence of cultural bias for the PLS-4 against English-speaking Hispanic children from low-income families.

Evaluating Descriptive Richness in Collection-Level Metadata

ABSTRACT When many collections are 1 brought together in a federation or aggregation, the attributes of the original collections can become difficult to discern. Collection-level 2 metadata has the potential to provide important context about the purpose and features of individual collections, but the qualitative aspects of collections are difficult to describe in a systematic way. This article reports on a content analysis of collection records in the Digital Collections and Content (DCC) aggregation, conducted to analyze the kinds of substantive and purposeful information represented across 202 cultural heritage collections. We found that the free-text Description field often provides more accurate and complete representation of subjects and object types than the specified fields; it consistently represents properties such as uniqueness, importance, comprehensiveness, provenance, and creator of items in digital collection, and other vital contextual information about the intentions of collectors and the value of collections for scholarly users. The results show that free-text collection metadata can be both concise and semantically rich, and can provide a valuable source of data for enhancing and customizing controlled vocabularies.

Cognitive interviewing methodology in the development of a pediatric item bank: a patient reported outcomes measurement information system (PROMIS) study

BackgroundThe evaluation of patient-reported outcomes (PROs) in health care has seen greater use in recent years, and methods to improve the reliability and validity of PRO instruments are advancing. This paper discusses the cognitive interviewing procedures employed by the Patient Reported Outcomes Measurement Information System (PROMIS) pediatrics group for the purpose of developing a dynamic, electronic item bank for field testing with children and adolescents using novel computer technology. The primary objective of this study was to conduct cognitive interviews with children and adolescents to gain feedback on items measuring physical functioning, emotional health, social health, fatigue, pain, and asthma-specific symptoms.MethodsA total of 88 cognitive interviews were conducted with 77 children and adolescents across two sites on 318 items. From this initial item bank, 25 items were deleted and 35 were revised and underwent a second round of cognitive interviews. A total of 293 items were retained for field testing.ResultsChildren as young as 8 years of age were able to comprehend the majority of items, response options, directions, recall period, and identify problems with language that was difficult for them to understand. Cognitive interviews indicated issues with item comprehension on several items which led to alternative wording for these items.ConclusionChildren ages 8–17 years were able to comprehend most item stems and response options in the present study. Field testing with the resulting items and response options is presently being conducted as part of the PROMIS Pediatric Item Bank development process.

Children's reports of activity and participation after sustaining a spinal cord injury: A cognitive interviewing study

Objective: To evaluate the readability, comprehension and response processes of test items designed to measure activity performance and participation by children with spinal cord injury (SCI).Methods: Eleven parents and 33 children participated. Descriptive statistics were used to calculate frequency of problems based on age, grade, domain and response scale. Agreement was evaluated by weighted kappa coefficient values.Results: Most (54%) of the problems were due to reading/comprehension, with the majority by children 8 years of age or younger (56%) who had not completed 2nd grade (51%). Agreement between child–parent reports ranged from poor–good, with strongest agreement for mobility items and weak agreement for chores, self-care and participation.Conclusion: Children with SCI 8 years of age and older who have competed 2nd grade are able to read, understand and respond to items associated with activity performance and participation.

“Everyone Has a Right to, Like, Check Their Box:” Findings on a Measure of Gender Identity from a Cognitive Testing Study with Adolescents

Efforts to monitor the health of transgender youth, a small but high-risk population, are hindered by a lack of knowledge about how to accurately measure gender identity. Adolescents (n = 30) participated in semistructured qualitative interviews after completing a close-ended transgender-inclusive measure of gender. Interviews explored item comprehension and respondent burden. Participants, who were diverse in age (range = 15-21), gender identity, sexual orientation, and race-ethnicity, were accurately classified as male, female, or transgender. All youth understood transgender as a difference between the physical body and a person's internal sense of self. Nontransgender youth frequently used an example (a woman in a man's body) in their explanations and were largely supportive of the transgender options. Most transgender youth found a response option that they felt was appropriate. Transgender response options were added to a gender measure without impacting the accuracy of nontransgender responses or burdening the nontransgender adolescents in our sample. A modified measure (Gender: male; female; transgender, male-to-female; transgender, female-to-male; transgender, do not identify as exclusively male or female) is recommended for testing in samples that vary by age, race-ethnicity, socioeconomic status, language, and geography. Additional suggestions for research in this area are provided.

An Investigation of the Relationship Between Health Literacy and Social Communication Skills in Older Adults

The purpose of this study was to examine connections between health literacy and social communication skills in older adults, a population that experiences chronic health conditions but is reported to have low health literacy and declines in communication skills. Sixty-three older adults were administered the Social Communication subtest of the Functional Assessment of Communication Skills in Adults and the Short Test of Functional Health Literacy in Adults. Correlational statistics indicated significant relationships between social communication skills and health literacy in older adults. Aspects of social communication that involve initiating discourse, facial expressions, and voice tone were positively related to reading comprehension items that involve informing, naming, and requesting information. Results of the study suggest that social communication skills may possibly serve as predictors of health literacy. Further study of the potential relationships between these variables is needed to improve health literacy assessment and intervention for older adults.

Video Listening Tests: What Are They Measuring?

This article examines how eight advanced ESL learners attend to and utilize the nonverbal information found in a video text. Using verbal report methodology in which test-takers verbalized aloud their thought processes while taking a video listening test, the results suggested that the test-takers varied in how they attended to and utilized the nonverbal information provided by the video texts. In addition, the data provided evidence that individual test-takers varied in their ability to utilize the nonverbal information both to make meaning of the spoken text and to answer the comprehension items associated with the text. The implications for a construct definition of listening ability that includes the ability to utilize the nonverbal information found in most spoken texts is discussed.

Psychological inflexibility in childhood and adolescence: Development and evaluation of the Avoidance and Fusion Questionnaire for Youth.

The authors describe the development and validation of the Avoidance and Fusion Questionnaire for Youth (AFQ-Y), a child-report measure of psychological inflexibility engendered by high levels of cognitive fusion and experiential avoidance. Consistent with the theory underlying acceptance and commitment therapy (ACT), items converged into a 17-item scale (AFQ-Y) and an 8-item short form (AFQ-Y8). A multimethod psychometric approach provides preliminary support for the reliability and validity of the AFQ-Y and AFQ-Y8. In 5 substudies, 3 samples (total N = 1369) were used to establish (a) item comprehension (n = 181), (b) initial item selection (n = 513), (c) final item reduction and development of a short form for research (n = 346), (d) comprehensive psychometric evaluation of the AFQ-Y and AFQ-Y8 (n = 329), and (e) convergent and construct validity for both versions of the AFQ-Y. Overall, results suggest that the AFQ-Y and AFQ-Y8 may be useful child-report measures of core ACT processes.

Using Cognitive Interviewing for Test Items to Assess Physical Function in Children with Cerebral Palsy

The purpose of this study was to assess the content, format, and comprehension of test items and responses developed for use in a computer adaptive test (CAT) of physical function for children with cerebral palsy (CP). After training in cognitive interviewing techniques, investigators defined item intent and developed questions for each item. Parents of children with CP (n = 27) participated in interviews probing item meaning, item wording, and response choice adequacy and appropriateness. Qualitative analysis identified 3 themes: item clarity; relevance, context, and attribution; and problems with wording or tone. Parents reported the importance of delineating task components, assistance amount, and environmental context. Cognitive interviewing provided valuable information about the validity of new items and insight to improve relevance and context. We believe that the development of CATs in pediatric rehabilitation may ultimately reduce the impact of the issues identified.

日本語版Care Planning Assessment Toolの作成と信頼性&amp;middot;妥当性の検討

To develop a Japanese version Care Planning Assessment Tool (CPAT) , a comprehensive geriatric assessment instrument for dementia care, originally developed in Australia. The J-CPAT is composed of 61 items, and 8 domains: Communication, Physical problems, Self-help skills, Confusion, Behavior, Social interaction, Psychiatric observation, and Care dependency. The development of the J-CPAT included translation into Japanese, assessment of item comprehension, back translation, production of final version, and its validity and reliability analysis. Through this process, an item about family interaction was added in the J-CPAT. The participants were 199 clients in residential and day care. Raters were professional carers with 2.5 hours training about the J-CPAT. We assessed the validity of the J-CPAT, using correlation between the scores of the J-CPAT, MMSE, NM-scale, N-ADL, and levels of care needs (Kaigodo). Ten pairs of carers were included in the inter-rater reliability analysis. Fourteen carers participated in the intrarater reliability study. Cronbach's alpha values in each J-CPAT domain were 0.74-0.95. Mean difference of each domain between test and retest was 0.4-3.6%. Weighted kappa values for all items for 10 pairs of raters were over 0.6. The correlation coefficients between the domain score of 'Confusion' in the J-CPAT and MMSE was -0.90 (p<0.01). Those between the domain scores of 'Physical problems', 'Self-help skills', 'Dependency on care' in the J-CPAT, and Kaigodo were sufficient (>0.62), and those between the item scores of the J-CPAT, NM-scale, and N-ADL were relevant (>0.60). The J-CPAT is considered to be an appropriate assessment instrument for dementia care in Japan.

ESL Reading Strategies: Differences in Arabic and Mandarin Speaker Test Performance

This study was undertaken to test the hypothesis that reading comprehension items, which elicit specific bottom‐up and top‐down strategies, favor certain linguistic/cultural groups. Verbal report data were collected from Arabic‐ and Mandarin‐speaking English as a second language (ESL) learners to identify the reading strategies involved in carrying out 32 reading questions. Then a confirmatory approach to differential item functioning was used to determine whether bottom‐up and top‐down items functioned differentially for equal‐ability Arabic and Mandarin ESL learners. Results revealed systematic group performance differences in four bottom‐up and three top‐down strategy categories. Items involving breaking a word into smaller parts, scanning, paraphrasing, and matching were found to favor Mandarin speakers, whereas items involving skimming, connecting, and inferring were found to favor Arabic speakers.

Lexical learning in sung and spoken story script contexts

Although most children seem to love music, our understanding of the role it plays in facilitating speech and language learning is limited, as is research validating its efficacy in the clinical setting. The purpose of this study was to examine how singing affects children’s quick incidental learning (QUIL) of novel vocabulary terms. Sixteen children with language delay and mild developmental delays were presented with spoken and sung story scripts containing eight novel words over two experimental sessions. Lexical probes indicated that children’s naming and comprehension of target lexical items did not significantly differ as a result of hearing sung versus spoken scripts versions. A significant increase was noted in the number of children’s unsolicited target word productions from Session 1 to Session 2 in the Sung Condition, indicating that sung input may enhance only particular aspects of quick incidental word learning.