Difficult Items Research Articles

What Do You Say When She Is No Longer Living With Cancer?

What Do You Say When She Is No Longer Living With Cancer?

Exploring company ability to meet supply chain security validation criteria

Purpose – The purpose of this paper is to develop a method to measure the difficulties of items required to achieve Authorized Economic Operator (AEO) validation and investigated companies’ abilities to obtain AEO certification based on an empirical analysis of 201 supply chain-related companies in Taiwan. Design/methodology/approach – The Rasch model was applied to convert the ordinal raw data collected from questionnaire surveys into values on an interval scale to measure companies’ abilities and item difficulties for AEO validation. The model was estimated using WINSTEP, which is an iterative computer program. Findings – The study results show that self-risk assessment and the formulation of security policies are the most difficult items to accomplish for AEO validation, whereas establishing security facilities is the easiest task to accomplish. Additionally, a company’s ability to obtain AEO validation was found to be positively correlated with a company’s turnover volume and its number of staff. Rese...

Rasch Analysis of the Routines-Based Interview Implementation Checklist

The Routines-Based Interview (RBI) is useful for developing functional outcomes/goals, for establishing strong relationships with families, and for assessing the family's true needs. In this study, the authors investigated the psychometric properties of the RBI Implementation Checklist, conducted by 120 early intervention professionals, specifically looking at the probability of correct responses on the items as a logistic function of the difference between the person and the item parameters. We selected Rasch analysis (Rasch, 1980) for this study so we could answer questions about both how the measure performed and how the interviewers performed and we related these performances to one another. Results indicate that scores on the RBI Implementation Checklist were reliable. The checklist could possibly benefit from more difficult items to measure the true performance of the few people who had scores higher than the most difficult items and also from additional items that focus on the family.

Improving the Measurement of Cognitive Ability in Geriatric Patients

Background/Aims: No single tool is available in geriatric clinical settings to quantitatively measure global cognitive ability at different stages ranging from normal functioning to severe impairment. We developed an adaptive test to measure cognitive ability in geriatric populations (Geriatric Rapid Adaptive Cognitive Estimate, GRACE); however, the test failed to discriminate between high-functioning individuals. This study aimed to estimate the extent to which adding more difficult items to the GRACE would improve measurement precision in the upper range of cognitive ability. Methods: The original data used to develop the GRACE was merged with newly collected data of patients who presented at local geriatric clinics. The Rasch analysis was used to estimate the difficulty level of the newly added items and evaluate whether the psychometric properties of the GRACE were improved. Results: One newly added item (Sequencing 5) had a higher difficulty level than all of the previous items in the GRACE. The rest of the new items were located in the high difficulty range. Conclusion: The psychometric properties of our adaptive screening tool were improved, and we were able to distinguish between individuals who had higher levels of cognitive functioning.

The development of an objective wine knowledge scale: the item response theory approach

Purpose– The purpose of this paper is to address the issue of objective knowledge operationalisation with specific focus on varying levels of scale items’ difficulty. The ultimate goal of the study was to develop a scale to measure objective wine knowledge, which would address the domain of wine knowledge and differentiate varying levels of consumer wine knowledge.Design/methodology/approach– The process of items’ development was guided by recommendations suggested by DeVellis (2003) in his influential work on theory and application of scale development. Examination of items’ performance was conducted through a series of field tests with consumer samples (N= 756) in a US wine region. Item response theory (IRT) approach was applied for items’ testing. The developed items were analysed using the two-parameter logistic model in Mplus Version 5.Findings– The study offers a 44-item test suitable for assessing wine knowledge across a broad spectrum of expertise. For example, if the goal is to assess wine knowledge differences among relatively knowledgeable respondents, a subset of more difficult items could be chosen. Alternatively, a test for novices could be constructed from easier scale’s items.Research limitations/implications– For researchers, the study offers conceptualisation of the wine knowledge domain, suggests a parsimonious instrument to measure the construct, offers a valid and reliable measure for use in testing theories of consumer knowledge and provides empirical evidence of the value and usefulness of the developed scale.Practical implications– For professionals, the proposed test may be used to test consumer knowledge and to help assess a prospective employee’s general knowledge of wine. The test can also be given at hospitality programs, outreach and continuing education programs.Originality/value– The current paper takes an alternative approach to classical test theory and offers an objective wine knowledge scale tested through IRT. This approach avoids shortcomings associated with classical measurement and offers an original scale that can discriminate among respondents with different levels of wine knowledge.

Adaptation of Collins' revised adult attachment dimensional scale to the Chilean context

The Adult Attachment Revised Scale is a measure of relationship attachment that contains three dimensions: closeness, dependence and anxiety. In Study 1, the scale was translated and applied to a sample of 420 university students in order to assess reliability. In addition, the scale criterion validity was estimated through measures of self-perception and affectivity. The results of Study 1 supported the internal consistency of two dimensions of attachment, but dependence was lower than expected (α = .62). A confirmatory factor analysis was conducted and failed to reach a convergent fit, thus demonstrating that two of the items in dependence were considered problematic. In Study 2, difficult items in Study 1 were reworded and the new version of the scale was applied to 81 participants. The results of two subsequent CFAs showed adequate fit of the scale and increased internal consistency with this new wording (α > .73), which replaced the expression depend on others by getting help from others, which is a better construct representation of the meaning of dependence in the context of attachment closeness. Therefore, not only did we adapt the three dimensions of the attachment scale, but also provided evidence of its validity related to locally adapted instruments.

Congruency effects on recognition memory: a context effect.

Two recent studies have reported that incongruent selective attention items are better remembered than congruent items on a surprise recognition memory test. These findings suggest that an increased need for cognitive control may trigger encoding mechanisms at the time of study that result in better recognition of those items at test, a form of the desirable difficulty effect. The experiments in this study demonstrate that this effect can depend on whether differences in selective attention difficulty are blocked or intermixed at the time of encoding. These results suggest that additional encoding time itself does not invariably result in better recognition for more difficult selective attention items. Instead, the dependence of recognition memory on encoding difficulty appears to reflect a context-sensitive control response to encoding difficulty.

Types of Student Errors in Mathematical Symbols, Graphs and Problem-Solving

This study aims to identify the level of difficulty, discrimination and errors based on symbols, graphs andproblem-solving in mathematics achievement. The participants consisted of 315 grade 10 students randomlyselected from eight secondary day schools situated in the state of Kedah, Malaysia. The mathematics written testconsisted of 15 open-ended items with five items each for symbols, graphs and problem-solving and all the itemswere analyzed using the difficulty and discrimination index. Semi-structured clinical interviews were alsocarried out among 20 selected students to identify the errors they had made in mathematics. In the contentanalysis, the descriptor code key was used to identify conceptual, careless, problem- solving and value errors.The findings showed that item 4 (Standard Form) of the symbols, item 6 (Linear Equation) of the graphs, anditem 15 (Line & Plane in 3 Dimension) of the problem-solving was the most difficult item in mathematics. All ofthese items had a good discrimination as well as difficulty index. Content analysis showed that 52 (57%)students made conceptual errors, 22 (24%) made careless errors, 12 (13%) made problem-solving errors and 5(6%) made value errors. The major reasons given for errors made were a lack of understanding, procedures beingforgotten, negligence in transcribing information from the question, carelessness and guesswork. This study hasimplications for the student learning process and understanding graphs because graphs are widely used in dailylife to manage, communicate and analyze information.

Scaling Cognitive Domains of the Montreal Cognitive Assessment: An Analysis Using the Partial Credit Model.

The psychometric properties of the Montreal Cognitive Assessment (MoCA) were examined by using the Partial Credit Model. The study sample included 897 participants who were distributed into two main subgroups: (I) the clinical group (90 patients with Mild Cognitive Impairment, 90 patients with Alzheimer's disease, 33 patients with Frontotemporal Dementia, and 34 patients with Vascular dementia, whose diagnoses were previously established according to a consensus that was reached by a multidisciplinary team, based on the international criteria) and (II) the healthy group (composed of 650 cognitively healthy community dwellers). The results show (i) an overall good fit for both the items and the persons' values, (ii) high variability for the cognitive performance level of the cognitive domains (ranging between 1.90 and -3.35, where "Short-term Memory" was the most difficult item and "Spatial Orientation" was the easiest item) and between the subjects on the scale, (iii) high reliability for the estimation of the persons' values, (iv) good discriminant validity and high diagnostic utility, and (v) a minimal differential item functioning effect related to of pathology, gender, age, and educational level. MoCA and its cognitive domains are suitable measures to use for screening the cognitive status of cognitively healthy subjects and patients with cognitive impairment.

Phonological facilitation effects on naming latencies and viewing times during noun and verb naming in agrammatic and anomic aphasia

Background: Phonological priming has been shown to facilitate naming in individuals with aphasia, as well as healthy speakers, resulting in faster naming latencies. However, the mechanisms of phonological facilitation (PF) in aphasia remain unclear.Aims: Within discrete vs. interactive models of lexical access, this study examined whether PF occurs via the sub-lexical or lexical route during noun and verb naming in agrammatic and anomic aphasia.Methods & Procedures: Thirteen participants with agrammatic aphasia and 10 participants with anomic aphasia and their young and age-matched controls (n = 20/each) were tested. Experiment 1 examined noun and verb naming deficit patterns in an off-line confrontation naming task. Experiment 2 examined PF effects on naming both word categories using eyetracking priming paradigm.Outcomes & Results: Results of Experiment 1 showed greater naming difficulty for verbs than for nouns in the agrammatic group, with no difference between the two word categories in the anomic group. For both participant groups, errors were dominated by semantic paraphasias, indicating impaired lexical selection. In the phonological priming task (Experiment 2), young and age-matched control groups showed PF in both noun and verb naming. Interestingly, the agrammatic group showed PF when naming verbs, but not nouns, whereas the anomic group showed PF for nouns only.Conclusions: Consistent with lexically mediated PF in interactive models of lexical access, selective PF for different word categories in our agrammatic and anomic groups suggest that phonological primes facilitate lexical selection via feedback activation, resulting in greater PF for more difficult (i.e., verbs in agrammatic and possibly nouns in anomic group) lexical items.

More is not Always Better: The Relation between Item Response and Item Response Time in Raven’s Matrices

The role of response time in completing an item can have very different interpretations. Responding more slowly could be positively related to success as the item is answered more carefully. However, the association may be negative if working faster indicates higher ability. The objective of this study was to clarify the validity of each assumption for reasoning items considering the mode of processing. A total of 230 persons completed a computerized version of Raven’s Advanced Progressive Matrices test. Results revealed that response time overall had a negative effect. However, this effect was moderated by items and persons. For easy items and able persons the effect was strongly negative, for difficult items and less able persons it was less negative or even positive. The number of rules involved in a matrix problem proved to explain item difficulty significantly. Most importantly, a positive interaction effect between the number of rules and item response time indicated that the response time effect became less negative with an increasing number of rules. Moreover, exploratory analyses suggested that the error type influenced the response time effect.

The influence of transcranial alternating current on fluid intelligence. A fMRI study

The aim of this study was to investigate the influence of transcranial alternating current stimulation (tACS) on fluid intelligence (gf). Furthermore, the study is the first to explore the relevance of the parietal cortex for fluid reasoning processes by means of tACS and fMRI. In a double-blind sham-controlled experiment 20 participants performed fluid intelligence tasks (matrices task, and paper folding) after theta transcranial alternating current stimulation was applied to the left parietal cortex. Stimulation induced brain activity changes were recorded during the task processing using fMRI. The results indicated a difficulty-specific stimulation effect: When solving difficult items of the matrices test verum tACS significantly increased fluid reasoning performance, as compared to sham tACS. No such difference was observed for items with easy item difficulty. Whole-brain analyses showed that left parietal brain stimulation was accompanied by less right sided activation in the areas of the frontal lobe, the fusiform gyrus and the occipital lobe, as well as left sided in the area of the middle occipital gyrus. Additional ROI analyses revealed a tendency for less activation in the left inferior parietal lobule. The results are discussed within the theoretical framework of the neural efficiency hypothesis as well as of the parieto-frontal-integration theory.

Oxytocin improves mentalizing – Pronounced effects for individuals with attenuated ability to empathize

The ability to predict the behavior of others based on their mental states is crucial for social functioning. Previous studies have provided evidence for the role of Oxytocin (OXT) in enhancing the ability to mentalize. It has also been demonstrated that the effect of OXT seems to strongly depend on socio-cognitive skills with more pronounced effects in individuals with lower socio-cognitive skills. Although recent studies indicate that mentalizing is related to empathy, no study has yet examined whether the effects of OXT on mentalizing depend on the ability to empathize. 71 male participants participated in a double-blind, between-subjects, placebo-controlled experiment. The Reading the Mind in the Eye Test (RMET) was used to investigate mentalizing abilities. We analyzed the effect of OXT on easy and difficult items of the RMET depending on differential empathy scores of the participants as assessed with the Empathy Quotient (EQ). Our results showed that OXT improves mentalizing for difficult but not for easy items. We generally observed increased mentalizing accuracy in participants with higher empathy scores. Importantly, however, whereas the performance in participants with higher empathy scores was comparable in both OXT and placebo condition, OXT specifically enhanced mentalizing accuracy in participants with lower empathy scores. Our findings suggest that OXT enhances mentalizing abilities. However, we also demonstrate that not all participants benefited from OXT application. It seems that the effects of OXT strongly depend on baseline social-cognitive skills such as empathy.

Disinfecting dental impressions: A necessity for all!

Before the impressions are dispatched to the dental laboratory, it is solely the dentist’s responsibility that they are disinfected in an appropriate manner. Uncertainty of impression disinfection risks both the health of the receiving dental technician and potential repeat disinfection of an already disinfected impression with detrimental consequences for its dimensions. Casts are one of the difficult prosthodontic items to be disinfected, and it is preferable to disinfect the impression so the resulting cast itself will not have to be disinfected. Therefore, proper disinfection of contaminated dental impressions and other dental items leaving the immediate chair‑side area remains the best approach to preventing the spread of infections in dentistry.

Development of Listening Comprehension Tests with Narrative and Expository Texts for Portuguese Students.

This investigation aimed to develop and collect psychometric data for two tests assessing listening comprehension of Portuguese students in primary school: the Test of Listening Comprehension of Narrative Texts (TLC-n) and the Test of Listening Comprehension of Expository Texts (TLC-e). Two studies were conducted. The purpose of study 1 was to construct four test forms for each of the two tests to assess first, second, third and fourth grade students of the primary school. The TLC-n was administered to 1042 students, and the TLC-e was administered to 848 students. The purpose of study 2 was to test the psychometric properties of new items for the TLC-n form for fourth graders, given that the results in study 1 indicated a severe lack of difficult items. The participants were 260 fourth graders. The data were analysed using the Rasch model. Thirty items were selected for each test form. The results provided support for the model assumptions: Unidimensionality and local independence of the items. The reliability coefficients were higher than .70 for all test forms. The TLC-n and the TLC-e present good psychometric properties and represent an important contribution to the learning disabilities assessment field.

Item analysis: An evaluation of multiple choice questions in Physiology examination

Background: Item analysis is the process of collecting, summarizing and using information from student’s responses to assess the quality of test items. Difficulty index and discrimination index are two parameters which help to evaluate the standard of multiple choice question (MCQ) used in an examination. Materials and Methods: An internal examination in Physiology was conducted at the end of one term. 148 students, out of 150, attended the exam. A total of 40 MCQ were given and analyzed. The data was analyzed by Microsoft Excel and recorded as mean ± standard deviation. The relationship between the difficulty index (P) and discrimination index (D) for each test item was determined by Pearson correlation analysis. Results: The mean difficulty index score was 0.54 ± 0.26 and mean discrimination score was 0.21 ± 0.14. On an average, about 10 of the MCQs were easy (P ≥ 70%) while 12 were difficult (P ≤ 30%) and the remaining 18 items were in acceptable range (P = 30-70%). In all, 2 items showed negative discrimination and 13 items exhibited poor discrimination. The remaining 25 items were in the range of acceptable to excellent discrimination. The discrimination index exhibited slight positive correlation with difficulty index (r = 0.3076 P = 0.05). However, it was not statistically significant. The maximal discrimination (d = 0.55) was observed with easy/difficult items (P = 0.6). Conclusion: The majority of the items were in acceptable range as far as difficulty and discrimination indices were concerned. Moderately easy/difficult items had the maximal discriminative ability. Too easy and too difficult items gave poor discrimination index.

Performance of Senior Teacher Education Students in a Pre-Licensure Test for General Education

The Licensure Examination for Teachers (LET) covers General Education and Professional Education components for the Elementary level examination and the General Education, Professional Education and Specialization components for the Secondary level examination. The study developed a 100-item pre-licensure test on the General Education component of the Teacher Education curriculum. The study looked into the performance of the students in the examination per subject area. It also looked into the discrimination and difficulty indices to assess the validity and reliability of the items as instruments to measure achievement. The respondents covered all students who took the examination. The study used descriptive – evaluative design. The performance of the students was measured through the percentage of students who got the correct answer. The data were treated statistically using means, percentages, and ranking. Item analysis was done to determine the indices of difficulty and discrimination while Kuder Richardson 21 was computed to determine the reliability of the test. The examinees registered a High performance. The highest performance of the respondents was in the Filipino component while the least performance is in Science. The easiest items per percentage of correct responses were in Filipino and Social Science while the most difficult item is in Science. Close to half of the items are of moderate difficulty. Only a few items have satisfactory discrimination. The study recommended the improvement of the test particularly on the choices to improve the reliability of the test. Learning areas where students performed poorly should be given focus on the teaching of the general education subjects.

The Interaction of Ability Differences and Guessing When Modeling Differential Item Functioning With the Rasch Model: Conventional and Tailored Calibration.

In educational testing, differential item functioning (DIF) statistics must be accurately estimated to ensure the appropriate items are flagged for inspection or removal. This study showed how using the Rasch model to estimate DIF may introduce considerable bias in the results when there are large group differences in ability (impact) and the data follow a three-parameter logistic model. With large group ability differences, difficult non-DIF items appeared to favor the focal group and easy non-DIF items appeared to favor the reference group. Correspondingly, the effect sizes for DIF items were biased. These effects were mitigated when data were coded as missing for item-examinee encounters in which the person measure was considerably lower than the item location. Explanation of these results is provided by illustrating how the item response function becomes differentially distorted by guessing depending on the groups' ability distributions. In terms of practical implications, results suggest that measurement practitioners should not trust the DIF estimates from the Rasch model when there is a large difference in ability and examinees are potentially able to answer items correctly by guessing, unless data from examinees poorly matched to the item difficulty are coded as missing.

C-83 * The Role of IQ in Effort Test Performance in Schizophrenia

Objective: Intelligence has recently been shown to predict performance on effort testing in healthy individuals, with lower IQ associated with higher rates of effort test failure. Similar results have been found in patients with epilepsy. However, the relationship between IQ and effort test performance has yet to be examined in psychiatric populations, including schizophrenia (SZ). The current study sought to address this question across two experiments. Method: Participants in Experiment 1 included 97 outpatients with SZ and 63 healthy controls who completed the Victoria Symptom Validity Test (VSVT). Experiment 2 included 46 outpatients with SZ and 33 CN. A very low rate of SZ failed effort testing (VSVT = 1%; WMT = 15%). Patients were separated into WASI full-scale IQ groups (60–79, 80–89, 90–99, 100+) and compared on demographics and effort test performance. Results: Higher current full-scale IQ was associated with higher personal and parental education, higher premorbid IQ, and greater likelihood of being male and Caucasian. IQ in the 60–79 or 80–89 range was associated with greater rates of failure on the WMT and lower scores on the VSVT difficult items. Patients falling above the 90 IQ range did not fail the WMT and had nearly perfect scores on the VSVT. Conclusion(s): Consistent with findings in healthy individuals and individuals with epilepsy, low IQ is a significant predictor of effort test performance in SZ. Our findings indicate that performance on effort tests, which are presumed to be relatively stable even in the face of intellectual deficits, may actually be influenced by intelligence.

The effects of theta transcranial alternating current stimulation (tACS) on fluid intelligence

The objective of the study was to explore the influence of transcranial alternating current stimulation (tACS) on resting brain activity and on measures of fluid intelligence. Theta tACS was applied to the left parietal and left frontal brain areas of healthy participants after which resting electroencephalogram (EEG) data was recorded. Following sham/active stimulation, the participants solved two tests of fluid intelligence while their EEG was recorded. The results showed that active theta tACS affected spectral power in theta and alpha frequency bands. In addition, active theta tACS improved performance on tests of fluid intelligence. This influence was more pronounced in the group of participants that received stimulation to the left parietal area than in the group of participants that received stimulation to the left frontal area. Left parietal tACS increased performance on the difficult test items of both tests (RAPM and PF&C) whereas left frontal tACS increased performance only on the easy test items of one test (RAPM). The observed behavioral tACS influences were also accompanied by changes in neuroelectric activity. The behavioral and neuroelectric data tentatively support the P-FIT neurobiological model of intelligence.