Forced-choice Questions Research Articles

The effect of forced-choice questions on children's suggestibility: A comparison of multiple-choice and yes/no questions.

The effect of forced-choice questions on children's suggestibility: A comparison of multiple-choice and yes/no questions.

Experiences with establishing and implementing learning management system and computer-based test system in medical college.

The purpose of this study was to establish an e-learning system to support learning in medical education and identify solutions for improving the system. A learning management system (LMS) and computer-based test (CBT) system were established to support e-learning for medical students. A survey of 219 first- and second-grade medical students was administered. The questionnaire included 9 forced choice questions about the usability of system and 2 open-ended questions about necessary improvements to the system. The LMS consisted of a class management, class evaluation, and class attendance system. CBT consisted of a test management, item bank, and authoring tool system. The results of the survey showed a high level of satisfaction in all system usability items except for stability. Further, the advantages of the e-learning system were ensuring information accessibility, providing constant feedback, and designing an intuitive interface. Necessary improvements to the system were stability, user control, readability, and diverse device usage. Based on the findings, suggestions for developing an e-learning system to improve usability by medical students and support learning effectively are recommended.

Sweet and Salt Pleasantness Are Not Related to Nutritional Status

The aim of this study was to investigate the relationship among nutritional status, taste sensitivity and pleasantness in overweight individuals. Threshold sensitivity of salt and sweet was determined by a two-alternative forced choice question involving each compound present at five different concentrations. Subjects received concentrated orange juice containing five different concentrations of sucrose (1%, 3%, 5%, 7% and 9% w / w).Pleasantness of salt was evaluated using mashed potato prepared at five different concentrations of salt 0%, 0.5%, 1.0%, 1.5% and 2.0% (w / w). A difference for sweet taste was observed between the groups studied, with lower sweet taste sensitivity thresholds detected in overweight subjects.

Repeated questioning in choice experiments: are we improving statistical efficiency or getting respondents confused?

This paper investigates options to obtain additional information about individual preferences in conjoint choice experiments when the number of respondents and the number of choice cards per respondent are fixed. We examine best/worst (or best/second best) repeated elicitation and forced-choice questions. Using simulations, we find that best/worst (or best/second best) produce the best efficiency gains over one-shot elicitation. Forced-choice questions also improve the statistical efficiency of the key estimates. We also use data from two empirical studies deploying best/worst (or best/second best) and forced-choice questions and find that the one with best/worst may yield undesirable response effects. The empirical study that uses forced choice results in stable, well-behaved estimates of the key coefficients, with little evidence of heteroskedasticity.

Minding the Gap

INTRODUCTION: A point of care test (POCT) for Chlamydia trachomatis detection is an urgent public health need. Technology advances in diagnostics have made solutions possible. Yet no reliable POCT exist. Our goal was to address the gap between chlamydia POCT needs and successful POCT development by determining which characteristics of POCT tests are most critical and if any flexibility in the attributes assigned those characteristics exist between technology developer and end user. METHODS: We employed a process known as WALEX (Warfare Analysis Laboratory Exercise) in combination with Design of Experiment (DOE) methodology using discrete choice experiments (DCE), to describe the attributes of the most realistic, rather than the most ideal POCT. The WALEX was conducted as interactive oral and simultaneous electronic discussion among experts with differing expertise, but linked by a common interest in development of a chlamydia POCT. RESULTS: Our studies demonstrated which features of the ideal chlamydia POCT were considered critical to test acceptance by users and which were open to negotiation. In particular, end users were more lenient on the requirement for the fastest ideal test and the lowest one time instrument costs, if the requirement for higher throughput, lowest cost and vaginal sample source collection were preserved. DOE methods used in forced choice question design provided confirmation of opinions derived from oral and electronic WALEX comments CONCLUSIONS: The WALEX in combination with DCE helped us achieve our goal in identifying the gaps in the chlamydia POCT and determining the most realistic solutions to bridge those gaps.

A paper that changed my practice: The question cube

When was the last time you came to the end of a consultation feeling unsatisfied (maybe even a little grumpy), unable to address a patient's problems neatly and unsure what to do next? Our response to patients with persistent symptoms that are proving difficult to treat may be to reconsider our original diagnosis, order more tests or search for a new drug that we have not tried before. Making a referral to a subspecialist or allied health colleague might buy us some time. Or perhaps we should consider an alternative therapist. Having often been in this situation, it was a joy to come across the Question Cube, a paper that changed my practice by offering a more satisfying way of approaching this type of encounter.1 The paper, given to me as part of the introductory reading for a family therapy course, offers practical advice on how to use questions to open up stuck consultations and steer them into previously unexplored areas. The technique is rewarding for the doctor and, if nothing else, may make a pleasant change for the patient. The key message of the paper is that questions, in addition to being used to elicit factual information, can be used strategically in a number of other ways. Most clinicians are familiar with open questions –‘How are you? How have the headaches been recently?’ These allow patients to expand on their symptoms but often along familiar lines and with no guarantee of a tidy conclusion. In these situations, closed questions (‘Have you taken any paracetamol today?’) can contain patients who have little problem talking but with minimal progression in the information presented. Closed questions ('Did the pain wake you during the night?') can also be used to obtain information from patients (e.g. many adolescents) whose typical response is either silence or a shrug. A follow-up is the forced choice question –‘Did you go to school yesterday or stay at home?’ Although the answers to questions such as these are inevitably brief, they do provide specific information. When working with patients who say very little, this is a valuable technique for allowing a conversation to develop. Forced choice questions also enable the clinician to suggest options or alternatives that the patient may not have considered and to test out hypotheses or hunches. ‘Do you think your symptoms been worse recently because you have had a lot on at school or because of problems at home?’ Ranking questions introduce ideas as to how others may be feeling. ‘Who would you say is most worried by your headaches? And who is the next most worried?’ while rating questions provide information regarding the importance or impact of a particular symptom. ‘On a scale of 1 to 10, how much of a problem are the headaches at the moment?’‘If not the headaches, what is the biggest problem for you at the moment?’ The value of these questions is that the answers can highlight issues, such as school non-attendance or the worries of a parent, which may be easier to address than the presenting problem. If you have your fair share of challenging patients but are too busy to enrol in another professional development course, this paper may be just what you are looking for.

An Evaluation of a Postoperative Rehabilitation Program After Spinal Surgery and Its Impact on Outcome

This study invited patients to evaluate the content and style of a rehabilitation program used as an intervention in a multicenter, factorial, randomized controlled trial of the postoperative management of spinal surgery patients. To determine the acceptability and content of the rehabilitation program. The use of rehabilitation after spinal surgery is sporadic, and the evidence for its benefit is mixed. A randomized controlled trial was conducted to determine whether functional outcome of spinal surgery could be improved by a postoperative rehabilitation program and/or an educational booklet provided at hospital discharge, each compared with usual care. Patients randomized to postoperative rehabilitation began the course 6 weeks after surgery. At the end of the course of 12 classes, patients completed an evaluation of content, style, and length using forced choice and open questions. Patients not attending the final class completed the evaluation at their 3-month review. Compliance with rehabilitation was poor, with 41% of subjects failing to attend any classes and 16% attending less than half. Compliance with the evaluation was 100% for the 105 patients attending 1 or more classes. The length, content, and approach to the classes were rated positively, and patients gave the class a median rating of 9 of 10, range 5 to 10, with 91% rating the classes as 7 or above. Compliance with a postoperative rehabilitation program was disappointing, but the program was assessed positively and of benefit by those who attended. Issues were raised in relation to timing and location of classes.

Caller self‐care decisions following teletriage advice

To examine caller self-care decisions following teletriage advice provided by nurses. The use of teletriage is gaining popularity as one way of enhancing capacity for self-care. Research from several countries suggests that teletriage reduces the use of other healthcare services without compromising safety. However, there is little or no research related to how often self-care advice is provided and whether or not callers follow the advice. A descriptive survey design was used with a random sample of 312 callers who were advised by a teletriage nurse to engage in self-care. Callers were randomly selected from all calls to a teletriage service each day of the month for nine months. Data were collected using a researcher-developed interview guide and analysed using a variety of inferential statistics for forced choice questions and content analysis for open-ended questions. The majority of callers who were advised to engage in self-care reported doing so. Callers with greater self-efficacy and satisfaction with the nurse interaction were more likely to follow advice to self-care. All callers would call the teletriage service again for the same or a different issue. Teletriage callers were confident in the advice provided and were willing to continue to use the service. This study indicates that teletriage programmes are a cost-effective way of addressing self-care needs of individuals who might otherwise visit an emergency department.

American Physical Therapy Association-Credentialed Clinical Instructors: A Survey Regarding Source of Motivation for Pursuing Credentialing

Background and Purpose. The American Physical Therapy Association (APTA) promotes clinical education credentialing for clinical instructors (CIs) through the Clinical Instructor Education and Credentialing Program (CIECP). APTA reports that since the program's inception, 34,657 CIs have received Basic CIECP credentialing status, which represents less than half of APTA's membership, and an unknown percentage of non-APTA members. To date, it is not known what motivates CIs to pursue credentialing, or if the motivation is more intrinsically or extrinsically driven. The purpose of this study was to determine the source of the motivational factors for physical therapist CIs who have pursued credentialing by APTA. Subjects. The subjects included 2,209 APTA-credentialed physical therapist CIs distributed throughout the United States. Methods. A pilot form of the survey was used to gain feedback from local credentialed CIs to establish face and content validity. Credentialed CIs were sent an email link to a 3-section survey. The first section consisted of demographic questions, while the second section consisted of 7-point Likert scale questions pertaining to intrinsic versus extrinsic motivation for becoming credentialed. Finally, the last section was forced choice questions also pertaining to motivational factors. Repeated measures ANOVAs were used to compare all responses in section 2 between the intrinsic and extrinsic oriented questions, with paired t tests to determine specific question response differences. Results. Analysis revealed significant differences favoring intrinsic motivation. The 2 most significant intrinsic factors included contribution to the education of student physical therapists (SPTs) and professional development. Within the open-ended section of the survey more than 50% of the respondents reported intrinsic motivational reasons for becoming APTA Basic CIECP CIs. Discussion and Conclusion. Results from both the close-ended and forcedchoice questions demonstrate that the driving forces behind a CI obtaining APTA credentialing are intrinsic. The primary intrinsic motivators were professional development and personal satisfaction of helping educate students.

The Potential Impact of Reduction in Federal GME Funding in the United States: A Study of the Estimates of Designated Institutional Officials

As a result of the decisions made in the Summer of 2011 to increase the federal debt limit of the United States,1 the Joint Select Committee on Deficit Reduction (the “Super Committee”) was formed to recommend to the Congress by 12 2011 reductions in federal spending to be accomplished over the next 10 years. Among the entitlement program elements being examined by the Super Committee, Medicare reimbursement for Graduate Medical Education (GME), the primary source of GME funding in the United States, has been identified as an opportunity for spending reductions. Specifically, the Medicare Payment Advisory Commission (MedPAC) has indicated that approximately 50% of the Indirect GME Reimbursement is not “empirically justified” on the basis of current costs of teaching hospitals intended to be covered by that reimbursement.2 This is seen by many as an opportunity for reduction. Furthermore, the Simpson Bowles Commission recommended a reduction in total GME funding in excess of 50% ($60 billion over 10 years) as a component of a comprehensive strategy to reduce federal deficit spending.3 This discussion occurs in the context of a predicted physician shortage, a mounting surge in the number of domestic graduates of schools of allopathic and osteopathic medicine designed to remedy that anticipated physician shortage,4 and slow growth in GME output over the past decade.5 However, this increase in medical school output must be accompanied by an increase in Graduate Medical Education, which is the required final pathway to entry into the unsupervised practice of medicine in the United States. As the body charged by the public and the profession with accreditation oversight of the vast majority of the GME programs in the United States, and to measure the quality of educational opportunities for physicians as they seek to prepare to meet the needs of the American public, the Accreditation Council for Graduate Medical Education (ACGME) attempted to estimate the impact of reductions in GME funding of the magnitude under discussion in our nation's capital on the educational pipeline for physicians.

Verbal problem‐solving in autism spectrum disorders: A problem of plan construction?

Children with autism spectrum disorders (ASD) adopt less efficient strategies than typically developing controls (TD) on verbal problem-solving tests such as the Twenty Questions Task. This study examined the hypotheses that this can be explained by differences in (i) planning processes or (ii) selective attention. Twenty-two children with ASD and 21 TD controls matched for age (M(age) = 13:7) and cognitive ability (M(FSIQ) = 96.42) were tested on an adapted version of Twenty Questions and two planning tasks. ASD participants could recognize effective questions as well as TD participants on a forced-choice question discrimination task, but were observed to construct plans that were significantly less efficient. ASD performance was also specifically reduced when items could not be physically removed from the testing array, although this effect could be ameliorated by keeping a written record of participant questions during search. These findings indicate that ASD participants are sensitive to the within-task executive demands of Twenty Questions, but that their inefficiency in strategy relates to planning processes and question selection pretask. The implications for understanding ASD problem-solving skills and their impact on everyday functioning are discussed.

CAML—Maximum likelihood consensus analysis

Consensus analysis enables estimation of individual differences in competencies and response tendencies when answer keys to dichotomous forced-choice questions are unknown. CAML, a set of functions written in R, implements maximum likelihood estimation for the general Condorcet model that underlies consensus analysis. CAML avoids problems of alternative approaches that have often rendered consensus analysis impractical or unfeasible in the past. It provides (1) measures of model fit, (2) a measure of consensus, (3) point and interval estimates of competencies and response tendencies, and (4) an estimate of the unknown answer key. The present article describes the general Condorcet model, the CAML algorithms, and the handling of the software. In addition, the validity of CAML results is tested in a recognition memory study using selective experimental manipulations of the parameters. The results show that CAML works very well in practice and provides valid estimates of competencies, response tendencies, and answer keys.

The acquisition of scientific knowledge: the influence of methods of questioning and analysis on the interpretation of children's conceptions of the earth

Studies of children's knowledge of the Earth have led to very different conclusions: some appear to show that children construct their own, non-scientific ‘theories’ (mental models) of the flat, hollow or dual Earth. Others indicate that many young children have some understanding of the spherical (scientific) Earth, and that their knowledge lacks the coherence of mental models. The reasons for these contrasting views were tested by interviewing French children (N = 178) aged 5–11 years and varying the different methods used in previous research, namely the types of questions (open and forced-choice), the form of representation (two-dimensional pictures and three-dimensional models), and the method of analysis (the mental model theorists' coding scheme and a statistical test for associations using MANOVA). Forced-choice questions resulted in higher proportions of scientific answers than open questions, and children appeared to have naïve mental models of the Earth only when the mental model theorists' coding scheme was used. These findings support the view that children tend to have ‘fragments’ of scientific knowledge, and that naïve mental models of the Earth are methodological artifacts. Copyright © 2011 John Wiley & Sons, Ltd.

Adding Information May Increase Overconfidence in Accuracy of Knowledge Retrieval

Feelings of retrospective confidence concerning the accuracy of a chosen answer might rely, among other things, on the amount of available information, regardless of its correctness. 43 participants, 26 women and 17 men (M age = 23.4 yr., SD = 3.5) in an intact group design, answered nine easy and nine difficult binary forced-choice questions and rated their confidence regarding the correctness of their choices. Participants were randomly assigned to one of three groups, differing in the additional information provided regarding the questions: a control group provided with no additional information, a correct information group, and a misleading information group. Performance was worst in the misleading information group, yet no difference in confidence was found between the correct and misleading information groups. The findings were interpreted as supporting the hypothesis that feelings of confidence partly reflect peripheral factors, indirectly related to choice processes.

A comparative study of listening comprehension measures in English as an additional language and native English-speaking primary school children

Abstract The aim of this study was to investigate the suitability of different measures of listening comprehension for Years 2, 3 and 4 children with English as an additional language (EAL). Non-standardised uses of reading comprehension measures are often employed as proxy measures of listening comprehension, i.e. for purposes for which they were not intended. The extent to which such uses can validly and reliably measure listening comprehension is disputed. In this study, one standardised and two unstandardised measures of listening comprehension were implemented: the listening comprehension subtest of the Wechsler Individual Achievement Test, Second UK Edition (WlAT-II); a listening comprehension version of the Neale Analysis of Reading Ability-Revised with original, open-ended questions (NARA-R [open-ended]); and a listening comprehension version of the NARA-R with modified, forced-choice questions (NARA-R [forced-choice]). It is argued that the NARA-R (forced-choice) is susceptible to guessing and is...

Examination results of medical students with dyslexia

dyslexia is a learning disorder, the primary sign of which is significant difficulty in learning to read and spell. However, accumulating evidence suggests that many people with dyslexia can overcome their reading difficulties and enjoy high levels of educational success. There is debate about the appropriateness of different forms of summative assessment for people with dyslexia, but there is little research investigating different examination formats, particularly in higher education, including medical education. Currently, medical school examinations comprise a range of different assessments, both written and performance-based, offering an opportunity to compare performance on different formats. This study compared results between students with and without dyslexia on all summative assessment types used at one UK medical school. examination scores were collated for all summative Year 1 and 2 examinations at Hull York Medical School (HYMS) over four cohorts entering from 2004 to 2007. These included scores on two types of forced-choice question (multiple-choice and extended matching question) examinations, on short written answer examinations and on performance in a 16-station objective structured clinical examination (OSCE). Results for written answers were gathered separately for basic science questions and for questions involving critical analysis and evidence-based medicine. an overall multivariate analysis of covariance (mancova) on examinations across both years controlling for gender, ethnicity and age on entry indicated that there was no significant overall effect of dyslexia on examination results. Regression analysis further showed that dyslexia was not a significant predictor on any of the examination forms in Year 1 or Year 2. there is no indication that any of the assessment methods used in HYMS, in common with many other medical schools, disadvantage students with dyslexia in comparison with their peers. In the light of these findings, we support the current view that a variety of assessment types should be included in the assessment of all medical students, as is already considered to be best practice.

Integrating Safety and Crew Resource Management (CRM) Aspects in the Recurrent Training of Cabin Crew Members

Recurrent training of cabin crew should include theoretical and practical instruction on safety as well as crew resource management (CRM) issues. The endeavors of Swiss International Air Lines Ltd. and Swiss Aviation Training Ltd. to integrate CRM and safety aspects into a single training module were evaluated. The objective of the integration was to make CRM more tangible and ease acquisition of competencies and transfer of CRM training content to practice by showing its relevance in relation to safety tasks. It was of interest whether the integrated design would be mirrored in a more favorable perception by the trainees as measured with a questionnaire. Participants reacted more positively to the integrated training than to stand-alone CRM training, although the integrated training was judged as being slightly more difficult and less oriented toward instructional design principles. In a range of forced-choice questions, the majority of participants opted for an integrated training format because it was seen as livelier and more interesting and also more practically relevant. For the forthcoming training cycle, a better alignment of training with instructional principles and an even higher degree of training integration by using simulator scenarios are striven for.

Diferencias sexuales en la experiencia subjetiva de los celos: una mirada desde la Psicología Evolucionista

Objective. In this study sexual differences are analyzed in relation to the experience of jealousy among young people and university students. Method. A non-experimental, descriptive, comparative study of a transversal nature was carried out. The sample consisted of 299 participants, 57.2% of whom were female and 42.8% male, who responded to forced choice questions designed by Buss, Larsen, Westen and Semmelroth (1992), with regard to the level of distress caused by emotional or sexual infidelity. Results. The female subjects presented a subjective experience of emotional jealousy when faced with possible infidelity on the part of their partner, higher than that of men with 77.2%. The males presented a subjective experience of sexual jealousy when faced with possible infidelity on the part of their partner, higher than that of females, with 32.8%. Conclusion. According to the evolutionary hypothesis, which explains the difference in jealousy from

Do Young Children Know What Makes A Picture Useful To Other People?

Abstract Even babies have an implicit appreciation of the relationship between realistic pictures and the objects that they depict, but a mature understanding of pictures involves an explicit appreciation of how pictures work. Adults appreciate that pictures are public representations that can communicate information to other people, and that some pictures are better at doing this than others. We explore the foundations of this understanding in young children. In three experiments, using yes/no and forced-choice questions, we find that 3- and 4-year olds understand that other people benefit from pictures that contain greater perceptual detail and that the more realistic the picture, the better it is as a symbolic vehicle.

Attitudes towards Diagnostic Tests and Therapies for Dry Eye Disease

Aim: The purpose of this study was to survey the attitudes of optometrists and ophthalmologists, located in a number of different countries, towards diagnostic tests and therapies for dry eye disease. Methods: A web-based questionnaire was used to survey attitudes using forced-choice questions and Likert scales. Results: Sixty-one respondents (23 ophthalmologists and 38 optometrists) reported a wide range of patient dry eye symptoms. A large variation in use of diagnostic tests was noted. Patient symptoms and fluorescein staining were reported to be significantly more valuable and more frequently performed than any other test. Artificial tear supplements and improved lid hygiene were the preferred therapeutic options selected by the entire group. The results demonstrated a wide variation in attitudes in relation to satisfaction with the range of available diagnostic and therapeutic options. Conclusions: This study indicates that the interest for the issue of dry eye is relatively limited amongst eye professionals, as demonstrated by the poor participation in the questionnaire.