Forced-choice Scale Research Articles

Remote expert viewing, laboratory tests or objective metrics: which one(s) to trust?

We present a study on the validity of quality assessment in the context of the development of visual media coding schemes. The work is motivated by the need for reliable means for decision-taking in standardization efforts of MPEG and JVET, i.e., the adoption or rejection of coding tools during the development process of the coding standard. The study includes results considering three means: objective quality metrics, remote expert viewing, which is a method designed in the context of MPEG standardization, and formal laboratory visual evaluation. The focus of this work is on the comparison of pairs of coded video sequences, e.g., a proposed change and an anchor scheme at a given rate point. An aggregation of performance measurements across multiple rate points, such as the Bjøntegaard Delta rate, is out of the scope of this paper. The paper details the test setup for the subjective assessment methods and the objective quality metrics under consideration. The results of the three approaches are reviewed, analyzed, and compared with respect to their suitability for the decision-taking task. The study indicates that, subject to the chosen test content and test protocols, the results of remote expert viewing using a forced-choice scale can be considered more discriminatory than the results of naïve viewers in the laboratory tests. The results further that, in general, the well-established quality metrics, such as PSNR, SSIM, or MS-SSIM, exhibit a high rate of correct decision-making when their results are compared with both types of viewing tests. Among the learning-based metrics, VMAF and AVQT appear to be most robust. For the development process of a coding standard, the selection of the most suitable means must be guided by the context, where a small number of carefully selected objective metrics, in combination with viewing tests for unclear cases, appears recommendable.

Though Forced, Still Valid: Examining the Psychometric Performance of Forced-Choice Measurement of Personality in Children and Adolescents.

Unveiling the roles personality plays during childhood and adolescence necessitates its accurate measurement, commonly using traditional Likert-type (LK) scales. However, this format is susceptible to various response biases, which can be particularly prevalent in children and adolescents, thus likely undermining measurement accuracy. Forced-choice (FC) scales appear to be a promising alternative because they are largely free from these biases by design. However, some argue that the FC format may not perform satisfactorily in children and adolescents due to its complexity. Little empirical evidence exists regarding the suitability of the FC format for children and adolescents. As such, the current study examined the psychometric performance of an FC measure of the Big Five personality factors in three children and adolescent samples: 5th to 6th graders (N = 428), 7th to 8th graders (N = 449), and 10th to 11th graders (N = 555). Across the three age groups, the FC scale demonstrated a better fit to the Big Five model and better discriminant validity in comparison to the LK counterpart. Personality scores from the FC scale also converged well with those from the LK scale and demonstrated high reliability as well as sizable criterion-related validity. Furthermore, the FC scale had more invariant statements than its LK counterpart across age groups. Overall, we found good evidence showing that FC measurement of personality is suitable for children and adolescents.

Development and validation of the general regulatory focus measure forced choice scale (GRFM-FC)

Development and validation of the general regulatory focus measure forced choice scale (GRFM-FC)

A 2PLM-RANK multidimensional forced-choice model and its fast estimation algorithm.

High-stakes non-cognitive tests frequently employ forced-choice (FC) scales to deter faking. To mitigate the issue of score ipsativity derived, many scoring models have been devised. Among them, the multi-unidimensional pairwise preference (MUPP) framework is a highly flexible and commonly used framework. However, the original MUPP model was developed for unfolding response process and can only handle paired comparisons. The present study proposes the 2PLM-RANK as a generalization of the MUPP model to accommodate dominance RANK format response. In addition, an improved stochastic EM (iStEM) algorithm is devised for more stable and efficient parameter estimation. Simulation results generally supported the efficiency and utility of the new algorithm in estimating the 2PLM-RANK when applied to both triplets and tetrads across various conditions. An empirical illustration with responses to a 24-dimensional personality test further supported the practicality of the proposed model. To further aid in the application of the new model, a user-friendly R package is also provided.

The Generalized Thurstonian Unfolding Model (GTUM): Advancing the Modeling of Forced-Choice Data

Forced-choice (FC) measurement has become increasingly popular due to its robustness to various response biases and reduced susceptibility to faking. Although several current Item Response Theory (IRT) models can extract normative person scores from FC responses, each has its limitations. This study proposes the Generalized Thurstonian Unfolding Model (GTUM) as a more flexible IRT model for FC measures to overcome these limitations. The GTUM (1) adheres to the unfolding response process, (2) accommodates FC scales of any block size, and (3) manages both dichotomous and graded responses. Monte Carlo simulation studies consistently demonstrated that the GTUM exhibited good statistical properties under most realistic conditions. Particularly noteworthy findings include (1) the GTUM's ability to handle FC scales with or without intermediate statements, (2) the consistently superior performance of graded responses over dichotomous responses in person score recovery, and (3) the sufficiency of 10 mixed pairs to ensure robust psychometric performance. Two empirical examples, one with 1,033 responses to a static version of the Tailored Adaptive Personality Assessment System and the other with 759 responses to a graded version of the Forced-Choice Five-Factor Markers, demonstrated the feasibility of the GTUM to handle different types of FC scales. To aid in the practical use of the GTUM, we also developed the R package “ fcscoring.”

Moving beyond Likert and Traditional Forced-Choice Scales: A Comprehensive Investigation of the Graded Forced-Choice Format

The graded forced-choice (FC) format has recently emerged as an alternative that may preserve the advantages and overcome the issues of the dichotomous FC measures. The current study presented the first large-scale evaluation of the performance of three types of FC measures (FC2, FC4 and FC5 with 2, 4 and 5 response options, respectively) and compared their performance to their Likert (LK) counterparts (LK2, LK4, and LK5) on (1) psychometric properties, (2) respondent reactions, and (3) susceptibility to response styles. Results showed that, compared to LK measures with the same number of response options, the three FC scales provided better support for the hypothesized factor structure, were perceived as more faking-resistant and cognitive demanding, and were less susceptible to response styles. FC4/5 and LK4/5 demonstrated similarly good reliability, while LK2 provided more reliable scores than FC2. When compared across the three FC measures, FC4 and FC5 displayed comparable psychometric performance and respondent reactions. FC4 exhibited a moderate presence of extreme response style, while FC5 had a weak presence of both extreme and middle response styles. Based on these findings, the study recommends the use of graded FC over dichotomous FC and LK, particularly FC5 when extreme response style is a concern.

Systematically Identifying Communication Deficits in Youth Under Psychiatric Care Through a Self-Report Scale

Background: Communication disorders are associated with psychopathology and social distress in adolescents, yet moderate or less severe deficits often go undiagnosed. Data from a quality improvement initiative was analyzed from the Communication Perception Estimate Scale (CoPES), a self-report scale for receptive-expressive and pragmatic language deficits in youth receiving mental health services. Objective: To identify the frequency of communication deficits (CDs) among youth in psychiatric care and identify patterns of self-reported symptoms. The scale may be used as a clinical screening tool for Language disorder and Social (Pragmatic) communication disorder. Methods: CoPES is a 31-item scale administered to 318 adolescents 11-18 years old, asking questions about receptive-expressive and pragmatic CDs. In the first 12 items, subjects identified the frequency of communication disorder symptoms using a forced-choice scale; in the following items, frustration with CDs, yes or no for earlier intervention, and current interest in getting help for CDs. Respondents are from five representative clinical services. Results: Patients were divided into self-identified groups: 14.5% significant Receptive-Expressive deficits only (R-ED); 6.6% significant Pragmatic deficits only (PD); 29.9% Both significant Receptive-Expressive and Pragmatic deficits (BCD); and 49.1% with limited or no deficits (NCD). The R-ED group expressed high rates of frustration and requests for help with Receptive-Expressive deficits. The PD group reported elevated rates of frustration in both areas. The BCD group indicated significantly more frustration and requests for help with both language areas compared to the NCD group. Conclusion: This data supports the need for greater awareness and consideration of communication disorders in youth presenting for psychiatric care.

Validity and reliability of measurement instruments of educative family life

This study is intended to obtain standard instruments of the educative family life. Its construct has been formulated based on the research results of family factors influencing the success of the children at school. The instruments involve children measurement instrument (CMI) and mother measurement instrument (MMI) that apply forced-choice scale inventories. The instrument calibration methods are test of construct validity, content validity–face validity, empirical validity and reliability. Through the calibration of the instruments, the results of the 573 family samples revealed the following results: 1) the construct of 12 family variables described the variations in the educational family life as 38.47% child perception and 41.15% mother perception; 2) 45.85% of the items of structure dimension of CMI are valid; 3) 70.50% of the items of process dimension of CMI are valid; 4) 45.65% of the structural dimension of MMI are valid; 5) 80% of the process dimension of MMI are valid; and 6) both instruments, CMI and MMI, are reliable. 
 
 Keywords: Children measurement instrument; instrument; mother measurement instrument; reliability; validity.

AutoFC: An R Package for Automatic Item Pairing in Forced-Choice Test Construction.

Recently, there has been increasing interest in adopting the forced-choice (FC) test format in non-cognitive assessments, as it demonstrates faking resistance when well-designed. However, traditional or manual pairing approaches to FC test construction are time- and effort- intensive and often involve insufficient considerations. To address these issues, we developed the new open-source autoFC R package to facilitate automated and optimized item pairing strategies. The autoFC package is intended as a practical tool for FC test constructions. Users can easily obtain automatically optimized FC tests by simply inputting the item characteristics of interest. Customizations are also available for considerations on matching rules and the behaviors of the optimization process. The autoFC package should be of interest to researchers and practitioners constructing FC scales with potentially many metrics to match on and/or many items to pair, essentially exempting users from the burden of manual item pairing and reducing the computational costs and biases induced by simple ranking methods.

Popular Psychology versus Scientific Evidence: Love Languages’ Factor Structure and Connection to Marital Satisfaction

Love is an essential part of human experience and love languages have been studied to validate its factors’ structures to explain what makes people feel loved. The current study addresses the gap that love research shall not rely on student samples and it needs to measure the actual outcome of love languages. This study aims to gather empirical evidence for love languages’ factor structure and its relation to the outcome variable. The method for this study is a quantitative survey with 250 couples reported their love languages using a rating-scale and forced-choice scale. The data analysis examined the factor structure of the love languages model and estimated the association between love languages compatibility and marital satisfaction. The factorial analysis showed that the five factors solution was not supported and love languages compatibility did not affect couples’ marital satisfaction. This result brought discussions on how popular psychology concepts need to be under the scrutiny of scientific investigation and that different contexts may have different factors on what makes people feel loved.

An Educational Intervention on Chinese Business Students’ Orientation Towards Corporate Social Responsibility

A one-day educational intervention with multiple activities was developed and operationalized with a sample of Chinese business students in Hong Kong, China. Its effectiveness in influencing students’ corporate social responsibility orientation (CSRO) was measured with a Chinese version of a forced choice scale using Economic, Legal, Ethical, and Discretionary (Philanthropy) dimensions by Carroll (1979, 1991). A repeated measures multivariate analysis of variance (MANOVA) showed significant differences in the Legal and Discretionary dimensions between the post-test Experimental (X) group (N=82) and Control (C) group (N=83); in the Legal, Ethical, and Discretionary dimensions within the pre-post X group. Such significant differences may be explained by the content of the activities, especially the service learning component. Overall, the intervention appeared effective in influencing students’ CSRO within a Chinese context. Since it was designed upon Western CSR literature, its applicability goes beyond the Chinese community.

Does forcing reduce faking? A meta-analytic review of forced-choice personality measures in high-stakes situations.

Forced-choice (FC) is a popular format for developing personality measures, where individuals must choose 1 or multiple statements from several options. Although FC measures have been proposed to reduce score inflation in high-stakes assessments, inconsistent results have been found in empirical studies regarding their effectiveness. In this study, we conducted a meta-analysis of studies comparing FC personality measure scores between low-stakes and (both simulated and actual) high-stakes situations. Results suggest that the overall score inflation effect size for FC personality measures is 0.06. In selection scenarios, score inflation for FC scales is much lower than the meta-analytic effect size for single-statement personality measures across most personality facets. The score inflation effect size was also found to vary across FC scale characteristics and study design factors. Specifically, FC scales were consistently found to be more faking-resistant when constructed with statements balanced in social desirability and with responses scored via a normative approach. FC scales constructed with the PICK format were also found to be faking-resistant, while more applicant-incumbent studies are needed to examine the fakability of MOLE FC scales. Evidence at the overall level supports the use of multidimensional scales and extremity balance of statements, but results are not consistent across personality facets, or when large samples are excluded. Personality facets of high relevance to the target job were found to exhibit larger inflation than facets of low relevance to the target job. Practical guidance on constructing and using FC personality measures for personnel selection purposes is provided. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Beautiful Bugs, Bothersome Bugs, and FUN Bugs: Examining Human Interactions with Insects and Other Arthropods

ABSTRACTBecause the ostensible majority of incidental human–insect (and other arthropods) interactions are negative, any interest in non-pretty “bugs” appears to be inherently demotivated. Three complementary studies explored US college students’ perceptions, knowledge, and experiences of insects to better understand folk classifications and to identify potentially new ways to present them to motivate human interest. Study 1, an open-ended survey (n = 236), found that knowledge of insects is limited to a mean of 13 insects. Of these 13 insects, most were also dichotomized as liked (beautiful bugs) or disliked (bothersome bugs). The second study, using semi-structured interviews (n = 60), revealed similar categories as found in the first study, providing further details about positive and negative perceptions of, attitudes to, and types of experiences people have with, insects and other closely related arthropods. The last study (n = 200) used a paired forced-choice scale with 10 silhouettes of insects and related arthropods to replicate and expand the findings from the first two studies. This study tested whether respondents would report interest in novel and unknown arthropods over commonly known and preferred ones. The results indicate little knowledge of the diversity of insects among a young, elite, middle-class sample of college students and the existence of two robust but small folk categories of insects/arthropods (beautiful, bothersome). Results from the third study indicated there is a group of potentially fascinating unfamiliar (FUN) insects/arthropods/bugs that could evoke interest if people were simply exposed to them. Implications for informal recreation and educational programming and a research agenda are presented.

An Investigation of Preservice Teachers’ Self-Efficacy for Teaching with Technology

<p>Self-efficacy has been persistently cited as a major component in understanding how frequently and successfully individuals use technology. Preservice teachers’ technology self-efficacy is a creditable indicator of graduates’ likelihood to use instructional technology throughout their careers; thus, it is imperative that new teachers graduate with a high sense of self-efficacy regarding their abilities to use technology as an effective teaching tool. The purpose of this study was to determine preservice teachers’ self-efficacy beliefs related to integrating technology into instruction. Elementary preservice teachers (n=28) from a single university completed a Likert-type survey using a forced choice scale with six points. Responses to the 5-item survey were converted to numerical data, and the research question was examined using descriptive statistics. Data indicated that almost all (93%) of the participants incorporated technology into the lessons they taught with the majority (68%) of participants reporting a high level of confidence in their ability to select and utilize technology in teaching. Further, 89% of the participants felt they could integrate technology across the curriculum with 80% indicating they were capable of determining the why, when, and how to do so most of the time.</p>

The Scale Properties of the Adolescent Form of the Francis Psychological Type Scales (FPTSA) Among Canadian Baptist Youth

A sample of 755 religiously committed young people between the ages of 12 and 18 attending Tidal Impact (a weeklong youth mission and service event sponsored by the Convention of Atlantic Baptist Churches in Eastern Canada) completed a trial 80-item form of the Francis Psychological Type Scales designed for use among adolescents. These data were employed to refine four ten-item forced-choice scales to distinguish between preferences for the two orientations (extraversion and introversion), the two perceiving processes (sensing and intuition), the two judging processes (thinking and feeling) and the two attitudes (judging and perceiving). The scale properties of the new instrument commend the 40-item Adolescent form of the Francis Psychological Type Scales (FPTSA) for future use.

Review: Attitudes of Jordanian nursing students to nursing research

There have been many studies of nurses’ interest in research but most of them have been from North America or Europe. This is the first such study to be carried out in an Arabic country. The development work on the scale used was of high quality. The items included were derived from earlier research by Funk et al. (1991). They were carefully piloted to ensure that they were intelligible and then translated into Arabic. The translation was vetted by the language department of the university and the items were piloted for a second time. A Lickert-type scale of 18 items was developed and students were asked to identify their attitudes to nursing research on one of four points ranging from strongly disagree (1) to strongly agree (4). This was a forced choice scale eliminating the possibility of a neutral middle position. A second questionnaire covered demographic data and there was a face sheet outlining the purpose of the research. Senior nursing students were targeted. In total 900 questionnaires were distributed and completed documents were returned by 68%. This is not a high response rate given that the students were in the nursing faculty, were obliged to have completed a course in nursing research as a condition of inclusion and could be easily contacted. The analysis of the data was sophisticated. There is some question about whether this was warranted because of nature of sample. Analysis of variance produced four dimensions of the scale items. The Mann–Whitney U-test was used to explore the differences between the attitudes of male and female students to research. Perhaps the most useful aspect of this research is that the scale can be used in all Arabicspeaking countries. Its use would enable universities to see if they are getting the message across. Testing at the start of the course and re-testing at the end of the module is the next logical development, All students should be required to participate, This would not only give their views but a measure of the success of the course in bringing about positive attitudes to nursing research. Although the sample was large and the response just over two-thirds, it was a self-selected group and the results cannot be generalised to all students following similar courses in

Face, content and construct validity of a new realistic trainer for conventional techniques in digestive surgery

Surgical simulation modules for "open" surgery are limited in contrast to well-studied and validated laparoscopic trainers. In this study, face, content and construct validity of a devised simulation module (Berlin Operation Trainer, BOPT) for handsewn anastomoses in digestive surgery were analysed. Participants of a skills course for digestive surgery (novices: 1-3 years of training; experts: more than 5 years of training) were timed on performing four defined handsewn digestive anastomoses on formalin fixed porcine intestine in the BOPT. Questionnaires were answered regarding impression with the simulation module concerning appearance and realism using a five-point Likert and a three-point forced choice scale. Face and content validities were evaluated based on the responses of participants and construct validity by comparing novices to experts. Data collected were analysed with Fisher's exact test and two-sample t test. Twenty-two novices (median: second postgraduate year) and 26 experts (median: seventh postgraduate year) were enrolled in the study. The BOPT showed strong face and content validities with average scores for satisfaction parameters above 4.2 +/- 0.41 and 4.1 +/- 0.22, respectively. Construct validity was adequate for anastomosis simulation in the BOPT based on different percentages of anastomosis complete during set time between novices and experts as shown for simple (68.2% vs. 92.3%, p = 0.038) and for difficult anastomosis (18.2% vs. 50.0%, p = 0.021). The BOPT is a suitable instrument for advanced surgical training for novices and experienced colleagues creating a realistic and demanding situation. Further studies have to evaluate if a more realistic preoperative training will support an effective transfer of learned techniques to the operating room.

Evaluation of the effects on conjunctival tissues of Optive eyedrops over one month usage

The objective was to compare the effect on conjunctival tissues of the repeated use, over a one-month period, of Optive compared to Hylocomod eyedrops by a population of dry eye sufferers. The rationale for the study was that among dry eye sufferers who attend eye care practices for symptomatic relief, a large number present with conjunctival anomalies evidenced by tissue staining and that conjunctival recovery is essential to their successful long term management. The hypothesis tested was that the decrease in conjunctival staining with Optive is at least as good, and possibly greater, than that with Hylocomod. The cohort population was made up of 47 subjects (11 male and 26 female) aged 42+/-16 years with at least mild dry eye symptoms and conjunctival and/or corneal staining. The population included contact lens wearers (n=26) and non-wearers (n=21). The subjects were randomly allocated to use one of the two study products; they were instructed to use the products as often as needed but at least three times a day. Conjunctival staining was rated on forced choice scales and measured objectively using digital photographs and image analysis. The findings showed that, whereas the staining at the start of the investigation was similar (p=0.318-0.664), staining after one month of use was significantly less with Optive than Hylocomod (p=0.028-0.002). The results demonstrated that the regular use of Optive over one month was significantly superior to Hylocomod in improving conjunctival status by producing a greater reduction in the staining of dry eye sufferers.

Kuder Career Search with Person Match: The Third-Generation Approach

The Kuder preference records have enjoyed a long and respected role in career guidance. Frederic Kuder introduced his first preference record in 1939, establishing the model of forced-choice responses and homogeneous vocational preference scales that was the basis for the present Kuder General Interest Survey. In its second generation, the Kuder Occupational Interest Survey, it was enlarged to include criterion group scales revealing the survey-taker's similarity with persons in an array of more than 140 occupations and college majors. A third generation has been developed in the past decade: Kuder Career Search with Person Match. In this version, the survey-taker's vocational preferences are matched not with occupations, but with each of a number of persons in an array of occupations representing the full spectrum of the world of work. This article introduces Kuder's radical approach to career guidance.

The impact on attitudes towards cultural difference of participation in a health focused study abroad program

The changes in attitudes towards cultural difference of seventeen participants in a three-week community health study abroad program to Nepal were compared with the changes in attitudes of a similar group who did not participate in the tour. Participants in the tour group were surveyed eight weeks prior to departure and in the last week of the tour using a twenty-six item questionnaire employing a six-point forced-choice response scale. The responses of participants in the lour group showed significant shifts in relation to eight items compared while the responses for the control group showed no significant shifts. Observed student advantages of participation in this study tour included the development of independent behaviour and positive cultural adjustment and adaptation.