Criterion-related Validity Study Research Articles

The unexplored territory of aesthetic needs and the development of the Aesthetic Needs Scale.

Human needs, and their fulfillment, are the building blocks of human development, personality, and well-being. However, no published paper in the field of psychology has focused on exploring aesthetic needs. Maslow (1986) gave the topic little more than a paragraph; and Dweck [1], in her elegant Unified Theory of Motivation, Personality, and Development, never mentions aesthetic needs. The aim of this article is to describe developing a scale for measuring the intensity of aesthetic needs. The structure, psychometric properties, and criterion-related validity of the scale were verified with three independent samples (total N = 592). The results of an EFA and two CFAs indicated a three-factor structure: 1) the need to aestheticize everyday life (aesthetic experiences of everyday objects and events unrelated to art, such as the presentation of food or the appearance of a workspace, etc.); 2) the need for contact with aesthetic creations (the arts); 3) the need to aestheticize the built and natural environments (urban spaces, architecture, parks, wild nature, etc.). In addition, our criterion-related convergent validity studies have shown that people with high aesthetic needs are characterized by experiencing more intense experiences in contact with works of art, have higher aesthetic competence in art, are more intensely involved in four forms of beauty, have a higher ability to integrate beauty, a stronger trait gratitude, curiosity about nature, greater sensitivity to disgust, and the need for internal and external stimulation. This scale may prove useful in research on individual differences and the psychology of aesthetics.

Evlilik Çatışma Alanları İndeksi ve Çatışma Çözme Stilleri Envanteri’nin Geliştirilmesi: Geçerlik ve Güvenirlik Çalışmaları

The purpose of this study is to develop the Conflict Areas Index (CAI) to identify possible conflict areas that couples experience in their marriages and to develop the Conflict Styles Inventory (CSI) to assess how they deal with these conflicts. Exploratory (N=374) and confirmatory (N=152) factor analysis and test-retest (N=49) examinations were conducted with a sample of 575 married individuals in total. For the first scale, principal component analysis (PCA) was used to examine the construct validity of the CAI. According to the results, a single component giving frequency values for 15 different conflict areas was identified. In the criterion-related validity studies, the scale had a significant, negative, and strong correlation with the Relationship Assessment Scale (RAS) (r=-.72) and the level of satisfaction with the relationship (r=-.67). The Cronbach's alpha coefficient was found as .91. For the second scale, the construct validity of the CSI was examined using Exploratory Factor Analysis. The results revealed a three-dimensional structure named destructive, passive, and active conflict resolution styles. In the criterion-related validity studies, it was found significant and positive correlations between destructive (r=.14) and passive (r=.18) conflict resolution styles with CAI scores and a significant and negative correlation between active resolution style with CAI scores. In addition, there were significant and negative correlations between destructive (r=-.15) and passive (r=-.12) conflict resolution styles with the RAS scores and a significant and positive correlation between active conflict resolution style with the RAS scores. The Cronbach's alpha coefficients were .88, .87, and .87 for destructive, passive, and active conflict resolution styles, respectively. The correlation values between the scores of the CSI within a 14-day time interval were ⍴=.94, ⍴=.91, and ⍴=.83 for destructive, passive, and active conflict resolution styles, respectively. These results indicated that CAI and CSI have sufficient psychometric properties that can be used to assess conflict areas and conflict resolution styles of married individuals.

Review of Statistical and Methodological Issues in the Forensic Prediction of Malingering from Validity Tests: Part II-Methodological Issues.

Forensic neuropsychological examinations to detect malingering in patients with neurocognitive, physical, and psychological dysfunction have tremendous social, legal, and economic importance. Thousands of studies have been published to develop and validate methods to forensically detect malingering based largely on approximately 50 validity tests, including embedded and stand-alone performance and symptom validity tests. This is Part II of a two-part review of statistical and methodological issues in the forensic prediction of malingering based on validity tests. The Part I companion paper explored key statistical issues. Part II examines related methodological issues through conceptual analysis, statistical simulations, and reanalysis of findings from prior validity test validation studies. Methodological issues examined include the distinction between analog simulation and forensic studies, the effect of excluding too-close-to-call (TCTC) cases from analyses, the distinction between criterion-related and construct validation studies, and the application of the Revised Quality Assessment of Diagnostic Accuracy Studies tool (QUADAS-2) in all Test of Memory Malingering (TOMM) validation studies published within approximately the first 20years following its initial publication to assess risk of bias. Findings include that analog studies are commonly confused for forensic validation studies, and that construct validation studies are routinely presented as if they were criterion-reference validation studies. After accounting for the exclusion of TCTC cases, actual classification accuracy was found to be well below claimed levels. QUADAS-2 results revealed that extant TOMM validation studies all had a high risk of bias, with not a single TOMM validation study with low risk of bias. Recommendations include adoption of well-established guidelines from the biomedical diagnostics literature for good quality criterion-referenced validation studies and examination of implications for malingering determination practices. Design of future studies may hinge on the availability of an incontrovertible reference standard of the malingering status of examinees.

Turkish Adaptation of the Situated Academic Writing Self-Efficacy Scale: Validity and Reliability Study

Abstract. This study aims to adapt the Situated Academic Writing Self-Efficacy Scale into Turkish and to conduct validity and reliability studies. The sample group of the study consisted of 245 (62 male, 183 female) university students. In the study, the Situated Academic Writing Self-Efficacy Scale, the Academic Procrastination Behavior Scale, and the Student Attitudes and Beliefs Regarding Authorship Scale were used. As a result of the exploratory factor analysis, it was seen that the scale had a three-factor structure, and this structure explained %57.01 of the total variance. The results of confirmatory factor analysis revealed that the fit values of the three-factor structure were at a good level (χ²=65.870, df=51, p=.079, χ²/sd=1.292, IFI=.966, CFI=.965, SRMR=.036, RMSEA=.047). In the criterion-related validity study, there was a negative relationship between the Situated Academic Writing Self-Efficacy Scale and the Academic Procrastination Behavior Scale scores (r=-.38, p<.01), and a positive relationship between the Student Attitudes and Beliefs Regarding Authorship Scale scores (r=.64, p<.01) significant correlation was found. The McDonald's ω coefficient calculated within the scope of the reliability study is .84 for the whole scale, .73 for reflection, .73 for creativity, and .68 for writing skills. In addition, the peer-half correlation was calculated as .73, and significant differences were found between the lower and upper groups of %27. When the item-total test correlations of the scale were examined, it was seen that all items were above .40. Finally, the test-retest correlation was calculated as .94. The findings revealed that the measurement tool is valid and reliable for use in the sample of university students in Turkey.

Using Workplace Personality to Guide Improvement of Law Enforcement Selection

Recurrent police-public conflict suggests misalignment in desired police behavior between police and the public. We explored differences in desired police characteristics between police and members of the American public. Although racial minorities endorsed more negative attitudes of police overall, we found no meaningful differences in desired police characteristics between police and the public or between racial minority and majority participants. Second, we combined multiple criterion-related validation studies in similar jobs via meta-analyses and synthetic validity analyses to identify personality predictors of police performance dimensions. Third, we assessed base rates and adverse impact of these personality characteristics in police. Incumbent officers scored significantly lower on desired characteristics and higher on undesired characteristics than applicants. Overall, scales measuring Emotional Stability, Agreeableness, Conscientiousness, Excitability, and Skepticism seem job-relevant across samples, predictive of performance, and unlikely to cause adverse impact. Focusing on these characteristics in hiring could contribute to positive changes in police performance.

A diagnostic tool for people with lumbar instability: a criterion-related validity study

BackgroundSeveral clinical tests used to identify patients with lumbar instability have reported diagnostic accuracy in separate studies with conflicting results. To augment the diagnostic process, tests that are better able to identify lumbar instability suitable for use in the clinical setting are required. The aim of this study was to identify the probability to diagnose patients with lumbar instability, using x-ray imaging as the reference standard.MethodsThis study was a cross-sectional, diagnostic validity study. One hundred forty participants with chronic low back pain underwent an x-ray assessment and 14 clinical examinations. Data were analysed using multivariate regression methods to determine which clinical tests were most diagnostic for lumbar instability when they were applied together.ResultsEighteen (12.85%) participants had radiological lumbar instability. Three clinical tests i) interspinous gap change during flexion-extension, ii) passive accessory intervertebral movement tests, iii) posterior shear test demonstrated an ability to diagnose lumbar instability of 67% when they were all positive. At this probability threshold, sensitivity, specificity, positive likelihood ratio (+LR), and negative likelihood ratio (−LR) were 5.56, 99.18%, 6.78, and 0.95.ConclusionsThese 3 clinical tests could be useful in identifying patients with lumbar instability in the general community. These three tests are simple to perform by physical therapists, reliable to use in a clinical setting, and safe for patients. We recommend physical therapists use these three tests to assess patients who are suspected of having lumbar instability, in the absence of an x-ray assessment, to receive appropriate targeted intervention or referral for further investigation.Trial registrationThai Clinial Trial Registry (TCTR 20180820001; 19th August 2018).

The Selfitis Behavior Scale: An Adaptation Study

This study aimed to adapt the Selfitis Behavior Scale (SBS) to measure university students' selfitis behavior in Turkey. Within the scope of the study, the SBS was translated from English to Turkish. Data were collected in the validity and reliability studies from four study groups, including 343 university students. According to the confirmatory factor analysis findings, the SBS had good fit indices. As is expected, the criterion-related validity study found moderate- and highlevel positive correlations between the SBS subscales and total scores and the Selfie Attitude Scale (SAS) subscales and total scores. The other criterion-related validity study demonstrated that individuals taking a higher number of selfies than the sum of the scale and subscales scored significantly higher than individuals taking a lower number of selfies. The internal consistency coefficient was calculated to be .92 for the SBS total score, .78 for the Environmental Enhancement Subscale (SBS-EE), .68 for the Social Competition Subscale (SBS-SC), .72 for the AttentionSeeking Subscale (SBS-AS), .84 for the Mood Modification Subscale (SBS-MM), .82 for the SelfConfidence Subscale (SBS-S), and .70 for the Subjective Conformity Subscale (SBS-SCon). The test-retest correlation values found by applying the measurement tool once in four weeks were .77 (SBS), .70 (SBS-EE), .60 (SBS-SC), .82 (SBS-AS), .80 (SBS-MM), .61 (SBS-S), and .61 (SBSSCon), respectively. The mentioned findings demonstrated that the SBS was a valid and reliable measurement tool capable of measuring the selfitis behavior of university students in Turkey.

The Machines Aren’t Taking Over (Yet): An Empirical Comparison of Traditional, Profiling, and Machine Learning Approaches to Criterion-Related Validation

Criterion-related validation (CRV) studies are used to demonstrate the effectiveness of selection procedures. However, traditional CRV studies require significant investment of time and resources, as well as large sample sizes, which often create practical challenges. New techniques, which use machine learning to develop classification models from limited amounts of data, have emerged as a more efficient alternative. This study empirically investigates the effectiveness of traditional CRV with a variety of profiling approaches and machine learning techniques using repeated cross-validation. Results show that the traditional approach generally performs best both in terms of predicting performance and larger group differences between candidates identified as top or non-top performers. In addition to empirical effectiveness, other practical implications are discussed.

Measuring the motive for power using conditional reasoning: some preliminary findings

ABSTRACT The conditional reasoning approach to personality assessment is a promising new method that has the potential to predict a range of work-related outcomes. Aiming to expand the approach to organizational leadership field, we describe six studies that explored the feasibility of using conditional reasoning to measure the motive for power. We identified an initial set of items based on work by Lawrence James and denoted this set the Conditional Reasoning Test for Power (CRT-P). In Study 1, we investigated CRT-P’s convergent and discriminant validity. In Studies 2a and 2b, we tested the implicit nature and fakeability of the CRT-P, by comparing test responses and eye-movement data between honest responding and simulated selection conditions. In Studies 3a and 3b, using large samples of employees, we tested if CRT-P scores predicted whether individuals occupied leadership positions. Finally, in Study 4 we tested whether CRT-P scores are related to subordinate ratings of leadership effectiveness. Our findings gave certain support to the CRT-P’s convergent and discriminant validity. Both faking studies indicated that the test is less fakeable compared to traditional self-report surveys, whereas the criterion-related validity studies demonstrated that CRT-P scores correlated both with organizational leadership occupancy and leadership effectiveness ratings.

Developing The Adolescent Happiness Scale: Validity and reliability study

The purpose of the current study is to develop the Adolescent Happiness Scale (AHS). A systematic approach was utilized for developing the scale. In this study, the data were collected from 1136 adolescents including 490 females and 646 males between 11-17 years of age. The psychometric properties of AHS were analyzed by means of item analysis, exploratory and confirmatory factor analyses, criterion-related validity, internal consistency and test-retest methods. The exploratory and confirmatory factor analyses results revealed that the scale had a single-factor structure. Within the scope of the criterion-related validity studies, significant positive correlations were found between AHS and positive affects and overall happiness levels, and a significant negative correlation was found between AHS and negative affects. The internal consistency coefficient of AHS was .92 for the first study and .91 for the second study. Test-retest reliability coefficient was determined to be .74. In conclusion, AHS can be recommended as a reliable and valid tool in measuring the happiness level of adolescents.

Turkish Adaptation of Caring Climate Scale and Reviewing Psychometry Properties: Validation and Reliability Study

The research objective in this study is to make adaptation of Caring Climate Scale (CCS) that is developed by Newton et al. (in 2007) to Turkish and examine its psychometric properties. The scale comprises of one-dimension and 13 items. 468 students who have been studying and doing physical exercise as certified athletes in the following Schools of Physical Education and Sports of Ahi Evran University, Marmara University, Akdeniz University, Selcuk University and Ege University constitute the research group of the study. Internal consistency, factor analysis of substance, test-retest and criterion-related validity studies applied for determining the psychometric properties of scale. The Cronbach’s alpha internal consistency coefficient value of the scale was determined as 0.801 for total point. The obtained data from the exploratory and confirmatory factor analysis has corroborative features for the single-factor structure of scale. Besides, the scale provided high-test–test-retest scores in analyses conducted. In this context, it is evaluated that the scale is a valid and reliable measuring instrument for Turkish sample group.

The adaptation of the Bergen Social Media Addiction Scale to Turkish and the evaluation its relationships with depression and anxiety symptoms

Objective: Social media addiction has become a considerable problem with the widespread use of social media. The aim of this research is to examine the psychometric properties of the Turkish version of the Bergen Social Media Addiction Scale, which was developed based on the 6-dimensional addiction model. Another aim of the study is to investigate the relationship between social media addiction and anxiety and depression symptoms. Methods: Bergen Social Media Addiction Scale is translated into Turkish by the translation-back translation method. Ex-ploratory factor analysis and confirmatory factor analysis were used to examine the construct validity of the Turkish version of the scale. The reliability was examined by internal consistency and test re-test method. Relationships between social media addiction and anxiety and depression symptoms were analyzed by correlation and regression analysis. Results: As a result of exploratory factor analysis for construct validity, it was found that the scale was in a one-dimensional structure as in the original form. The results of the confirmatory factor analysis indicated that the one-dimensional model wasan acceptable fit. In the criterion-related validity study, it was found that the scale had a positive correlation with the Internet Addiction Scale. The internal consistency reliability and test re-test reliability coefficient of the scale was found to be acceptable. It was found that social media addiction predicted the anxiety and depression symptoms positively. Conclusions: According to the findings of this research can be said of Bergen Social Media Addiction Scale it can be used in studies to be conducted in Turkey. Preventive studies are recom-mended because of the increase in social media addiction may increase anxiety and depression symptoms.

Same Book, Different Bookmarks: The Development and Preliminary Validation of the Bible Verse Selection Task as a Measure of Christian Fundamentalism

The development and preliminary validation of a new measure of Christian fundamentalism required a multi-stage process. In an initial exploratory study, participants indicated which of a set of Bible verses were most central to their faith, and factor analysis was used to identify verses that appeared to tap a latent dimension of religious fundamentalism (Study 1). These relationships were retested with a new method in a new sample (Study 2), and the items that predicted fundamentalism in both samples were incorporated into a new measure of Christian fundamentalism, the Bible Verse Selection Task (BVST). The forced-choice format of the BVST may be less impacted by social desirability response styles that may affect scores on existing fundamentalism scales (Studies 3 and 4) while preserving useful levels of criterion-related validity (Study 5) and convergent evidence of construct validity (Study 6). These studies provide initial psychometric evidence for the BVST as an internally consistent measure of Christian fundamentalism that predicts scores on other fundamentalism scales and related constructs including traditionalism, authoritarianism, and political conservativism.

Applying Item Response Trees to Personality Data in the Selection Context

Self-report personality scales are used frequently in personnel selection. Traditionally, researchers have assumed that individuals respond to items within these scales using a single-decision process. More recently, a flexible set of item response (IR) tree models have been developed that allow researchers to investigate multiple-decision processes. In the present research, we found that IR tree models fit the data better than a single-decision IR model when fitted to seven self-report personality scales used in a concurrent criterion-related validity study. In addition, we found evidence that the latent variable underlying the direction of a response (agree or disagree) decision process predicted job performance better than latent variables reflecting the other decision processes for the best fitting IR tree model.

Social anxiety questionnaire (SAQ): Development and preliminary validation

BackgroundThe Social Anxiety Questionnaire (SAQ) was designed to assess five dimensions of social anxiety as posited by the Clark and Wells' (1995; Clark, 2001) cognitive model. MethodsThe development of the SAQ involved generation of an item pool, followed by a verification of content validity and the theorized factor structure (Study 1). The final version of the SAQ was then assessed for reliability, temporal stability (test re-test reliability), and construct, criterion-related, and contrasted-group validity (Study 2, 3, and 4). ResultsFollowing a systematic process, the results provide support for the SAQ as reliable, and both theoretically and empirically valid measure. A five-factor structure of the SAQ verified and replicated through confirmatory factor analyses reflect five dimensions of social anxiety: negative self-processing; self-focused attention and self-monitoring; safety behaviours; somatic and cognitive symptoms; and anticipatory and post-event rumination. LimitationsResults suggest that the SAQ possesses good psychometric properties, while recognizing that additional validation is a required future research direction. It is important to replicate these findings in diverse populations, including a large clinical sample. ConclusionsThe SAQ is a promising measure that supports social anxiety as a multidimensional construct, and the foundational role of self-focused cognitive processes in generation and maintenance of social anxiety symptoms. The findings make a significant contribution to the literature, moreover, the SAQ is a first instrument that offers to assess all, proposed by the Clark–Wells model, specific cognitive-affective, physiological, attitudinal, and attention processes related to social anxiety.

Development of a reaction scale for dementia patients during recreational activities: a trial study of interobserver reliability and criterion-related validity

Development of a reaction scale for dementia patients during recreational activities: a trial study of interobserver reliability and criterion-related validity

Communicating Criterion-Related Validity Using Expectancy Charts: A New Approach

Often, personnel selection practitioners present the results of their criterion-related validity studies to their senior leaders and other stakeholders when trying to either implement a new test or validate an existing test. It is sometimes challenging to present complex, statistical results to non-statistical audiences in a way that enables intuitive decision making. Therefore, practitioners often turn to expectancy charts to depict criterion-related validity. There are two main approaches for constructing expectancy charts (i.e., use of Taylor-Russell tables or splitting a raw dataset), both of which have considerable limitations. We propose a new approach for creating expectancy charts based on the bivariate-normal distribution. The new method overcomes the limitations inherent in the other two methods and offers a statistically sound and user-friendly approach for constructing expectancy charts.

Clinical assessment of thoracic axial rotation: A criterion-related validity study of a digital inclinometer and iPhone

Clinical assessment of thoracic axial rotation: A criterion-related validity study of a digital inclinometer and iPhone

Why Some Situational Judgment Tests Fail To Predict Job Performance (and Others Succeed)

Situational judgment tests (SJTs) occasionally fail to predict job performance in criterion-related validation studies, often despite much effort to follow scholarly recipes for their development. This commentary provides some plausible explanations for why this may occur as well as some tips for SJT development. In most cases, we frame the issue from an implicit trait policy (ITP) perspective (Motowidlo, Hooper, & Jackson, 2006a, 2006b) and the measurement of general domain knowledge. In other instances, we believe that the issue does not have a direct tie to the ITP concept, but our experience suggests that the issue is of sufficient importance to include in this response. The first two issues involve challenges gathering validity evidence to support the use of SJTs, and the remaining issues deal more directly with SJT design considerations.

Development and validation of a short form aggressive beliefs and attitudes scale

This research outlines six studies (total N=3867) that develop and validate an Aggressive Beliefs and Attitudes — Short Form scale for use within the normal nonclinical adult population (e.g., organizational psychology). In the first two samples, exploratory factor analysis reduced the original 30 item scale to a shorter, more parsimonious, eight item measure. In the third sample, confirmatory factor analysis found that the proposed model and items fit the data extremely well. Examination of the nomological network underlying the new measure in Samples 3–4 displayed relationships with positive affect, negative affect, agreeableness, conscientiousness, and neuroticism ranging from r=.23 to −.23, while relationships with anger, hostility, physical aggression, and verbal aggression ranged from r=.49 to .30. Finally, in two criterion-related validity studies the relationships between the aggressive beliefs and attitudes scales, both original and short forms, and workplace deviance were examined with independent samples of employed adults. The short form scale was significantly related to workplace deviance in both samples (r=.37 and .43). Furthermore, these relationships were of identical magnitude to the full 30 item measure, suggesting that the short form scale comparably captures aggression-related behaviors with a greatly reduced number of items.