Non-cognitive Attributes Research Articles

Cognitive performance and labour market outcomes

We use the U.S. National Longitudinal Survey of Youth 1979 and other sources to examine how cognitive performance near the end of secondary schooling relates to labour market outcomes through age fifty. Our preferred estimates control for individual and family backgrounds, non-cognitive attributes, and survey years. We find that returns to cognitive skills rise with age. Although estimated gains in lifetime incomes are close to those reported earlier, our preferred estimates make multiple offsetting improvements. Returns to cognitive skill are greater for blacks and Hispanics than for non-Hispanic whites, both in relative and absolute terms, with gains in work hours being more important than in hourly wages.

Process-oriented evaluation of an international faculty development program for Asian developing countries: a qualitative study

BackgroundNon-English-speaking developing countries in Southeast Asia have been provided only limited opportunities for faculty development in the education of health professions. Although there exist a few programs that have been shown to be effective, they are frequently presented with few explanations on how and why the programs work due to their outcome-oriented nature. This study explores the process of the Lee Jong-Wook Fellowship for Health Professional Education, an international faculty development program designed for capacity building of educators of health professions in Southeast Asian developing countries.MethodsFellows were from Cambodia, Myanmar, and Laos. Qualitative data were collected from two types of semi-structured interviews – group and individual. Thematic analysis was conducted to explore the factors related to the effectiveness of the program, framed by four components of faculty development, which included context, facilitators, program, and participants.ResultsFrom the thematic analysis, the authors identified a total of 12 themes in the four components of faculty development. In the context domain, the resource-poor setting, a culture that puts emphasis on hierarchy and seniority, and educational environment depending on individual commitment rather than broad consensus emerged as key factors. In the facilitators domain, their teaching methods and materials, mutual understanding between teacher and learner, and collaboration between facilitators mainly influenced the learning during the fellowship. In the program domain, the key advantages of the fellowship program were its applicability to the workplace of the fellows and enough allowed time for practice and reflection. Finally, in the participants domain, Fellows valued their heterogeneity of composition and recognized cognitive as well as non-cognitive attributes of the participants as essential.ConclusionsThis process-oriented evaluation reveals the diverse factors that contributed to achieving the intended outcomes of the fellowship. Although much evidence from best practices in faculty development are still valid, the findings suggest that the selection strategies, learning environment, and English communication should be given more consideration when organizing a program targeting these people and cultures. A comprehensive understanding of the process would contribute to developing tailored strategies for educators of health professions in developing countries in similar settings.

The Selection of Potential Undergraduate Students who Lack Customary Academic Qualifications: is a toolkit possible?

Widening participation in higher education is now seen as desirable in much of the West, if only for its economic benefits. It can, however, present problems for admissions tutors as applicants can vary widely in age and experience, and may lack conventional educational qualifications and material evidence of their potential for success. This study focused on the selection of students for a one-year, university-based, foundation course aimed at preparing students with diverse backgrounds for undergraduate study in a variety of disciplines. The aim was to consider the potential of 'non-cognitive attributes' in the selection process, and reflect on their inclusion (with caveats) in selection toolkits (assortments of ways used to identify students for admission). To that end, information was collected from research literature, the views of foundation studies tutors, potential students, existing students, and from correlates between various attributes and examination attainment. It was concluded that a single toolkit could not do justice to the applicants' variety. Instead, a toolbox and a choice of tools to reflect the age, gender, and target discipline would be better. Furthermore, it could strengthen decisions if admissions tutors were supported by teachers of the discipline concerned, in order to draw on their implicit discipline knowledge.

Admitting Deserving Medical Students from Rural and Disadvantaged: Patan Academy of Health Sciences' Approach.

Patan Academy of Health Sciences (PAHS) aims to produce physicians who would be able and willing to serve in the rural areas. Recognizing the critical importance of student selection strategy, among others, in achieving the program goals, it has adopted an innovative scheme for selecting medical students. This paper describes PAHS medical student selection scheme that favors enrollment of deserving applicants from rural and disadvantaged groups so as to help improve distribution of physicians in rural Nepal. A student admission committee comprising a group of medical educators finalized a three-step student selection scheme linked with scholarships after reviewing relevant literatures and consultative meetings with experts within and outside Nepal. The committee did local validation of Personal Quality Assessment (PQA) that tested cognitive ability and personality traits, Admission OSPE (Objective Structured Performance Examination) that assessed non-cognitive attributes of applicants. It also provided preferential credits to applicants' socio-economic characteristics to favor the enrollment of deserving applicants from rural and disadvantaged groups through Social Inclusion Matrix (SIM). Three different categories of scholarship schemes namely Partial, Collaborative and Full were devised with Partial providing 50% and other two categories each providing 100% coverage of tuition fee. PAHS student selection scheme succeeded in enrolling more than half of its students from rural areas of Nepal, including about 10% of the students from that of the most backward region of the country. About one third of students were female and about the same were from public and community school. Sixty percent of students receive different categories of scholarships. Limited findings indicate the success of the selection scheme in enrolling high proportion of applicants from rural and disadvantageous groups and enable them to pursue study by providing scholarships.

How important is personality in the selection of medical school students?

Admittance to medical school has traditionally been determined on the basis of students' performance on the Medical College Admission Test (MCAT) and grade point average (GPA), which assess cognitive abilities. To supplement these predictors, medical schools also consider a semi-structured interview, which assesses non-cognitive attributes. Successful performance as a medical student is determined by performance in courses and clinical rotations. The traditional cognitive predictors generally contribute to the prediction of course requirements. However, these traditional predictors often demonstrate weak relations with clinical performance, suggesting that other predictors are necessary. Using approximately 300 medical students, the current study investigated a) the ability of the traditional admission tools to predict course and clinical performance, and b) the incremental validity of personality predictors, which were chosen on the basis of a personality-oriented job analysis. The traditional predictors accounted for a significant proportion of variance in course performance, with personality accounting for incremental variance. Clinical performance was only predicted by personality and the traditional predictors did not demonstrate predictive validity.

A new admission system model for teacher colleges

Purpose Aspects of intellectual competence would not be sufficient for quality teaching that requires a mix of intellectual and personal qualities. The purpose of this paper was to elicit personal attributes of teachers’ college applicants. Design/methodology/approach This qualitative case study consisted of 99 participants aged 20-24 years of average socio-economic and achievement backgrounds. The authors constructed five qualitative tools, in addition to the two existing standardized exams. Using techniques of the grounded theory, the authors sorted and synthesized the data through three-phase coding. Findings The authors elicited a general concept map of attributes, from which a personal attribute profile for each participant emerged. This model makes fine differentiations between individuals, providing a personal attribute profile pool of applicants useful for any admission committee and for empowering students’ strengths during studies. Research limitations/implications A solid database of non-cognitive attributes opens the door to further research, which will take into consideration multiple variables, such as student knowledge, beliefs and aptitudes. Practical implications Training teachers to apply research methodology into practice and limiting the length of tools make the mission possible, interesting and useful. Figuring out how to collect valid measures of such data about large numbers of college students would be a major challenge. Social implications As teachers occupy a central position in the educational enterprise, they become guardians of the country’s collective socio-cultural legacy. In the current context of teacher shortages, the authors offer a model that has the potential of improving recruitment of applicants for teaching and raising teacher quality. Originality/value No previous attempt that uses qualitative methodology for this purpose was found.

The relative effects of socio-economic, demographic, non-cognitive and cognitive influences on student achievement in Australia

This paper examines socio-economic, demographic, non-cognitive and cognitive influences on early childhood cognitive ability and subsequent student achievement at school analyzing Australian longitudinal data. The clearly dominant influences on achievement are early childhood cognitive ability and prior achievement. Socio-economic status (which includes accurate measures of family income) has only moderate relationships with early childhood cognitive ability and student achievement. Its effects on student achievement are small when taking into account early childhood cognitive ability and especially prior achievement. This is also the case for family size, family type and to some extent, Indigenous status, but not for gender and language background. Of the non-cognitive attributes examined only ‘persistence’ has a moderate impact on student achievement. Fixed effects analyses show very small and statistically insignificant effects for family income and father's occupational status, but consistent, albeit small, effects for persistence. This study demonstrates that, contrary to popular belief, student achievement is not strongly linked to family income or socio-economic status.

Cognitive Performance and Labor Market Outcomes

We use information from the National Longitudinal Survey of Youth 1979 (NLSY79) and supplementary data sources to examine how cognitive performance, measured at approximately the end of secondary schooling, is related to the labor market outcomes of 20 through 50 year olds. Our estimates control for a wide array of individual and family background characteristics, a limited set of non-cognitive attributes, survey year dummy variables and, sometimes, geographic place effects. The analysis reveals five main findings. First, cognitive performance is positively associated with future labor market outcomes at all ages. The relationship is attenuated but not eliminated by the addition of controls for non-cognitive characteristics, while the inclusion of place effects does not change the estimated associations. Second, the returns to cognitive skill increase with age. Third, the effect on total incomes reflects a combination of positive impacts of cognitive performance for both hourly wages and annual work hours. Fourth, the returns to cognitive skill are greater for women than men and for blacks and Hispanics than for non-Hispanic whites, with differential effects on work hours being more important than corresponding changes in hourly wages. Fifth, the average gains in lifetime incomes predicted to result from greater levels of cognitive performance are only slightly above those reported in prior studies but the effects are heterogeneous, with larger relative and absolute increases, in most models, for nonwhites or Hispanics than for non-Hispanic whites, and higher relative but not absolute returns for women than men.

Use of the Interview in Resident Candidate Selection: A Review of the Literature.

Although the resident candidate interview is costly and time-consuming for both applicants and programs, it is considered critically important for resident selection. Noncognitive attributes, including communication skills and professionalism, can be assessed by the personal interview. We conducted a review of the literature on the residency interview to identify the interview characteristics used for resident selection and to ascertain to what extent the interview yields information that predicts future performance. We searched PubMed and Scopus using the following search terms: residency, internship, interview, selection, and performance. We extracted information on characteristics of the interview process, including type of interview format, measures taken to minimize bias by interviewers, and testing of other clinical/surgical skills. We identified 104 studies that pertained to the resident selection interview, with highly varied interview formats and assessment tools. A positive correlation was demonstrated between a medical school academic record and the interview, especially for unblinded interview formats. A total of 34 studies attempted to correlate interview score with performance in residency, with mixed results. We also identified a number of studies that included personality testing, clinical skills testing, or surgical skills testing. Our review identified a wide variety of approaches to the selection interview and a range of factors that have been studied to assess its effectiveness. More research needs to be done not only to address and ascertain appropriate interview formats that predict positive performance in residency, but also to determine interview factors that can predict both residents' "success" and program attrition.

Accelerated Nursing Degree Programs: Insights Into Teaching and Learning Experiences

The Study of Teaching and Learning in Accelerated Nursing Degree Programs explores how nurse educators are adapting their teaching practices for accelerated, second‐degree nursing program students. To provide findings on topics including instructional practices and the roles and attitudes of faculty, a web survey was administered to almost 100 staff members from schools of nursing that received grant and scholarship funds through the Robert Wood Johnson Foundation's New Careers in Nursing program for accelerated nursing students. The study revealed that nursing school faculty have positive perceptions of working with accelerated nursing students and that instructional approaches do not differ much between traditional and nursing students. At the same time, the factors described as most predictive of accelerated nursing student success were noncognitive attributes such as motivation and commitment to the nursing profession; prior degrees in the science or health fields were not necessarily seen as predictive of the success of second‐degree students working toward an accelerated bachelor's or master's degree in nursing.

Excellence Redefined for the 21st Century

Based on the perspectives of our symposium contributors, certain twenty-first century themes have become increasingly important in a global economy: 1) individuality; 2) connectedness; and 3) non-cognitive attributes. Individuals are asserting their influence in ways never seen before, thanks in part to technology and democracy. The innovators aren’t doing it alone ironically; they are collaborating, or connecting, with like-minded peers and thinking across disciplines. They also possess certain non-cognitive attributes like motivation, perseverance, and adaptability that employers are increasingly looking for. Recognizing these three themes and their place in education is, therefore, critical to twenty-first century excellence. In this concluding article, the author illuminates each theme and offers a more fitting education approach that hopefully reflects an evolved perspective since A Nation at Risk was first published.

Workers' Risk Tolerance and Occupational Injuries.

This study explores the relationship between individuals' risk tolerance and occupational injuries. We analyze data from a national representative survey of U.S. workers that includes information about injuries, risk tolerance, cognitive and noncognitive attributes, and risky behaviors. We measure risk tolerance through questions regarding individuals' willingness to gamble on their lifetime income. We estimate zero-inflated count models to assess the role played by such measures on workers' recurrent injuries. We discuss some implications of our results for future research and occupational safety policies. Our results highlight the concurrent and changing role played by individual, work, and environmental factors in explaining recurrent incidents. They show that risk tolerance affects recurrent injuries, although not in the direction that proponents of the concept of proneness would expect. Our measure of risk aversion shows that individuals who are somewhat more risk tolerant have fewer recurrent injuries than those who are risk averse. But the estimated relationship is U-shaped, not monotonic and, therefore, not easy to predict. At the same time, we find that individuals' "revealed risk preferences"-specific risky behaviors-are related to higher injury probabilities. Demanding working conditions, measures of socioeconomic status, health, and safety problems experienced by workers during their youth remain among the most important factors explaining the phenomena of recurrent injuries. So our results contribute also to the important debate about the relationship between health and socioeconomic status.

Non-cognitive attributes predict medical and nursing students’ intentions to migrate or work rurally: An eight-country cross-sectional survey in Asia and Africa

Outcomes & Evaluation: During the two months of active pilot project implementation, the number of ANC visits at the participating health center increased by 20%; however, these results were not sustained and ANC visit numbers returned to their pre-implementation baseline. Based on this data, and findings from the interviews conducted, the implementation phase has been increased from two months to one year. In order to enhance the evidence supporting this model, an expanded monitoring and evaluation plan will be implemented, including monitoring of the number of male partners presenting to the clinic for testing, and the number of births attended by a skilled birth attendant. Going Forward: The use of incentives, and cultural expectations surrounding them will be a continued topic of interest for this project. The context-specific appropriateness and effectiveness of monetary and non-monetary incentives will be monitored. Additionally, the strict doctrine of the Ethiopian Orthodox Church (EOC) surrounding sexuality limits the amount and type of outreach conducted by religious leaders; for example the EOC prohibits condom use. Respecting the authority and convictions of the church is paramount to the success of this program. Public health professionals working in this context must look for ways to compromise as they work toward mutually established goals. Funding: Strengthening Care Opportunities through Partnerships in Ethiopia (SCOPE). Abstract #: 01ETC108

Older Men as Learners

To date, little attention has been placed on older men (aged 50+ years) as learners, with much of the literature on adult learning concerned with younger age-groups and issues around gender equity directed mainly at women. This article examines the impact of community-based men’s sheds on informal and nonformal learning by older men in Ireland. It considers older men’s attitudes to learning, learning behavior, and the noncognitive attributes—motivation, perseverance, and beliefs about capabilities—that underpin learning behavior. This descriptive study used a mixed-methods approach, involving questionnaires and focus groups, with all sheds registered with the Irish Men’s Sheds Association invited to participate. It is concluded that men’s sheds provide space for hands-on learning activities that add value to the lives and experiences of men beyond work, fostering a yearning to carry on learning. Crucially, shed-based conversations have an important role in helping older men with difficult life transitions and are an important site for future studies of masculinity in later life.

The validity of a behavioural multiple-mini-interview within an assessment centre for selection into specialty training.

BackgroundEntry into specialty training was determined by a National Assessment Centre (NAC) approach using a combination of a behavioural Multiple-Mini-Interview (MMI) and a written Situational Judgement Test (SJT). We wanted to know if interviewers could make reliable and valid decisions about the non-cognitive characteristics of candidates with the purpose of selecting them into general practice specialty training using the MMI. Second, we explored the concurrent validity of the MMI with the SJT.MethodsA variance components analysis estimated the reliability and sources of measurement error. Further modelling estimated the optimal configurations for future MMI iterations. We calculated the relationship of the MMI with the SJT.ResultsData were available from 1382 candidates, 254 interviewers, six MMI questions, five alternate forms of a 50-item SJT, and 11 assessment centres. For a single MMI question and one assessor, 28% of the variance between scores was due to candidate-to-candidate variation. Interviewer subjectivity, in particular the varying views that interviewer had for particular candidates accounted for 40% of the variance in scores. The generalisability co-efficient for a six question MMI was 0.7; to achieve 0.8 would require ten questions. A disattenuated correlation with the SJT (r = 0.35), and in particular a raw score correlation with the subdomain related to clinical knowledge (r = 0.25) demonstrated evidence for construct and concurrent validity. Less than two per cent of candidates would have failed the MMI.ConclusionThe MMI is a moderately reliable method of assessment in the context of a National Assessment Centre approach. The largest source of error relates to aspects of interviewer subjectivity, suggesting enhanced interviewer training would be beneficial. MMIs need to be sufficiently long for precise comparison for ranking purposes. In order to justify long term sustainable use of the MMI in a postgraduate assessment centre approach, more theoretical work is required to understand how written and performance based test of non-cognitive attributes can be combined, in a way that achieves acceptable generalizability, and has validity.

Introduction to the Special Issue on Conceptualizing and Predicting Performance of Military Officers

Increased complexity and uncertainty in the operating environment for Army officers have highlighted the need to ensure that current selection and training systems are well aligned with evolving job requirements for these officers. This article introduces a special issue exploring recent efforts to meet this challenge. It addresses the need to fully understand the major components of an officer’s job, a topic more fully explored in two articles in this issue. It discusses research to more fully measure noncognitive attributes that contribute to officer performance than historical screening methods have attempted. Three articles in this issue demonstrate how this research has proceeded in three precommissioning contexts: Reserve Officers’ Training Corps (ROTC), Officer Candidate School (OCS), and the United States Military Academy (USMA). Finally, this article discusses considerations associated with implementing new selection and training tools, considerations particularly relevant to two of the articles that address implementation in two of these precommissioning contexts.

Council of Emergency Medicine Residency Directors' standardized letter of recommendation: the program director's perspective.

The standardized letter of evaluation (SLOE) was created in 1997 to provide residency program directors (PDs) with a summative evaluation that incorporates normative grading (i.e., comparisons to peers applying to emergency medicine [EM] training). Although the standard letter of recommendation (SLOR) has become increasingly popular and important in decision-making, it has not been studied in the past 12 years. To assess the SLOR's effectiveness and limitations, the perspective of EM PDs was surveyed in this study. After validation of the questionnaire by 10 retired PDs, the survey was sent to the PD of each of the 159 EM residencies that existed at that time. The survey was circulated via the Council of Emergency Medicine Residency Directors' (CORD) listserv from January 24, 2013, to February 13, 2013. Weekly e-mail reminders to all PDs served to increase participation. A total of 150 of 159 PDs (94.3%) completed the questionnaire. Nearly all respondents (149 of 150; 99.3%) agreed that the SLOR is an important evaluative tool and should continue to be used. In the application process, 91 of 150 (60.7%) programs require one or more SLORs, and an additional 55 (36.7%) recommend but do not require a SLOR to be considered for interview. When asked to identify the top three factors in deciding who should be interviewed, the SLOR was ranked first (139 of 150; 92.7%), with EM rotation grades ranked second (73 of 150; 48.7%). The factors that were most often identified as the top three that diminish the value of the SLOR in order were 1) "inflated evaluations" (121 of 146; 82.9%), 2) "inconsistency between comments and grades" (106 of 146; 72.6%), and 3) "inadequate perspective on candidate attributes in the written comments" and "inexperienced authors" (60 of 146; 41.1% each). The SLOR appears to be the most important tool in the EM PD's armamentarium for determining which candidates should be interviewed for residency training. Although valuable, the SLOR's potential utility is hampered by a number of factors, the most important of which is inflated evaluations. Focused changes in the SLOR template should be mindful that it appears, in general, to be successful in its intended purpose.

Imperfect physician assistant and physical therapist admissions processes in the United States.

We compared and contrasted physician assistant and physical therapy profession admissions processes based on the similar number of accredited programs in the United States and the co-existence of many programs in the same school of health professions, because both professions conduct similar centralized application procedures administered by the same organization. Many studies are critical of the fallibility and inadequate scientific rigor of the high-stakes nature of health professions admissions decisions, yet typical admission processes remain very similar. Cognitive variables, most notably undergraduate grade point averages, have been shown to be the best predictors of academic achievement in the health professions. The variability of non-cognitive attributes assessed and the methods used to measure them have come under increasing scrutiny in the literature. The variance in health professions students’ performance in the classroom and on certifying examinations remains unexplained, and cognitive considerations vary considerably between and among programs that describe them. One uncertainty resulting from this review is whether or not desired candidate attributes highly sought after by individual programs are more student-centered or graduate-centered. Based on the findings from the literature, we suggest that student success in the classroom versus the clinic is based on a different set of variables. Given the range of positions and general lack of reliability and validity in studies of non-cognitive admissions attributes, we think that health professions admissions processes remain imperfect works in progress.

The effectiveness of tools used to evaluate successful critical decision making skills for applicants to healthcare graduate educational programs: a systematic review protocol

The effectiveness of tools used to evaluate successful critical decision making skills for applicants to healthcare graduate educational programs: a systematic review protocol

Preferential Admission and MBA Outcomes: Mismatch Effects by Race and Gender

Abstract We consider the “mismatch” hypothesis in the context of graduate management education. Both blacks and Hispanics, conditional on a rich set of human capital variables, prior earnings and work experience, and non-cognitive attributes, are favored in admission to top 50 Master of Business Administration (MBA) programs. To test for mismatch effects, we provide two comparisons: (1) comparable individuals (in terms of race, gender, and credentials) at different quality MBA programs and (2) individuals of differing race or gender (but with similar credentials) at comparable MBA programs. Despite admission preferences, blacks and Hispanics enjoy similar or even higher returns to selectivity than whites.