Responses to Past-Behavior Questions in Face-To-Face and Asynchronous Video Interviews: Storytelling, Interview Performance and Criterion Validity

Asynchronous video interviews (AVIs) have become popular selection methods due to their flexibility and cost savings but might introduce new forms of bias. For instance, interviewees often complete them from home, their surroundings might signal personal or protected statuses, and technology issues might distort the information provided. This paper leverages two complementary studies to examine (a) the AVI completion decisions, recording quality, and background elements present in high- and low-stakes job interviews, (b) to what extent these AVI-specific elements and interviewees’ characteristics can bias performance ratings, and (c) whether evaluation standardization can help mitigate such biases. Study 1 used mock interviews with (N = 626) Prolific participants evaluated by professional hiring managers. Study 2 involved high-stakes interviews with (N = 523) real applicants for competitive education programmes evaluated by trained raters using either standardized or unstandardized approaches. AVI elements (attire, room tidiness, technical issues, background) were coded in both studies. Results showed that completion decisions depended on AVI stakes and could influence evaluations. Issues with recording quality were rare and modestly related to AVI evaluations. AVI backgrounds signalling personal or protected statuses were very rare and unrelated to evaluations. Evaluations standardization reduced bias only in relation to sex-based differences, but not other interviewee characteristics.

The use of artificial intelligence (AI)-scored video interviews has increased significantly in the past five years and gained mainstream momentum with the shift to remote prehire tools during the pandemic. Asynchronous video interviews (AVIs) hold the promise of a more efficient hiring process, that is more candidate friendly, with less bias than traditional screening methods, and that meets the measurement standards of the industrial-organizational profession. However, little published research has documented these benefits. This chapter summarizes much of the established research on AVIs and the AI methods to score these interviews, presents a methodology for designing autoscored AVIs, and discusses the latest published (and unpublished) psychometric results concerning reliability, convergent validity, criterion validity, and adverse impact. The chapter submits that these results are on par with the published research on structured interviews, and thus the scale and scope of using AVIs for prehire needs are above and beyond traditional assessment methods. The chapter also discusses the utility of multidisciplinary approaches to designing innovative talent assessments and reminds readers of the benefits of following IO assessment principles in the design of these novel approaches.

ABSTRACTThe personnel selection field has observed the rising use of asynchronous video interviews (AVI). The current study investigates whether follow‐up questions (probes) can optimize the applicant experience in AVIs. Across two experimental studies with participants recruited from Prolific, we investigated whether AVIs with probing promote applicant reactions (e.g., the opportunity to perform perceptions) toward the AVI and how probing influences interview behaviors, applicant perceptions, and interview performance ratings. In Study 1, 404 participants were randomly assigned to either an AVI with probing or an AVI without probing. Results indicated that probing directly improved the opportunity to perform perceptions and interview performance ratings. In addition, probing positively impacted honest impression management and motivation to perform indirectly through participants' perceived opportunity to perform. However, mediation analyses suggested that the effect of probing on interview performance ratings was driven by response length. In Study 2 (n = 271), we teased apart the effects of the inherently added response time that probing affords applicants with an additional condition that matched the response time of probes. Relative to Study 1, probing only slightly improved the opportunity to perform perceptions, but the effect of probing on the opportunity to perform perceptions was greater when compared to an AVI with an equivalent response time. In addition, probing positively impacted interview performance ratings, above and beyond their increased response time. Implications, limitations, and directions for future research are discussed.

Does the use of synchrony and artificial intelligence in video interviews affect interview ratings and applicant attitudes?

Since COVID-19 pandemic, the use of asynchronous video interviews(AVIs) has increased in the selection process. The purpose of this study was to investigate the effects of the interviewer video on the applicants' reactions and impression management behavior during the AVIs process. Accordingly, interviewees were randomly assigned to three conditions and participated in the AVIs. The first condition is that every process of the AVIs was conducted through text only. The second and the third condition both provide videos that ask questions and listen to the applicants' answers, but the former includes virtual character interviewers while the latter includes real person interviewers. Results indicated that the most positive interviewees' reactions were shown in the third condition. In detail, the third condition showed higher social presence, interpersonal treatment, and perceived usefulness compared to the first condition in the AVIs process, and the interviewer video that includes real person interviewers had positive effects on procedural justice and organizational attractiveness through these variables. In the case of the second condition, there was no significant difference in the applicants’ reactions to the first condition, and it showed a lower social presence than in the third condition. Hence it indicates that the virtual characters cannot substitute the positive effects of real person interviewers. Also, there was no notable difference in honesty and deceptive impression management behavior between the three conditions. This study is meaningful in that it proposed an effective way to improve the participants’ reactions in conducting the AVIs, and is expected to be used as a basis for research related to the construction of AVIs.

Some research suggests that job applicants tend to express negative perceptions of asynchronous video interviews (AVIs). Drawing from basic psychological needs theory, we proposed that these negative perceptions arise partly from the lack of human interaction between applicants and the organization during an AVI, which fails to satisfy applicants' need for relatedness. Recruiting participants through Prolific, we conducted two experimental studies that aimed to manipulate the level of relatedness support through a relatedness‐need supportive introductory video containing empathetic messaging and humor. Using a vignette approach, participants in study 1 (N = 100) evaluated a hypothetical AVI that included one of two introductory videos: relatedness‐supportive versus neutral messaging. The relatedness‐supportive video yielded higher relatedness need satisfaction (d = 0.53) and organizational attraction ratings (d = 0.49) than the neutral video. In study 2, participants (N = 231) completed an AVI that included one of the two videos and evaluated their AVI experience. In contrast to the vignette study, we observed no significant differences between groups for relatedness need satisfaction, organizational attraction, nor other outcomes. Our findings provided little evidence that humor and empathic video messaging improves reactions to an AVI and illustrated the limitations on the external validity of vignette designs.

Ready? Camera rolling… action! Examining interviewee training and practice opportunities in asynchronous video interviews

The efficiency of asynchronous video interviews (AVIs) — on-demand, fully digital job interviews — makes them increasingly appealing to hiring managers. However, little is known about the psychological drivers shaping important hiring outcomes, such as applicants’ intentions to accept job offers and self-disclose with AVIs. Across one field study and three controlled experiments, this research demonstrates that job applicants report lower job acceptance intentions following asynchronous video interviews (AVIs) compared to face-to-face interviews. Moreover, job applicants exhibit a lower psychological comfort with AVIs, which, in turn, reduces their job acceptance intentions and willingness to self-disclose (Study 2a) even when controlling for perceived behavioural control, privacy concerns, and affective commitment (Study 2b). Finally, Study 3 demonstrates that these effects were stronger for job applicants with higher scores on interpersonal trust propensity (Study 3). Together, these results reveal the psychological mechanisms through which AVIs affect job acceptance intentions and self-disclosure, deepen our understanding of the individual differences moderating applicants’ experience of AVIs, and provide actionable insights for integrating AVIs into hiring practices.

Better job application systems: Objectively assessing measures of job performance from asynchronous video interviews

To assess the effect of using of asynchronous video interviewing as a screening tool for obstetrics and gynecology residency selection. This project was part of a quality-improvement effort to enhance the resident application process. Applications to a single obstetrics and gynecology residency program were scored using standardized criteria. In the 2018 Match, top-scored applicants were invited to in-person interviews, and second-tier applicants were asked to complete a three-question asynchronous video interview. Video interviews were scored and used to invite the remaining applicants for in-person interviews. In the 2019 Match, video interviewing was expanded to all applicants with top application scores, and the video score was used to determine in-person interview invitations. Applicants for 2019 were surveyed on their views regarding video interviewing. Half of the candidates interviewed in person in the 2018 season were screened by the video interview process compared with 82% in the 2019 season. The mean in-person interview score increased from 59.0 in 2017, before screening with asynchronous video interviews, to 62.2 in 2018 (effect size 0.50; 95% CI 0.09-0.90) In 2018, a nonsignificant correlation was seen between the video interview score and rank list percentile (r=0.22, P=.15, n=27) and in-person interview score (r=0.18, P=.12, n=46). United States Medical Licensing Examination step 1 and 2 scores were not correlated with video interview scores in either season. Most (58%) candidates indicated that the asynchronous video interview was an effective way to tell their story; however, only 42% were comfortable with the interview process. Video interviewing may have promise as a tool for program directors to use to select candidates from a competitive applicant pool by measuring important noncognitive skills. Acceptance by obstetrics and gynecology applicants was mixed.

Asynchronous video interviews (AVIs) have become increasingly popular as alternatives (or complements) to more traditional face-to-face interviews. Yet, AVI research has been largely focused on applicant reactions or behaviors, and we still know very little about what influences how applicants are rated. Importantly, because AVIs afford applicants the flexibility to record their responses from their homes, the background they choose could influence raters’ judgments. This study examines whether raters’ (N=276 Prolific respondents with prior hiring experience) initial impressions and final ratings differ if applicants record their AVIs from a home-office, a bedroom, or use background blurring settings, as well as the role played by response quality. Final interview scores were positively associated with both initial impressions and applicant response quality. Yet, background type (or the use of blurring) was not associated with initial impressions or final interview scores.

With the advent of new technology, traditional job interviews have been supplemented by asynchronous video interviews (AVIs). However, research on psychometric properties of AVIs is limited. In this study, 710 participants completed a mock AVI responding to eight personality questions (Extraversion, Conscientiousness). We collected self- and observer reports of personality, interview performance ratings, attractiveness, and AVI meta-information (e.g., professional attire, audio quality). Then, we automatically extracted the words, facial expressions, and voice characteristics from the videos and trained machine learning models to predict the personality traits and interview performance. Our algorithm explained substantially more variance in observer reports of Extraversion and Conscientiousness (average R2 = 0.32) and interview performance (R2 = 0.44), than self-reported Extraversion and Conscientiousness (average R2 = 0.12). Consistent with Trait Activation Theory, the explained variance in personality traits increased when participants responded to trait-relevant, compared to trait-irrelevant, questions. The test-retest reliability of our algorithm was somewhat stable over a time period of seven months, but lower than desired reliability standards in personnel selection. We examined potential sources of bias, including age, gender, and attractiveness, and found some instances of algorithmic bias (e.g., gender differences were often amplified in favor of women).

Into the void: A conceptual model and research agenda for the design and use of asynchronous video interviews

In the context of global technological advancements, asynchronous video interviews (AVIs) have emerged as an innovative tool in recruitment, offering potential to transform traditional hiring practices. This study aims to enhance understanding of the opportunities and challenges associated with AVIs in recruitment processes by examining the perspectives of human resources (HR) professionals in Türkiye. A qualitative research methodology with a phenomenological approach was employed. 15 HR professionals experienced in asynchronous video interviewing from diverse organizations participated in online, open-ended, semi-structured, in-depth interviews. The collected data were analyzed through thematic analysis using MAXQDA 2024. Findings indicated that AVIs offer significant opportunities, such as improving process efficiency, enhancing candidate experience, promoting fairness and inclusivity, and supporting organizational goals. However, the study identified critical challenges, including diminishing candidate experience, undermining fairness and increasing deceptive impression management, reducing job satisfaction among HR professionals, and imposing operational and financial constraints. As one of the first studies to explore HR professionals’ perceptions of AVIs in Türkiye, this study provides valuable insights into their adoption in developing economies and highlights the broader implications of AVIs in the global recruitment practices. The findings emphasize the need for tailored strategies to maximize benefits, address challenges, and balance the needs of both candidates and HR professionals.

Asynchronous video interviews (AVIs) are increasingly used by organizations in their hiring process. In this mode of interviewing, the applicants are asked to record their responses to predefined interview questions using a webcam via an online platform. AVIs have increased usage due to employers' perceived benefits in terms of costs and scale. However, little research has been conducted regarding applicants' reactions to these new interview methods. In this work, we investigate applicants' reactions to an AVI platform using self-reported measures previously validated in psychology literature. We also investigate the connections of these measures with nonverbal behavior displayed during the interviews. We find that participants who found the platform creepy and had concerns about privacy reported lower interview performance compared to participants who did not have such concerns. We also observe weak correlations between nonverbal cues displayed and these self-reported measures. Finally, inference experiments achieve overall low-performance w.r.t. to explaining applicants' reactions. Overall, our results reveal that participants who are not at ease with AVIs (i.e., high creepy ambiguity score) might be unfairly penalized. This has implications for improved hiring practices using AVIs.