Lineup Fairness Research Articles

How to measure lineup fairness: concurrent and predictive validity of lineup-fairness measures

ABSTRACT The current study examined the concurrent and predictive validity of four families of lineup-fairness measures – mock-witness measures, perceptual ratings, face-similarity algorithms, and resultant assessments (assessments based on eyewitness participants’ responses) – with 40 mock crime/lineup sets. A correlation analysis demonstrated weak or non-significant correlations between the mock-witness measures and the algorithms, but the perceptual ratings correlated significantly with both the mock-witness measures and the algorithms. These findings may reflect different task characteristics – pairwise similarity ratings of two faces versus overall similarity ratings for multiple faces – and suggest how to use algorithms in future eyewitness research. The resultant assessments did not correlate with the other families, but a multilevel analysis showed that only the resultant assessments – which are based on actual eyewitness choices – predicted eyewitness performance reliably. Lineup fairness, as measured using actual eyewitnesses, differs from lineup fairness as measured using the three other approaches.

The Jena Eyewitness Research Stimuli (JERS): A database of mock theft videos involving two perpetrators, presented in 2D and VR formats with corresponding 2D and 3D lineup images

Empirical investigations into eyewitness identification accuracy typically necessitate the creation of novel stimulus materials, which can be a challenging and time-consuming task. To facilitate this process and promote further research in this domain, we introduce the new Jena Eyewitness Research Stimuli (JERS). They comprise six video sequences depicting a mock theft committed by two different perpetrators, available in both two-dimensional (2D) and 360° format, combined with the corresponding lineup images presented in 2D or three-dimensional (3D) format. Images of one suspect and eight fillers are available for each lineup. We evaluated lineup fairness by using mock eyewitness paradigm and noted a Tredoux’s E of 4.687 for Perpetrator 1 and 5.406 for Perpetrator 2. Moreover, no bias towards the perpetrators was observed in the lineups. We incorporated 360° videos and 3D lineup images to encourage the adoption of innovative data formats in experimental investigations of eyewitness accuracy. In particular, compatibility with Virtual Reality (VR) makes JERS a promising tool for advancing eyewitness research by enabling researchers to construct controlled environments that offer observers an immersive experience. JERS is freely accessible for the use of academic purposes via the Open Science Framework (OSF).

Measuring lineup fairness from eyewitness identification data using a multinomial processing tree model

The mock-witness task is typically used to evaluate the fairness of lineups. However, the validity of this task has been questioned because there are substantial differences between the tasks for mock witnesses and eyewitnesses. Unlike eyewitnesses, mock witnesses must select a person from the lineup and are alerted to the fact that one lineup member might stand out from the others. It therefore seems desirable to base conclusions about lineup fairness directly on eyewitness data rather than on mock-witness data. To test the importance of direct measurements of biased suspect selection in eyewitness identification decisions, we assessed the fairness of lineups containing either morphed or non-morphed fillers using both mock witnesses and eyewitnesses. We used Tredoux’s E and the proportion of suspect selections to measure lineup fairness from mock-witness choices and the two-high threshold eyewitness identification model to measure the biased selection of the suspects directly from eyewitness identification decisions. Results obtained in the mock-witness task and the model-based analysis of data obtained in the eyewitness task converged in showing that simultaneous lineups with morphed fillers were significantly more unfair than simultaneous lineups with non-morphed fillers. However, mock-witness and eyewitness data converged only when the eyewitness task mimicked the mock-witness task by including pre-lineup instructions that (1) discouraged eyewitnesses to reject the lineups and (2) alerted eyewitnesses that a photograph might stand out from the other photographs in the lineup. When a typical eyewitness task was created by removing these two features from the pre-lineup instructions, the morphed fillers no longer lead to unfair lineups. These findings highlight the differences in the cognitive processes of mock witnesses and eyewitnesses and they demonstrate the importance of measuring lineup fairness directly from eyewitness identification decisions rather than indirectly using the mock-witness task.

Experimental validation of a multinomial processing tree model for analyzing eyewitness identification decisions

To improve police protocols for lineup procedures, it is helpful to understand the processes underlying eyewitness identification performance. The two-high threshold (2-HT) eyewitness identification model is a multinomial processing tree model that measures four latent cognitive processes on which eyewitness identification decisions are based: two detection-based processes (the detection of culprit presence and absence) and two non-detection-based processes (biased and guessing-based selection). The model takes into account the full 2 × 3 data structure of lineup procedures, that is, suspect identifications, filler identifications and rejections in both culprit-present and culprit-absent lineups. Here the model is introduced and the results of four large validation experiments are reported, one for each of the processes specified by the model. The validation experiments served to test whether the model’s parameters sensitively reflect manipulations of the processes they were designed to measure. The results show that manipulations of exposure duration of the culprit’s face at encoding, lineup fairness, pre-lineup instructions and ease of rejection of culprit-absent lineups were sensitively reflected in the parameters representing culprit-presence detection, biased suspect selection, guessing-based selection and culprit-absence detection, respectively. The results of the experiments thus validate the interpretations of the parameters of the 2-HT eyewitness identification model.

A validation of the two-high threshold eyewitness identification model by reanalyzing published data

The two-high threshold (2-HT) eyewitness identification model serves as a new measurement tool to measure the latent cognitive processes underlying eyewitness identification performance. By simultaneously taking into account correct culprit identifications, false innocent-suspect identifications, false filler identifications in culprit-present and culprit-absent lineups as well as correct and false lineup rejections, the model capitalizes on the full range of data categories that are observed when measuring eyewitness identification performance. Thereby, the model is able to shed light on detection-based and non-detection-based processes underlying eyewitness identification performance. Specifically, the model incorporates parameters for the detection of culprit presence and absence, biased selection of the suspect and guessing-based selection among the lineup members. Here, we provide evidence of the validity of each of the four model parameters by applying the model to eight published data sets. The data sets come from studies with experimental manipulations that target one of the underlying processes specified by the model. Manipulations of encoding difficulty, lineup fairness and pre-lineup instructions were sensitively reflected in the parameters reflecting culprit-presence detection, biased selection and guessing-based selection, respectively. Manipulations designed to facilitate the rejection of culprit-absent lineups affected the parameter for culprit-absence detection. The reanalyses of published results thus suggest that the parameters sensitively reflect the manipulations of the processes they were designed to measure, providing support of the validity of the 2-HT eyewitness identification model.

Validity of mock-witness measures for assessing lineup fairness

ABSTRACT Although eyewitness researchers have used mock-witness measures to assess aspects of lineup fairness, they have paid little attention to their validity. The current study tested predictive validity, convergent validity, and discriminant validity of mock-witness measures from a meta-analytic perspective. Overall, mock-witness measures had predictive validity, particularly in target-absent (TA) lineups – the lineup fairness estimated by the measures reliably predicted eyewitnesses’ choosing behaviors and discriminability of a suspect from fillers in TA lineups. However, correlations between lineup fairness estimated by mock-witnesses and eyewitness performance were significant in target-present (TP) lineups only when eyewitnesses had a moderate memory for the perpetrator. Multitrait-multimethod correlations demonstrated significant intradomain correlations between mock-witness measures and other lineup fairness indices and nonsignificant interdomain correlations between the mock-witness measures and indices reflecting memory strength for the perpetrator, which supported convergent validity and discriminant validity, respectively. The implications for research and practice are discussed.

Lineup fairness: propitious heterogeneity and the diagnostic feature-detection hypothesis

Researchers have argued that simultaneous lineups should follow the principle of propitious heterogeneity, based on the idea that if the fillers are too similar to the perpetrator even an eyewitness with a good memory could fail to correctly identify him. A similar prediction can be derived from the diagnostic feature-detection (DFD) hypothesis, such that discriminability will decrease if too few features are present that can distinguish between innocent and guilty suspects. Our first experiment tested these predictions by controlling similarity with artificial faces, and our second experiment utilized a more ecologically valid eyewitness identification paradigm. Our results support propitious heterogeneity and the DFD hypothesis by showing that: 1) as the facial features in lineups become increasingly homogenous, empirical discriminability decreases; and 2) lineups with description-matched fillers generally yield higher empirical discriminability than those with suspect-matched fillers.

Evaluating lineup fairness: Variations across methods and measures.

Triers of fact sometimes consider lineup fairness when determining the suggestiveness of an identification procedure. Likewise, researchers often consider lineup fairness when comparing results across studies. Despite their importance, lineup fairness measures have received scant empirical attention and researchers inconsistently conduct and report mock-witness tasks and lineup fairness measures. We conducted a large-scale, online experiment (N = 1,010) to examine how lineup fairness measures varied with mock-witness task methodologies as well as to explore the validity and reliability of the measures. In comparison to descriptions compiled from multiple witnesses, when individual descriptions were presented in the mock-witness task, lineup fairness measures indicated a higher number of plausible lineup members but more bias toward the suspect. Target-absent lineups were consistently estimated to be fairer than target-present lineups-which is problematic because it suggests that lineups containing innocent suspects are less likely to be challenged in court than lineups containing guilty suspects. Correlations within lineup size measures and within some lineup bias measures indicated convergent validity and the correlations across the lineup size and lineup bias measures demonstrated discriminant validity. The reliability of lineup fairness measures across different descriptions was low and reliability across different sets of mock witnesses was moderate to high, depending on the measure. Researchers reporting lineup fairness measures should specify the type of description presented, the amount of detail in the description, and whether the mock witnesses viewed target-present and/or -absent lineups. (PsycINFO Database Record

Supplemental Material for Evaluating Lineup Fairness: Variations Across Methods and Measures

Supplemental Material for Evaluating Lineup Fairness: Variations Across Methods and Measures

Showups versus lineups: An evaluation using ROC analysis

Showups (a one-person identification) were compared to both simultaneous and sequential lineups that varied in lineup fairness and the position of the suspect in the lineup. We reanalyzed data from a study by Gronlund, Carlson, Dailey, and Goodsell (2009), which included simultaneous and sequential lineups, and using the same stimuli and procedures, collected new data using showup identifications. Performance was compared using ROC analysis, which is superior to traditional measures such as correct and false identification rates, and probative value measures. ROC analysis showed that simultaneous lineups consistently produced more accurate identification evidence than showups, but sequential lineups were sometimes no more accurate than showups, and were never more accurate than simultaneous lineups. These results supported prior suppositions regarding the suggestiveness of showups, revealed a misconception about the superiority of sequential lineups, and demonstrated why eyewitness identification procedures need to be evaluated using ROC analyses.

Searching for the sequential line-up advantage: A distinctiveness explanation

Gronlund (2005) proposed that one factor leading to a sequential line-up advantage could be the greater likelihood of recollecting distinctive information about a perpetrator when using a sequential line-up. Since then questions have been raised about the robustness of the sequential advantage and the possible moderating role of line-up fairness and suspect position. We manipulated these factors as well as suspect/target distinctiveness in two experiments. A sequential advantage occurred only after encoding a distinctive target, both for biased line-ups (Experiment 1) and fair line-ups (Experiment 2). Remember-Know results were consistent with the greater use of a recall-to-reject strategy in target-absent sequential line-ups. This provided support for the first process-based explanation of the sequential line-up advantage. No consistent position effects were found, but this might be due to the line-up recognition paradigm used, in which each participant viewed a line-up for each of several targets. Theory-based explorations of eyewitness identification are necessary to continue to delineate the underpinnings of the sequential line-up advantage.

Lineup composition, suspect position, and the sequential lineup advantage.

N. M. Steblay, J. Dysart, S. Fulero, and R. C. L. Lindsay (2001) argued that sequential lineups reduce the likelihood of mistaken eyewitness identification. Experiment 1 replicated the design of R. C. L. Lindsay and G. L. Wells (1985), the first study to show the sequential lineup advantage. However, the innocent suspect was chosen at a lower rate in the simultaneous lineup, and no sequential lineup advantage was found. This led the authors to hypothesize that protection from a sequential lineup might emerge only when an innocent suspect stands out from the other lineup members. In Experiment 2, participants viewed a simultaneous or sequential lineup with either the guilty suspect or 1 of 3 innocent suspects. Lineup fairness was varied to influence the degree to which a suspect stood out. A sequential lineup advantage was found only for the unfair lineups. Additional analyses of suspect position in the sequential lineups showed an increase in the diagnosticity of suspect identifications as the suspect was placed later in the sequential lineup. These results suggest that the sequential lineup advantage is dependent on lineup composition and suspect position.

Predicting Eyewitness Identification Accuracy with Mock Witness Measures of Lineup Fairness: Quality of Encoding Interacts with Lineup Format

After viewing a staged crime under varying levels of encoding optimality, eyewitnesses made identifications from lineups that were either presented to them as a simultaneous line of people, or individually, in sequence. Lineups did not contain the perpetrator of the staged crime, and were purposefully constructed to have varying levels of bias. There was an interaction between encoding optimality and type of lineup, with sequential-presentation lineups leading to more correct rejections than simultaneous-presentation lineups for moderate encoding optimality. Sequential-presentation lineups did not provide good protection against mistaken identifications in the two other conditions of encoding optimality. Measures of the fairness of the lineups significantly predicted identification rates for both simultaneous-presentation lineups and sequential-presentation lineups, regardless of encoding optimality.

A NATIONAL SURVEY OF US POLICE ON PREPARATION AND CONDUCT OF IDENTIFICATION LINEUPS

A great deal of research has been aimed at identifying the factors that produce errors in eyewitness identification. However, most of this work has been conducted in laboratory environments using undergraduates and naive lay persons as research participants. Little information is available on what police officers do in the course of their identification activities. The present research investigated the procedures that police officers report employing when constructing and administering eyewitness identification lineups. We developed a survey that addresses a number of these issues, including experience, sources of lineup foils, lineup formation and display, lineup fairness evaluation, presence of suspect's attorney, witness instructions, historical records, and legal challenges. Surveys were sent to 500 US police jurisdictions; 220 were returned. Several survey items showed results consistent with previous laboratory research, however police officers reported using some procedures that are different from those established as most effective in the research literature. This paper summarizes the results of our survey and discusses the implications for future research and forensic procedures.

Are police video identifications fair to African–Caribbean suspects?

AbstractAnalysis of lineups from criminal cases has demonstrated that video technology can produce lineups that are less biased against the suspect than live lineups, and that White suspects are less likely to be identified from a live lineup than suspects of other ethnic origins. The present study assessed the fairness of video lineups of White Europeans and of African–Caribbeans used in actual criminal cases. African–Caribbean and White European participants selected the suspect from each lineup on the basis of the original witness description of the culprit. There was no reliable difference in the fairness of video lineups as a function of the ethnic origin of the lineup members. It is concluded that, within the context of the video system studied, use of video can provide a safeguard against bias against ethnic minorities that may occur in live lineups. Copyright © 2003 John Wiley & Sons, Ltd.

Validity of the mockwitness paradigm: testing the assumptions.

Mockwitness identifications are used to provide a quantitative measure of lineup fairness. Some theoretical and practical assumptions of this paradigm have not been studied in terms of mockwitnesses' decision processes and procedural variation (e.g., instructions, lineup presentation method), and the current experiment was conducted to empirically evaluate these assumptions. Four hundred and eighty mockwitnesses were given physical information about a culprit, received 1 of 4 variations of lineup instructions, and were asked to identify the culprit from either a fair or unfair sequential lineup containing 1 of 2 targets. Lineup bias estimates varied as a result of lineup fairness and the target presented. Mockwitnesses generally reported that the target's physical description was their main source of identifying information. Our findings support the use of mockwitness identifications as a useful technique for sequential lineup evaluation, but only for mockwitnesses who selected only 1 lineup member. Recommendations for the use of this evaluation procedure are discussed.

A direct measure of facial similarity and its relation to human similarity perceptions.

Research is reported on a measure of facial similarity in which the similarity of 2 faces is defined as the Euclidean distance between them in a principal-component space. Five studies were conducted in which participants rated sets of facial images, and in which the measure was applied to 2 problems in the eyewitness literature. Comparisons of ratings with distances derived from the principal-component analysis suggest that the measure corresponds reasonably well to perceptions of facial similarity. In addition, the measure correlates strongly with empirical measures of lineup fairness and is related to eyewitness identification performance. Further potential applications include a software tool for constructing arrays of faces of varying similarity, and a software tool for reconstructing facial images from memory.

Postdictors of eyewitness errors: can false identifications be diagnosed?

Eyewitness researchers have "postdicted" identification accuracy using witness confidence (S. L. Sporer, S. Penrod, D. Read, & B. Cutler, 1995), response latency (S. L. Sporer, 1993, 1994), and endorsement of statements consistent with using relative versus absolute judgment strategies (D. Dunning & L. B. Stern, 1994; R. C. L. Lindsay & K. Bellinger, 1999). All of these measures were collected from 321 introductory psychology students who had viewed a staged crime and completed a lineup identification task. Some participants received feedback after identification (G. L. Wells & A. L. Bradfield, 1998). Lineup fairness was also used as a postdictor of eyewitness accuracy. Discriminant function analysis indicated that 75.2% of choosers and 63.0% of nonchoosers were correctly classified. Decision time and lineup fairness were the best postdictors of accuracy. The implications for postdicting real eyewitness decisions are discussed.

Measures of lineup fairness: do they postdict identification accuracy?

Various measures have been proposed to test the fairness of lineups (Doob & Kirshenbaum, 1973; Malpass, 1981; Wells, Leippe and Ostrom, 1979). In three experiments, we examined the relationship of identification accuracy to measures of lineup bias (proportion of mock witness choices, defendant bias, functional size) and lineup size (effective size, number of acceptable lineup members). Lineup bias measures successfully postdict false positive choices, but only for some lineup procedures. Lineup size measures rarely postdict false positive decisions. Lineup fairness measures do not postdict false identifications from sequential lineups. Implications for measuring and reporting lineup fairness in court are discussed. Copyright © 1999 John Wiley & Sons, Ltd.

Measuring lineup fairness

SUMMARY The fairness of line-ups and photospreads is a traditional concern of research and policy development in the area of eyewitness identification. Quantification of fairness, the construction of fairness indices, and the development of evaluation procedures started in the 1970s and continues to this day. This paper reviews the historical development of the field as an introduction to the articles that follow. The entire set of articles addresses current questions and raises new issues of measuring the fairness of identification procedures. Copyright # 1999 John Wiley & Sons, Ltd. The literature on the measurement of line-up fairness began with the report by Doob and Kirschenbaum (1973) of their attempt to measure the bias contained in an identification process brought to their attention by a defence attorney. The verbal description of the oAender was extremely vague. She stated only that the criminal was ‘rather good looking’. The possibility existed that the witness had simply chosen the best-looking person from the line-up, whether he was the oAender or not. To test this conjecture, Doob and Kirschenbaum gave the description of the criminal and a photograph of the line-up to a set of persons who had never seen the criminal. They were asked to choose the suspect from the line-up. If these ‘mock witnesses’ (Wells et al., 1979) were able to identify the suspect, never having seen him but solely on the basis of the verbal description, the line-up could be considered biased in the sense that the alternative line-up members were not full (eAective, functional) alternatives to the suspect. The underlying assumption of Doob and Kirschenbaum’s method was that the alternatives (distractors, fillers, or foils) in a line-up ought to protect innocent suspects from identifications based solely on their match to the witness’s description of the criminal (recall). Instead, the witness should have to rely on the presumably more informative data provided by recognition of the face as a match of the suspect’s appearance to the witness’ memory image derived from having witnessed the original event (Wells, 1993). Thus line-up foils ought to be plausible choice alternatives: the suspect should not stand out in comparison to them. The reductio ad absurdum would be to place a suspect described as a European‐American in a line-up with an African‐American, an Asian‐American, an elephant, and a refrigerator. Doob and Kirschenbaum’s (1973) procedure oAered an attractive paradigm for post hoc evaluation of fairness in line-ups and photospreads, through the use of the