Forensic Voice Comparison Research Articles

Balancing validity and reliability as a function of sampling variability in forensic voice comparison

In forensic comparison sciences, experts are required to compare samples of known and unknown origin to evaluate the strength of the evidence assuming they came from the same- and different-sources. The application of valid (if the method measures what it is intended to) and reliable (if that method produces consistent results) forensic methods is required across many jurisdictions, such as the England & Wales Criminal Practice Directions 19A and UK Crown Prosecution Service and highlighted in the 2009 National Academy of Sciences report and by the President’s Council of Advisors on Science and Technology in 2016. The current study uses simulation to examine the effect of number of speakers and sampling variability and on the evaluation of validity and reliability using different generations of automatic speaker recognition (ASR) systems in forensic voice comparison (FVC). The results show that the state-of-the-art system had better overall validity compared with less advanced systems. However, better validity does not necessarily lead to high reliability, and very often the opposite is true. Better system validity and higher discriminability have the potential of leading to a higher degree of uncertainty and inconsistency in the output (i.e. poorer reliability). This is particularly the case when dealing with small number of speakers, where the observed data does not adequately support density estimation, resulting in extrapolation, as is commonly expected in FVC casework.

Do long-term acoustic-phonetic features and mel-frequency cepstral coefficients provide complementary speaker-specific information for forensic voice comparison?

A growing number of studies in forensic voice comparison have explored how elements of phonetic analysis and automatic speaker recognition systems may be integrated for optimal speaker discrimination performance. However, few studies have investigated the evidential value of long-term speech features using forensically-relevant speech data. This paper reports an empirical validation study that assesses the evidential strength of the following long-term features: fundamental frequency (F0), formant distributions, laryngeal voice quality, mel-frequency cepstral coefficients (MFCCs), and combinations thereof. Non-contemporaneous recordings with speech style mismatch from 75 male Australian English speakers were analyzed. Results show that 1) MFCCs outperform long-term acoustic phonetic features; 2) source and filter features do not provide considerably complementary speaker-specific information; and 3) the addition of long-term phonetic features to an MFCCs-based system does not lead to meaningful improvement in system performance. Implications for the complementarity of phonetic analysis and automatic speaker recognition systems are discussed.

Modeling Lexical Tones for Speaker Discrimination.

Fundamental frequency (F0) has been widely studied and used in the context of speaker discrimination and forensic voice comparison casework, but most previous studies focused on long-term F0 statistics. Lexical tone, the linguistically structured and dynamic aspects of F0, has received much less research attention. A main methodological issue lies on how tonal F0 should be parameterized for the best speaker discrimination performance. This paper compares the speaker discriminatory performance of three approaches with lexical tone modeling: discrete cosine transform (DCT), polynomial curve fitting, and quantitative target approximation (qTA). Results show that using parameters based on DCT and polynomials led to similarly promising performance, whereas those based on qTA generally yielded relatively poor performance. Implications modeling surface tonal F0 and the underlying articulatory processes for speaker discrimination are discussed.

Effect of identical twins on deep speaker embeddings based forensic voice comparison

Deep learning has gained widespread adoption in forensic voice comparison in recent years. It is mainly used to learn speaker representations, known as embedding features or vectors. In this work, the effect of identical twins on two state-of-the-art deep speaker embedding methods was investigated with special focus on metrics of forensic voice comparison. The speaker verification performance has been assessed using the likelihood-ratio framework by likelihood ratio cost and equal error rate. The AVTD twin speech dataset was applied. The results show a significant reduction in speaker verification performance when twin samples are present. Neither the adaptation of LR score calculation to twin samples, nor fine-tuning the pre-trained speaker embedding models seemed to be able to leverage this limitation. It was found that the recognition of same or different speakers was possible even in the case of identical twins but the performance dropped greatly. The lowest EER of the best performing model was 3.4% in the case of non-twin; at the same time, EER was 25.3% when twins were present. This doesn’t mean that the presented methods are useless in case of identical twins, but it must be taken into consideration that in case of a higher likelihood-ratio score (which indicates same speakers on the tested samples), the possibility of twins must also be considered in a real casework.

Speaker identification in courtroom contexts – Part III: Groups of collaborating listeners compared to forensic voice comparison based on automatic-speaker-recognition technology

Expert testimony is only admissible in common-law systems if it will potentially assist the trier of fact. In order for a forensic-voice-comparison expert’s testimony to assist a trier of fact, the expert’s forensic voice comparison should be more accurate than the trier of fact’s speaker identification. “Speaker identification in courtroom contexts – Part I” addressed the question of whether speaker identification by an individual lay listener (such as a judge) would be more or less accurate than the output of a forensic-voice-comparison system that is based on state-of-the-art automatic-speaker-recognition technology. The present paper addresses the question of whether speaker identification by a group of collaborating lay listeners (such as a jury) would be more or less accurate than the output of such a forensic-voice-comparison system. As members of collaborating groups, participants listen to pairs of recordings reflecting the conditions of the questioned- and known-speaker recordings in an actual case, confer, and make a probabilistic consensus judgement on each pair of recordings. The present paper also compares group-consensus responses with “wisdom of the crowd” which uses the average of the responses from multiple independent individual listeners.

Forensic Audio and Voice Analysis: TV Series Reinforce False Popular Beliefs

People’s perception of forensic evidence is greatly influenced by crime TV series. The analysis of the human voice is no exception. However, unlike fingerprints—with which fiction and popular beliefs draw an incorrect parallel—the human voice varies according to many factors, can be altered deliberately, and its potential uniqueness has yet to be proven. Starting with a cursory examination of landmarks in forensic voice analysis that exemplify how the voiceprint fallacy came about and why people think they can recognize people’s voices, we then provide a thorough inspection of over 100 excerpts from TV series. Through this analysis, we seek to characterize the narrative and aesthetic processes that fashion our perception of scientific evidence when it comes to identifying somebody based on voice analysis. These processes converge to exaggerate the reliability of forensic voice analysis. We complement our examination with plausibility ratings of a subset of excerpts. We claim that these biased representations have led to a situation where, even today, one of the main challenges faced by forensic voice specialists is to convince trial jurors, judges, lawyers, and police officers that forensic voice comparison can by no means give the sort of straightforward answers that fingerprints or DNA permit.

Likelihood Ratio Based Voice Comparison Using Cepstral Coefficients and GAN

In order to identify suspects in digital forensics, Forensic Voice Comparison (FVC) entails comparing trace voices with known suspect voice samples. The FVC uses semi-automatic approaches to identify speakers using acoustic features. Two Australian datasets such as forensic evaluation dataset 1 & English dataset 2 were used. The current research on FVC samples is noise-reduced using a stationary noise reduction algorithm within a likelihood ratio-based framework. To demonstrate the power of knowledge representation, acoustic features like Mel Frequency Cepstral Coefficients(MFCC) and Linear Predictive Cepstral Coefficients (LPCC) were extracted. The suspect is then identified by classifying the model using the Generative Adversarial Network(GAN) and Artificial Neural Network (ANN). The accuracy is 80% & 90% for MFCC for dataset 1 & 2. Similarly, accuracy is 20% & 40% for LPCC for dataset 1 & 2. The overall comparison of acoustic features with two different datasets the MFCC performs better accurate results for Australian English dataset 2 with varying sample sizes.

Forensic Speaker Recognition: Review of Methods and Evaluations

Abstract: Forensic voice comparison is quite challenging because the quality of speech recordings in casework is often poor and there is often mismatch between the recording conditions of the known- and questioned-speaker recordings. Further, the field of forensic voice comparison is characterized by a large diversity of methods, procedures and evaluation of comparison results. The present paper aims to provide a concise review of various comparison methods and assessment of comparison results. The need of an extensive amount of experimental studies and statistical evaluation on a large database is highlighted to use the likelihoodratio framework for reporting conclusions that has been now firmly established as a theoretical framework for any forensic discipline.

Evidential value of voice quality acoustics in forensic voice comparison

Voice recordings in forensic voice comparison casework typically involve speech style mismatch and are separated by days or weeks, but studies that aim to empirically validate the evidential value of speech features rarely include systematic comparisons on contemporaneous vs. non-contemporaneous recordings and match vs. mismatch in speech style. This study addresses this gap and focuses on the acoustics of laryngeal voice quality, since voice quality has been reported to be one of the most popular and useful features for forensic voice comparison. 75 male speakers aged 18–45 were selected from a forensically-oriented database of Australian English speakers in Sydney/New South Wales. The evidential strength of a number of spectral tilt and additive noise parameters were tested under the Bayesian likelihood-ratio framework. Results show that system performance using these parameters as input were stable across 50 replications. When speech style is controlled for, VQ parameters yielded promising results and better system validity was achieved when using more VQ parameters. However, they offered limited speaker-discriminatory value when speech style mismatch is involved, and non-contemporaneous recordings only led to a small decline in performance. Overall, forensic practitioners should be cautious when using spectral tilt measures and additive noise measures as speaker discriminants in forensic casework.

Effects of language mismatch in automatic forensic voice comparison using deep learning embeddings.

In forensic voice comparison, deep learning has become widely popular recently. It is mainly used to learn speaker representations, called embeddings or embedding vectors. Speaker embeddings are often trained using corpora mostly containing widely spoken languages. Thus, language dependency is an important factor in automatic forensic voice comparison, especially when the target language is linguistically very different from that the model is trained on. In the case of a low-resource language, developing a corpus for forensic purposes containing enough speakers to train deep learning models is costly. This study aims to investigate whether a model pre-trained on multilingual (mostly English) corpus can be used on a target low-resource language (here, Hungarian), not represented by the model. Often multiple samples are not available from the offender (unknown speaker). Samples are therefore compared pairwise with and without speaker enrollment for suspect (known) speakers. Two corpora are used that were developed especially for forensic purposes and a third that is meant for traditional speaker verification. Speaker embedding vectors are extracted by the x-vector and ECAPA-TDNN techniques. Speaker verification was evaluated in the likelihood-ratio framework. A comparison is made between the language combinations (modeling, LR calibration, and evaluation). The results were evaluated by Cllrmin and EER metrics. It was found that the model pre-trained on a different language but on a corpus with a significant number of speakers can be used on samples with language mismatch. Sample duration and speaking style also seem to affect the performance.

The voice as a material clue: a new forensic Algerian Corpus.

Dialects have received bigger interest in recent years as they are increasingly used on the web and social media. Because Algerian Arabic dialects suffer from a lack of appropriate speech corpora for speech recognition, a rich dialect corpus is needed to approach Algerian Accent recognition. The latter remains a key feature in the field of Forensic Voice Comparison (FVC) systems. This paper presents a new large-scale forensic Algerian speech corpus called Sawt El-Djazaïr. An important criterion in dealing with forensic corpora is the presence of session variability. For this purpose, we collected celebrity recordings in various regions of Algeria, from different social networks, in various scenarios, and at different times. In addition, we also recorded 87 participants using cellular calls and voice over IP (VoIP) applications including Viber, WhatsApp, and Google Meet. The corpus of approximately 50hours covers various speech topics and is spoken in twelve Algerian sub-dialects. The design guidelines of the proposed corpus are described along with the grouping of dialects across different geographical locations. Sawt El-Djazaïr is available to the research community upon request.

What does method validation look like for forensic voice comparison by a human expert?

Method validation has gained traction within forensic speech science. The community recognises the need to demonstrate that the analysis methods used are valid, but finding a way to do so has been more straightforward for some analysis methods than for others. This article addresses the issue of method validation for the Auditory Phonetic and Acoustic (AuPhA) approach to forensic voice comparison. Although it is possible to take inspiration from general regulatory guidance on method validation, it is clear that these cannot be transposed on to all forensic analysis methods with the same degree of success. Particularly with respect to an analysis method like AuPhA, and in a field of the size and characteristics of forensic speech science, a bespoke approach to method validation is required. In this article we address the discussions that have been taking place around method validation, and illustrate one possible solution to demonstrating the validity of voice comparison by a human expert using the AuPhA method. In doing so we consider the constraints placed on sole practitioners, which generally go unacknowledged.

The impact of mismatched recordings on an automatic-speaker-recognition system and human listeners

The so-called ‘mismatch’ is a factor which experts in the forensic voice comparison field encounter regularly. Therefore, we decided to explore to what extent the automatic-speaker-recognition system’s and the earwitness’ ability to identify speakers is influenced when recordings are acquired in different languages and at different times. 100 voices in a database of 300 recordings (100 speakers recorded in three mutually mismatched sessions) were compared with an automatic-speaker-recognition software VOCALISE based on i-vectors and x-vectors, and by 39 respondents in simulated voice parades. Both the automatic and perceptual approach seem to have yielded similar results in that the less complex the mismatch type, the more successful the identification. The results point to the superiority of the x-vector approach, and also to varying identification abilities of listeners.

A plague on both your houses: The debate about how to deal with ‘inconclusive’ conclusions when calculating error rates

To the Editor, There is an ongoing debate about how to deal with ‘inconclusive’ conclusions when calculating error rates. Recent contributions to this debate include, in chronological order, Dror and Scurich (2020), Weller and Morris (2020), Biedermann and Kotsoglou (2021), Arkes and Koehler (2021), Dror (2022) and Arkes and Koehler (2022). All the proposed solutions are inappropriate because they do not address the real problem. The real problem is not what to do with ‘inconclusive’ conclusions but the fact that forensic practitioners report conclusions as ‘identification’, ‘inconclusive’ and ‘exclusion’ at all. The real solution is for practitioners to abandon this practice and, instead, adopt the logically correct framework for interpretation of forensic evidence, the likelihood-ratio framework. Rather than reporting each of their conclusions as one of three categories, practitioners should report them as continuously-valued likelihood-ratio values.1 When conducting tests, if the test pair is a same-source pair, then a good output by the forensic-evaluation system would be a large positive log-likelihood-ratio value, a poorer output would be a small positive log-likelihood-ratio value, a worse output would be a small negative log-likelihood-ratio value, and a much worse output would be a large negative log-likelihood-ratio value. Mutatis mutandis, for a different-source pair, for which a good output would be a large negative log-likelihood-ratio value.2 When calculating error rates in a categorical framework, a penalty value of 1 is assigned to each response that is the incorrect category and a penalty value of 0 is assigned to each response that is the correct category, then the penalty values are averaged. In contrast, an appropriate validation metric for a system that outputs likelihood ratios assigns a small penalty value to each log-likelihood-ratio value that is large in the correct direction, a less small penalty value to each log-likelihood-ratio value that is small in the correct direction, a large penalty value to each log-likelihood-ratio value that is small in the incorrect direction, and an even larger penalty value to each log-likelihood-ratio value that is large in the incorrect direction, then the penalty values are averaged. The penalty values are assigned using a continuously-valued function of the continuously-valued log-likelihood-ratio outputs of the forensic-evaluation system. A well-established validation metric that has these properties is the log-likelihood-ratio cost (Cllr; Brümmer and du Preez, 2006). It has been in use in forensic science since at least 2007 (González-Rodríguez et al., 2007), and its use is recommended in the recently-published Consensus on validation of forensic voice comparison (Morrison et al., 2021).3 An appendix in the latter publication provides an introduction to Cllr and its interpretation.

The effect of sampling variability on overall performance and individual speakers’ behaviour in likelihood ratio-based forensic voice comparison

The effect of sampling variability on overall performance and individual speakers’ behaviour in likelihood ratio-based forensic voice comparison

Speaker identification in courtroom contexts – Part I: Individual listeners compared to forensic voice comparison based on automatic-speaker-recognition technology

Expert testimony is only admissible in common law if it will potentially assist the trier of fact to make a decision that they would not be able to make unaided. The present paper addresses the question of whether speaker identification by an individual lay listener (such as a judge) would be more or less accurate than the output of a forensic-voice-comparison system that is based on state-of-the-art automatic-speaker-recognition technology. Listeners listen to and make probabilistic judgements on pairs of recordings reflecting the conditions of the questioned- and known-speaker recordings in an actual case. Reflecting different courtroom contexts, listeners with different language backgrounds are tested: Some are familiar with the language and accent spoken, some are familiar with the language but less familiar with the accent, and others are less familiar with the language. Also reflecting different courtroom contexts: In one condition listeners make judgements based only on listening, and in another condition listeners make judgements based on both listening to the recordings and considering the likelihood-ratio values output by the forensic-voice-comparison system.

Sub-band cepstral distance as an alternative to formants: Quantitative evidence from a forensic comparison experiment

This paper demonstrates the potential of the sub-band parametric cepstral distance (PCD) formulated by Clermont and Mokhtari (1994), as an alternative to formants in acoustic phonetic research. As a cepstrum-based measure, the PCD is automatically and reliably extracted from the speech signal. By contrast, formants are time-consuming and often difficult to estimate, a well-known bottleneck for studies based on large-scale datasets. The PCD measure gives flexibility in selecting the frequency limits of any sub-band of interest within the available full band. We suggest that, if sub-band selection were guided by the acoustic–phonetic theory of speech production, PCD analysis could facilitate phonetically meaningful cepstral comparisons without relying directly on formants. We evaluate this idea by exploiting the PCD properties in the context of forensic voice comparison as an application example. The cepstral data were obtained from the vowels uttered by 306 male Japanese speakers. Similar patterns of results were observed using formants and sub-band PCDs, the latter yielding better performance. This suggests that sub-band PCDs are able to capture the spectral characteristics that we normally quantify through formants, but with better reliability and efficiency. The PCD results reported here are encouraging for other types of acoustic phonetic studies in which comparisons of spectral characteristics are required.

How Voice Analysis Can Help Solve Crimes

Imagine you are a police officer investigating a crime in which somebody has made a threatening phone call. There may not be any physical evidence like DNA or fingerprints, but the person’s voice is captured on a recording. Experts can analyze the voice recording and compare it with the voice of a known suspect to find out how likely it is that they are the same person. This method is called forensic voice comparison. It works because people’s voices contain lots of information about them. Your voice tells a story about where you grew up and learned to speak, and your voice also depends on your individual biological make-up. In this article, we explain how this information is used to compare recordings of voices in criminal cases.

The effect of sampling variability on systems and individual speakers in likelihood ratio-based forensic voice comparison

The likelihood ratio (LR) framework has been widely adopted in voice (and other forensic) evidence evaluation. However, in developing any forensic comparison system, it is necessary to make subjective and pragmatic decisions, which in turn may affect the results that system produces. One such decision relates to not only the size of the samples used for training and testing the system, but also which specific individuals are used in the samples. The current study explores the relationship between sampling variability (i.e. the choice of speakers used for training and testing systems, rather than sample size) and the choice of linguistic features used. The first three formants and f0 from the vocalic portion of the filled pause um were used as input, as well as both vowel and nasal durations. 25 speakers were used in test, training and reference sets respectively. Experiments were carried out using all 31 logically possible combinations of features, and replicated 100 times using different configurations of 25 training and reference speakers. The results show that (a) overall, Cllr mean reduces with more features involved and no clear pattern is observed in Cllr range; meanwhile, considerable fluctuation is observed within individual speakers; (b) while the majority of speakers yield stronger mean LLRs in systems with three or more features, a few speakers can be well-separated using one or two features; (c) sampling variability in the training and reference speakers has limited effect on individual test speakers’ LR outputs in same-speaker (SS) comparisons, but a marked effect on different-speaker (DS) LRs.

Forensic comparison of fired cartridge cases: Feature-extraction methods for feature-based calculation of likelihood ratios

We describe and validate a feature-based system for calculation of likelihood ratios from 3D digital images of fired cartridge cases. The system includes a database of 3D digital images of the bases of 10 cartridges fired per firearm from approximately 300 firearms of the same class (semi-automatic pistols that fire 9 mm diameter centre-fire Luger-type ammunition, and that have hemispherical firing pins and parallel breech-face marks). The images were captured using Evofinder®, an imaging system that is commonly used by operational forensic laboratories. A key component of the research reported is the comparison of different feature-extraction methods. Feature sets compared include those previously proposed in the literature, plus Zernike-moment based features. Comparisons are also made of using feature sets extracted from the firing-pin impression, from the breech-face region, and from the whole region of interest (firing-pin impression + breech-face region + flowback if present). Likelihood ratios are calculated using a statistical modelling pipeline that is standard in forensic voice comparison. Validation is conducted and results are assessed using validation procedures and validation metrics and graphics that are standard in forensic voice comparison.