Assessing the competence and credibility of human sources of intelligence evidence: contributions from law and probability

Abstract

These are perilous times in which our security is under continual threat by persons and organizations around the world and at home. Information supplied to us by human sources concerning the capabilities and intentions of these persons and organizations is crucial to our ability to recognize and prevent these threatening actions. We have all seen the many news accounts of our need for more and better HUMINT (human intelligence). But a major issue is the following: when we obtain an item of HUMINT, to what extent can we believe it? Our ability to make these assessments involves determining the competence and credibility of the human source providing this item. These assessments are very difficult and the persons making them need all the help they can get. Fortunately, much assistance comes from the fields of law and probability. This paper provides an account of how we are now exploiting the very rich legacy of experience and scholarship accumulated in the field of law over the past 500 years or so regarding questions to ask about the competence and credibility of witnesses. For quite some time, we have been attempting to bring valuable insights from the field of law concerning various evidential and inferential matters to the attention of persons in the intelligence community. On this occasion, we will describe a computer-assisted system called MACE (method for assessing the credibility of evidence), which we are developing to exploit what we have learned from law regarding assessments of the competence and credibility of sources of HUMINT. This system is designed to assist in addressing the question: to what extent can we believe this particular item of HUMINT that has just been supplied to us by a human source? MACE employs two different, but entirely complementary, probability systems to help us answer this question. Baconian probability methods help us answer the question: how much evidence do we have about this human source and how completely does it answer questions about the source's competence and credibility? Bayesian probability methods allow us to determine how strong is the evidence we do have about this particular human source, and it also provides an assessment of the posterior odds favouring the extent to which we can believe or disbelieve what this source is telling us.