Automating the analysis of voting systems

Abstract

Voting is a well-known technique to combine the decisions of peer experts. It is used in fault tolerant applications to mask errors from one or more experts using n-modular redundancy (NMR) and n-version programming. Voting strategies include: majority, weighted voting, plurality; instance runoff voting, threshold voting, and the more general weighted k-out-of-n systems. Before selecting a voting schema for a particular application, we have to understand the various tradeoffs and parameters and how they impact the correctness, reliability, and confidence in the final decision made by a voting system. In this paper, we propose an enumerated simulation approach to automate the behavior analysis of voting schemas with application to majority and plurality voting. We conduct synthetic studies using a simulator that we develop to analyze results from each expert, apply a voting mechanism, and analyze the voting results. The simulator builds a decision tree and uses a depth-first traversal algorithm to obtain the system reliability among other factors. We define and study the following behaviors: 1) the probability of reaching a consensus, Pc; 2) reliability of the voting system, R; 3) certainly index, T; and 4) the confidence index, C. The parameters controlling the analysis are the number of participating experts, the number of possible output symbols that can be produced by an expert, the probability distribution of each expert's output, and the voting schema. The paper presents an enumerated simulation approach for analyzing voting systems which can be used when the use of theoretical models are challenged by dependencies between experts or uncommon probability distributions of the expert's output.