A Replication of Two Free Text Keystroke Dynamics Experiments under Harsher Conditions

Abstract

Replication of experiments lies at the very core of the scientific process. In spite of this, the relevance and relative count of replication studies has fallen sharply in the recent past in several disciplines and a similar trend is apparent in computing science and software engineering. Keystroke dynamics studies have not been exempted from this rule, where replication studies are still no more than a handful and have often lead to astonishingly varying error rates in comparison with the original study, both for static passwords and free texts. The effect of the typing environment and the user emotional state is obviously a concern for free text analysis, as much as his physiological states like stress and tiredness. Thus, the real world performance of a certain method in a specific setting could vary drastically compared with the laboratory setting even though the latter might not be consciously biased. In this paper we examine and compare the performance of two techniques for keystroke dynamics analysis in a free text dataset under evaluation conditions which are harsher than the originally used: the free text extension to the R and A distances method of Bergadano, Gunetti and Picardi and the authors' method based on finite context modeling. Some of their key properties proved to be extrapolatable outside the realm of ideal conditions; the impostor pass rate of A and R distances showed little change, combined distances proved consistently better than pure ones and their best combination remained the same. Others did not, like the relative efficacy of n-graphs, the performance of A distances and -expectedly- the false alarm and correct classification rate for both methods.