Post-audits for managing cyber security investments: Bayesian post-audit using Markov Chain Monte Carlo (MCMC) simulation

Abstract

With increasing security spending in organizations, evaluation of the quality and effectiveness of IT security investments has become an important component in managing these projects. The academic literature, however, is largely silent on post-audit of such investments, which is a formal evaluation of IT resource allocation decisions. IT post-audits are considered a useful risk management tool for organizations and are often emphasized in security certifications and standards. To fill this research gap and contribute to practice, we suggest post-auditing of IT security investments using the generic Markov Chain Monte Carlo (MCMC) simulation approach. This approach does not place stringent conjugate assumptions and can handle high-dimensional Bayesian post-audit inference problems often associated with information security resource allocation decisions. We develop two Bayesian post-audit models using the MCMC method: (1) measuring the effectiveness of an IT security investment using posterior mean score ratios (MSR), and posterior crossover error rates (CER); and (2) measuring the effectiveness through detection of a denial of service (DOS) attack using Bayesian estimation to statistically compare the degree of divergence using the concept of entropy. We demonstrate the utility of the proposed methodology using an email intrusion detection system application.