Test sample size determination for biometric systems based on confidence elasticity

Abstract

Many researchers estimated the confidence interval of the evaluation index such as FMR and studied the relationship between the confidence interval width of FMR and sample size. In our research, we firstly indicated the relationship between confidence interval width w and sample size n by the equation: w^2=b1/n/n+b2/n. Apparently, the more test samples, the narrower of the confidence interval and the more convincible of the evaluation will come out. Most biometric test such as Fingerprint Verification Competitions (FVC) and NIST Fingerprint Evaluation collect as many samples as possible to get a convinced evaluation. However, based on the relationship between confidence interval and sample size, a big expansion of sample size only brings a little effect in reducing the confidence interval width when the sample size is very large. It has not been discussed till now whether it is worth achieving a narrow confidence interval by collecting a very big database. In this paper, we propose the concept of confidence elasticity which is defined by the ratio of the percentage change in confidence interval width to the percentage change in sample size to indicate the cost-effectiveness of the collecting data. Then we determine the test sample size of a deployed finger-vein biometric system according to empirical confidence elasticity. Experimental result shows that if we enlarge the sample size 150 which used in FVC2006 to 632 (about 4 times), the confidence interval width will reduce from 2.4% to 1.2% (about 1/2) based on the confidence elasticity of 0.5.