Assessing risk due to small sample size in probability of detection analysis using tolerance intervals

Abstract

Small sample size (e.g. 6-30) poses risk in results of probability of detection (POD) analysis using tolerance intervals. This method is also called as the limited sample or LS POD. The analysis is performed either during NDE procedure qualification or for assessment of reliability of an NDE procedure. The risk is primarily due to sampling error. Smaller samples are not likely to be random to the population or representative of the population. The small samples are likely to be biased. Biased samples have smaller standard deviation compared to the population. POD analysis with small biased sample can lead to overestimation of POD. Many sampling schemes are available in statistics to mitigate sampling risk. Primary objective of POD analysis is to determine a decision threshold from signal response measurements of a sample such that it is less than or equal to population decision threshold for 90% POD. Sampling error implies that this NDE reliability condition is violated. One of sampling types is called a representative sample. Representative samples reduce variance in POD estimates but also reduce magnitude of the error. Sampling sensitivity analysis for some sampling types is performed here using repetitive random sampling or Monte Carlo method. Six sampling types are considered for comparison. Some of the sampling types are similar to drawing a representative sample. LS POD model assumes random sampling. Therefore, random sampling is used as a basis for comparison with each sampling type. The sampling types used in the analysis are, A. Nominal and worst-case sampling, B. Worst-case sampling, C. Nominal case sampling, D. Random sampling, E. Random target, and sub-target sampling. F. Nominal target and sub-target sampling. Results of Monte Carlo simulation indicate that type F sampling can mitigate sampling risk and is also more practical to implement. Type A sampling may also mitigate the sampling risk, but it may be less practical to implement.