Unpredictability is an important security property of Physically Unclonable Function (PUF) in the context of statistical attacks, where the correlation between challenge-response pairs is explicitly exploited. In the existing literature on PUFs, the Hamming Distance Test, denoted by HDT( t ), was proposed to evaluate the unpredictability of PUFs, which is a simplified case of the Propagation Criterion test PC( t ). The objective of these test schemes is to estimate the output transition probability when there are t or fewer than t bits flips, and ideally this probability value should be 0.5. In this work, we show that aforementioned two test schemes are not enough to ensure the unpredictability of a PUF design. We propose a new test, which is denoted as HDT( e , t ). This test scheme is a fine-tuned version of the previous schemes, as it considers the flipping bit pattern vector e along with parameter t . As a contribution, we provide a comprehensive discussion and analytic interpretation of HDT( t ), PC( t ), and HDT( e , t ) test schemes for Arbiter PUF (APUF), Exclusive-OR (XOR) PUF, and Lightweight Secure PUF (LSPUF). Our analysis establishes that HDT( e , t ) test is more general in comparison with HDT( t ) and PC( t ) tests. In addition, we demonstrate a few scenarios where the adversary can exploit the information obtained from the analysis of HDT( e , t ) properties of APUF, XOR PUF, and LSPUF to develop statistical attacks on them, if the ideal value of HDT( e , t ) = 0.5 is not achieved for a given PUF. We validate our theoretical observations using the simulated and Field Programmable Gate Array (FPGA) implemented APUF, XOR PUF, and LSPUF designs.