Website privacy policies outline how a website handles personal information. Many such policies are written vaguely, making it difficult for users to comprehend what data is collected and how itâs used, posing a threat to privacy and security. The empirical study in this paper aims to address this issue by applying transformer models to automatically detect the level of vagueness in privacy policy texts. We calculate the vagueness scores based on annotations obtained through arithmetic mean, geometric mean, and harmonic mean of the individual annotations and discretize the policy text into four distinct vagueness categories. Our detailed numerical study shows that transformer models perform well for the vagueness classification tasks and they can be effective in enhancing the clarity and transparency of privacy policies. Our experiments also highlight the importance of reliable annotations and show that the transformer model performance can be enhanced with more reliable annotations with lower standard deviation and entropy values.
Read full abstract