Measuring the Effectiveness of Privacy Policies for Voice Assistant Applications

Abstract

Voice Assistants (VA) such as Amazon Alexa and Google Assistant are quickly and seamlessly integrating into people’s daily lives. The increased reliance on VA services raises privacy concerns such as the leakage of private conversations and sensitive information. Privacy policies play an important role in addressing users’ privacy concerns and informing them about the data collection, storage, and sharing practices. VA platforms (both Amazon Alexa and Google Assistant) allow third-party developers to build new voice-apps and publish them to app stores. Voice-app developers are required to provide privacy policies to disclose their apps’ data practices. However, little is known whether these privacy policies are informative and trustworthy or not on emerging VA platforms. On the other hand, many users invoke voice-apps through voice and thus there exists a usability challenge for users to access these privacy policies. In this paper, we conduct the first large-scale data analytics to systematically measure the effectiveness of privacy policies provided by voice-app developers on two mainstream VA platforms. We seek to understand the quality and usability issues of privacy policies provided by developers in the current app stores. We analyzed 64,720 Amazon Alexa skills and 16,002 Google Assistant actions. Our work also includes a user study to understand users’ perspectives on privacy policies of voice-apps. Our findings reveal a worrisome reality of privacy policies in two mainstream voice-app stores. For the 17,952 skills and 9,955 actions that have privacy policies, there are many voice-apps with incorrect privacy policy URLs or broken links. We found that 1,755 Alexa skills and 192 Google actions provide a broken privacy policy URL. Amazon Alexa has more than 56% of skills with duplicate privacy policy URLs. While the Google Assistant platform has 9.0% of actions with duplicate privacy policy URLs. There are also skills/actions with inconsistency between the privacy policy and description. 6,047 Google actions do not have a privacy policy although they are required to provide one. Google and Amazon even have official voice-apps violating their own requirements regarding the privacy policy. We have reported our findings to both Amazon Alexa and Google Assistant teams, and received acknowledgments from both vendors.