Voice privacy using CycleGAN and time-scale modification

Abstract

Extensive use of Intelligent Personal Assistants (IPA) and biometrics in our day-to-day life asks for privacy preservation while dealing with personal data. To that effect, efforts have been made to preserve the personally identifiable characteristics from human voice using different speaker anonymization techniques. In this paper, we propose Cycle Consistent Generative Adversarial Network (CycleGAN) to modify (transform) the speaker’s gender as well as the other prosodic aspects using their Mel cepstral coefficients (MCEPs) and fundamental frequency (i.e., F0). For effective anonymization in the context of voice privacy, we propose two-level (i.e., double) anonymization, where first-level anonymization is done using CycleGAN, followed by second-level anonymization using time-scale modification. The speaker anonymization and intelligibility are measured objectively using the automatic speaker verification (ASV) and automatic speech recognition (ASR) experiments, respectively, on development and test sets of Librispeech and VCTK datasets. For CycleGAN-based anonymization, the average % EERs (% WERs) are 40.3% (8.89%) and 40.95% (9.37%) with original enrollments and anonymized trials of the development and test datasets, respectively. The average % EERs (% WERs) for double anonymization are 46.19% (9.95%) and 44.76% (10.34%) with original enrollments and anonymized trials of the development and test datasets, respectively. For the voice privacy evaluation , the performance of ASV system is much important, when the enrollments and trials both are anonymized (called as A-A case), which is also briefly discussed in this work. The average % EERs for A-A case (test set) are 24.29% and 2.81% using CycleGAN-based anonymization and double anonymization, respectively. Objective evaluation for more advanced attack model (i.e., attacker having anonymized data) is also explored in this study. The performance reflected the robustness of proposed anonymization approach towards voice privacy. The subjective tests using 101 listeners and corresponding analysis of variance (ANOVA) and Tukey–Kramer-based Ad-hoc tests are also carried out in order to quote statistical significance of our results. The subjective test show that the CycleGAN and double anonymization approaches give better naturalness, intelligibility, and speaker dissimilarity than the state-of-the-art x-vector-based baseline system.