Users are usually required to share several types of data, including their personal data, as different providers strive to offer high-quality services that are often tailored to end-users’ preferences. However, when it comes to personalizing services, there are several challenges for meeting user’s needs and preferences. For content personalization and delivery of services to end users, services typically create user profiles. When user profiles are created, user data is collected and organized to meet the personalization requirements of the services. In this paper, we provide an overview of current research activities that focus on user profiling and ways to protect user data privacy. The paper presents different types of data that services collect from users on examples of commonly used Internet services. It proposes data categorization as a prerequisite for controlled data sharing between users and Internet services. Furthermore, it discusses how data generalization can be used for anonymization purposes on examples of the proposed data categories. Finally, it gives an overview of the privacy framework being developed and gives guidelines for future work focusing on data generalization methods in order to reduce user privacy risks.