DMMP: A distillation-based multi-task multi-tower learning model for personalized recommendation

Abstract

Recommendation systems involve the matching and ranking stages. The click-through rate (CTR) and the conversion rate (CVR) predictions are two fundamental modules in recommendation systems. Most candidate generation matching models leverage a two-tower architecture to model the CTR prediction task. However, items with low-quality but attractive titles, i.e., click baits, may be recommended to the user, which worsens the user’s experience. Therefore, both click and conversion tasks should be modeled during the matching stage to improve user engagement. An intuitive way is to model these two tasks with a three-tower matching model. However, its efficiency and effectiveness are limited. By inheriting the merits of knowledge distillation and multi-task learning, we propose a distillation-based multi-task multi-tower model (DMMP) for personalized recommendation. Specifically, the MTL-based teacher network builds the task-shared and task-specific expert networks, and employs a customized multi-gate control network to merge these expert networks adaptively within the task embedding layer. Two auxiliary pCTR and pCTCVR tasks are incorporated to model CVR directly across the whole space. By sharing the feature representation parameters with the CTR modeling, the CVR modeling can be trained with richer samples. Three-tower-based candidate generation student network incorporates a user preference gating network to learn the task scores so that personalized candidates can be produced. Furthermore, we design an adaptive weighting strategy for total loss to eventually adjust the importance and relevance of different networks. Extensive experiments on public and industrial datasets validate the effectiveness of DMMP.