Recommender systems aim to filter information effectively and recommend useful sources to match users’ requirements. However, the exponential growth of information in recent social networks may cause low prediction accuracy for recommendation systems. This article proposes a unified personalized recommendation architecture referred to as PSRec, which incorporates user preference and social relationship into matrix factorization framework. Specifically, PSRec generates two collections for textual reviews and contextual information respectively, and performs preference learning for each user via the Latent Dirichlet Allocation topic model. Moreover, PSRec exploits the inner relations within the social circle for recommendation, including direct trust relationship and indirect trust relationship. Additionally, it’s certificated that PSRec can converge at a sub-linear rate via theoretical analysis. Experimental results over DoubanMovie, CiaoDVDs and Yelp demonstrate the superiority of the proposed PSRec, which can achieve significant improvements and provide much better user experience while compared with other benchmark models.