Domain adaptation provides the possibility of utilizing the knowledge gained from an auxiliary domain to accomplish the task in another related domain. In this paper, a common feature representation of these domains is learned through a new proposed idea to reduce inter-domain differences more precisely which results in higher accuracy for unsupervised domain adaptation. To decrease these divergences, the proposed method finds a subspace that reduces data distribution discrepancy between domains. To achieve this goal, minimization of the divergence of the class conditional probability distributions is considered. By considering class conditional distributions, more discriminative information of data is preserved compared to applying marginal distributions. Also, an explicit parametric distribution of the source and target domains is considered to reduce the discrepancy between the two domains data which results in higher accuracy compared to the other relevant domain adaptation methods. Experimental studies on benchmark image classification tasks confirmed our assumptions and indicated the significant improvement given by the proposed method compared to the other state-of-the-art methods.