We have studied two approaches to predict quark and lepton mixing patterns from the same flavor symmetry group in combination with CP symmetry. The first approach is based on the residual symmetry $Z_2$ in the charged lepton sector and $Z_2\times CP$ in the neutrino sector. All lepton mixing angles and CP violation phases depend on three real parameters $\theta_{l}$, $\delta_{l}$ and $\theta_{\nu}$. This approach is extended to the quark sector, the up and down quark mass matrices are assumed to be invariant under a $Z_2$ subgroup and $Z_2\times CP$. The necessary and sufficient conditions for the equivalence of two mixing patterns are derived. The second approach has an abelian subgroup and a single CP transformation as residual symmetries of the charged lepton and neutrino sectors respectively. The lepton mixing would be determined up to a real orthogonal matrix multiplied from the right hand side. Analogously we assume that a single CP transformation is preserved by the down (or up) quark mass matrix, and the residual symmetry of the up (or down) quark sector is be an abelian subgroup. As an example, we analyze the possible mixing patterns which can be obtained from the breaking of $\Delta(6n^2)$ and CP symmetries. We find $\Delta(294)$ combined with CP is the smallest flavor group which can give a good fit to the experimental data of quark and lepton mixing in both approaches.