In this paper, we propose a simple model based on the S3 flavor symmetry, in which a perturbation ansatz is suggested that the flavor symmetry is explicitly broken down via S3 [Formula: see text]Z2 [Formula: see text] in the up-quark sector and via S3 [Formula: see text]Z3 [Formula: see text] in the down-quark sector. Our model is successful in obtaining the strong hierarchy of mass matrices of both up-type quarks and down-type quarks, but unsuccessful in obtaining the realistic mixing between the third generation and the first two, as in other models based on S3 flavor symmetry proposed in the literature. Through detailed analysis, we find the reason for the unsuccesses in our model which is related to the reducibility of the three-dimensional representation of the S3 group. We conclude that any perturbations in mass matrices preserving the partial symmetry of S3 cannot help getting realistic mixing between the third generation and first two without fine-tuning.