To estimate the mixing matrix in underdetermined mixing systems, we propose a novel method by exploiting the sparsity of sources. We utilize the pairwise relationships among all of the mixture representations to detect the single source points in the time-frequency (TF) domain, i.e., the positions where only one source contributed dominantly. The mixture representations at these single source points are then clustered to estimate the underlying mixing matrix. Since the pairwise relationships among all mixtures are considered in the TF domain, the proposed method can achieve an accurate mixing matrix estimation and be robust in noisy cases. Experimental results indicate that our method is effective in mixing matrix estimation and outperforms five peer methods.