Metastasis of primary tumors remains a challenge for early diagnosis and prevention. The cellular properties and molecular drivers of metastatically competent clones within primary tumors remain unclear. Here, we generated 10-16 single cell-derived lines from each of three colorectal cancer (CRC) tumors to identify and characterize metastatic seeds. We found that intrinsic factors conferred clones with distinct metastatic potential and cellular communication capabilities, determining organ-specific metastasis. Poorly differentiated or highly metastatic clones, rather than drug-resistant clones, exhibited poor clinical prognostic impact. Personalized genetic alterations, instead of mutation burden, determined the occurrence of metastatic potential during clonal evolution. Additionally, we developed a gene signature for capturing metastatic potential of primary CRC tumors and demonstrated a strategy for identifying metastatic drivers using isogenic clones with distinct metastatic potential in primary tumors. This study provides insight into the origin and mechanisms of metastasis and will help develop potential anti-metastatic therapeutic targets for CRC patients.