Rare earth elements and yttrium (REY) in deep-sea sediments have emerged as potential strategic resources. In order to investigate the provenance, sedimentary environment, and mechanism of REY enrichment of pelagic sediments in Indian Ocean, we analyzed the grain size, bulk major and trace element concentrations and mineralogy of clay fractions (<2 μm) of sediments in the Wharton Basin, Indian Ocean. Based on the characteristics of biostratigraphic, we speculated that the sediments were deposited during the period from 11 Ma to 2 Ma. Sediments in the upper part (0–100 cm) of the sequence were mainly derived from aeolian material in Australia, whereas sediments in the lower part (100–258 cm) were mainly sourced from volcanic and hydrothermal materials which associated with submarine volcanic activity. The oxygen-rich environment below the calcite compensation depth (CCD) is crucial for the formation of REY-rich sediments. Additionally, there is a strong correlation between ΣREY, clay-grained sediments, Al2O3, and Fe2O3, suggesting that clay fractions contribute significantly to the enrichment of REY in the deep-sea sediments.