During development, oligodendrocytes contact and wrap neuronal axons with myelin. Similar to neurons and synapses, excess myelin sheaths are produced and selectively eliminated, but how elimination occurs is unknown. Microglia, the resident immune cells of the CNS, engulf surplus neurons and synapses. To determine if microglia also prune myelin sheaths, we used zebrafish to visualize and manipulate interactions between microglia, oligodendrocytes, and neurons during development. We found that microglia closely associate with oligodendrocytes and specifically phagocytose myelin sheaths. By using a combination of optical, genetic, chemogenetic, and behavioral approaches we revealed that neuronal activity bidirectionally balances microglial association with neuronal cell bodies and myelin phagocytosis in the optic tectum. Furthermore, multiple strategies to deplete microglia caused oligodendrocytes to maintain excessive and ectopic myelin. Our work reveals a neuronal activity-regulated role for microglia in modifying developmental myelin targeting by oligodendrocytes.