Sheltered from deleterious mutations, genes with overlapping or partially redundant functions may be important sources of novel gene functions. While most partially redundant genes originated in gene duplications, it is much less clear why genes with overlapping functions have been retained, in some cases for hundreds of millions of years. A case in point is the many partially redundant genes in vertebrates, the result of ancient gene duplications in primitive chordates. Their persistence and ubiquity become surprising when it is considered that duplicate and original genes often diversify very rapidly, especially if the action of natural selection is involved. Are overlapping gene functions perhaps maintained because of their protective role against otherwise deleterious mutations? There are two principal objections against this hypothesis, which are the main subject of this article. First, because overlapping gene functions are maintained in populations by a slow process of "second order" selection, population sizes need to be very high for this process to be effective. It is shown that even in small populations, pleiotropic mutations that affect more than one of a gene's functions simultaneously can slow the mutational decay of functional overlap after a gene duplication by orders of magnitude. Furthermore, brief and transient increases in population size may be sufficient to maintain functional overlap. The second objection regards the fact that most naturally occurring mutations may have much weaker fitness effects than the rather drastic "knock-out" mutations that lead to detection of partially redundant functions. Given weak fitness effects of most mutations, is selection for the buffering effect of functional overlap strong enough to compensate for the diversifying force exerted by mutations? It is shown that the extent of functional overlap maintained in a population is not only independent of the mutation rate, but also independent of the average fitness effects of mutation. These results are discussed with respect to experimental evidence on redundant genes in organismal development.