Why do firms choose to locate in the informal sector? Researchers often argue that the high cost of regulation prevents informal firms from becoming formal and productive. Our results point to a more nuanced story.Using data from surveys of microenterprises in South Africa, Namibia, Botswana, Kenya, Uganda, Tanzania, and Rwanda, we find that the labor productivity of informal firms is virtually indistinguishable from that of formal firms in East Africa, but very different in Southern Africa. We provide a theoretical model to explain this result, based on the key assumption that firms may evade taxes subject to a cost (or concealment cost) that is increasing and convex in the firm’s employment size. Consequently, the productivity distributions reflect the differences in concealment costs and the opportunity cost of formality. Greater enforcement of laws and better provision of services such as finance and electricity to formally registered firms in Southern Africa means that firms are more likely to register; those that do not are likely to be operating as “survivalist” firms. But in East Africa, weak enforcement of tax payment and no significant difference in access to services between formal and informal firms means that these variables do not explain the allocation of firms across the informal-formal divide.We conclude that in countries with weak business environments, informal firms are just as likely as formal firms to increase their productivity as they grow. Thus, interventions to increase productivity and lower the cost of formality may be helpful. But in countries with strong business environments such as those in Southern Africa, owners of informal firms are likely to be better off entering the labor market as wage labor. In the latter case, investment in education or vocational training is probably more important.