When an individual is exposed to Mycobacterium tuberculosis (Mtb) three outcomes are possible: bacterial clearance, active disease, or latent infection. It is generally believed that most individuals exposed to Mtb become latently infected and carry the mycobacteria for life. How Mtb is maintained during this latent infection remains largely unknown. During an Mtb infection in mice, there is a phase of rapid increase in bacterial numbers in the murine lungs within the first 3 weeks, and then bacterial numbers either stabilize or increase slowly over the period of many months. It has been debated whether the relatively constant numbers of bacteria in the chronic infection result from latent (dormant, quiescent), non-replicating bacteria, or whether the observed Mtb cell numbers are due to balance between rapid replication and death. A recent study of mice, infected with a Mtb strain carrying an unstable plasmid, showed that during the chronic phase, Mtb was replicating at significant rates. Using experimental data from this study and mathematical modeling we investigated the limits of the rates of bacterial replication, death, and quiescence during Mtb infection of mice. First, we found that to explain the data the rates of bacterial replication and death could not be constant and had to decrease with time since infection unless there were large changes in plasmid segregation probability over time. While a decrease in the rate of Mtb replication with time since infection was expected due to depletion of host's resources, a decrease in the Mtb death rate was counterintuitive since Mtb-specific immune response, appearing in the lungs 3–4 weeks after infection, should increase removal of bacteria. Interestingly, we found no significant correlation between estimated rates of Mtb replication and death suggesting the decline in these rates was driven by independent mechanisms. Second, we found that the data could not be explained by assuming that bacteria do not die, suggesting that some removal of bacteria from lungs of these mice had to occur even though the total bacterial counts in these mice always increased over time. Third and finally, we showed that to explain the data the majority of bacterial cells (at least ~60%) must be replicating in the chronic phase of infection further challenging widespread belief of nonreplicating Mtb in latency. Our predictions were robust to some changes in the structure of the model, for example, when the loss of plasmid-bearing cells was mainly due to high fitness cost of the plasmid. Further studies should determine if more mechanistic models for Mtb dynamics are also able to accurately explain these data.