This paper estimates the impact of parents' time investment in young children, their socioeconomic status and family structure on long-term outcomes of children. We developed and estimated a model of dynastic households in which altruistic individuals choose fertility, labor supply, and time investment in children sequentially, using data on two generations from the PSID. Parental behavior affects the education outcomes of children, their labor market earnings, and also their marriage market outcomes. The dynastic framework provides a natural way to aggregate the outcomes of children as it is measured by their valuation function. Time allocation patterns differ by race, gender, education and family structure (number of children and marital status). Both the returns to parental time investment and the costs vary across these groups. On average black individuals invest less time with their children. Our estimation results show that despite the fact that the valuation function is higher for whites and the fact that conditional on education, blacks earn less than whites, on the margin, there are no significant race differences in the rate of returns to paternal time investment, and blacks have a higher return to maternal time investment than whites. The main reason for the lower parental time investment by blacks is the differences in family structure. There is a significantly higher proportion of black single mothers than white single mothers and the opportunity costs of time for single mothers are higher than the opportunity costs of married mothers due to income sharing and transfers within married households. We also find that the returns to maternal time investment are significantly higher for boys. This implies that mothers act in a compensatory manner, favoring low ability children in the family. Since girls already have a higher likelihood of achieving a high level of education than boys, mothers seem to invest more time in boys than in girls as the number of children increases. Our findings suggest a significant quality-quantity trade-off. The level of investment per child is smaller the larger the number of children, thus, this decline in the per-child investment is driven by the time constraint and the opportunity costs of time and not by the properties of the production function technology of children. We, also find that quality-quantity trade-off for blacks is significantly larger than that of whites. This is mainly due to the higher fertility of single black female and the resulting greater time constraint they face.