Low seed fertility seriously limits the survival and adaption of rare plant species. Here, we identified a seed-specific gene, DiZF-C3H1, from the dove tree and verified its function. Overexpression of DiZF-C3H1 caused retarded root development, delayed anthesis, abnormal floral organs, and deformed siliques in transgenic Arabidopsis lines. No offspring were obtained in transgenic Arabidopsis lines due to serious seed abortion. Therefore, we performed further verification in tobacco. Similarly, overexpression of DiZF-C3H1 retarded root development and reduced berry size and seed yield in transgenic tobacco lines. Moreover, although transgenic tobacco offspring were obtained, the viability of transgenic seeds was reduced and their germination was delayed. In addition, faded flowers were observed in transgenic tobacco lines. Taken together, DiZF-C3H1 was verified to play a negative role in root growth, floral organ development, and especially seed development in Arabidopsis and tobacco. This appears to be a deleterious gene for these model plants with high seed fertility. However, this function might be of special significance for Davidia, whose seed dormancy period is extremely long; DiZF-C3H1 might play a critical role in the distinctive reproduction strategy adopted by this rare and endangered species.