Plant fibers with length limited by natural growing usually require to be twisted into continuous yarns for reinforcing the host matrix. This makes such twisted yarn reinforced plant fiber composites (TYPCs) are hierarchical with two-scale reinforcements: the plant fibers compacted in the yarn and the twisted yarns at the mesoscale. In the present paper, to uncover the internal relationship between such hierarchical structure and the mechanical properties of TYPCs, a bottom-up theoretical model is established. Synergistic effects of the filler content, filler aggregation degree, and porosity at each level are underlined. Accordingly, a three-step homogenization method based on Mori-Tanaka method is applied. To validate the effectiveness and accuracy of this model, comparisons with existing prediction models and available experimental data are made. Furthermore, detailed parameter analysis indicates that the performance of TYPCs depends not only on the filler content, but also on the filler distribution. The enhancement of TYPCs’ properties with increasing filler content may be offset by the weakening effect caused by filler aggregations. Moreover, the helical structure of yarns should not be ignored. The increase of the yarn twisting angle will improve the transverse and shear moduli of TYPCs while sacrificing the modulus in the longitudinal direction to a certain extent.