The existing structural characterization methods of building materials as well as thermal conductivity calculation models can not be directly applied when considering fibrous building thermal insulation materials with intricate skeletons. In this study, rock wool is considered the research object, and the X-ray tomography method is used to characterize the pore structure and extract structural parameters. Based on the actual pore morphology of the material, the pores in the representative elementary volume (REV) are reasonably assumed to be cubic. Additionally, considering the contact thermal resistance of fibers and the randomness of pore distribution, the curved pattern of heat conduction pathway with the ever-changing diameter is reasonably simplified into a series-parallel form of the curved heat chain with the same diameter. On this basis, the fractal models of effective thermal conductivity are established in horizontal and vertical directions. Finally, a modified formula for the effective thermal conductivity calculation of fiber thermal insulation materials is proposed, taking into account the influence of fiber orientation on the material thermal conductivity. It was shown that the relative deviation between the calculated correction value and the experimental value of the effective thermal conductivity is all less than 16.13%.