Prediction of creep properties is crucial but still very challenging for the heat-resistant alloys, for which data-driven materials genome technologies provide a promising approach. To this end, we have explored the application of deep learning to quantitatively predict the creep rupture time of Fe–Cr–Ni heat-resistant alloys using textual (chemical composition and creep testing conditions) and visual (as-cast microstructure) data. The multi-layer perceptron (MLP) and the convolutional neural network (CNN) are adopted to analyze the textual and visual data, respectively. Then the multi-source heterogeneous data are integrated by the fusion deep learning model, which achieves a significant enhancement in prediction accuracy compared with the single MLP and CNN models. The results demonstrate that the fusion model is efficient in leveraging information from diverse types of data and learning the relation between various features and creep properties in a high-throughput, statistically robust, and physically meaningful manner. The developed model can be regarded as a new workflow to predict creep properties and give insights into the inverse design of alloys. Our work is equally applicable to other classes of materials and properties as well, where the accumulated data can be reconsidered to guide discoveries.