This research aimed to extract flaxseed mucilage (FM) and investigate its rheological properties (static and dynamic tests) compared with animal oil. In the next stage, the D-optimal design was applied to investigate the effect of FM (0–60%) and FF (0–30%) replacements with animal oil and cake flour, respectively on the sponge cake's physicochemical, textural, and sensory properties. According to the flow behavior test, FM showed Newtonian behavior while animal oil had pseudoplastic behavior. The results of the dynamic test showed with an increase in frequency, the loss modulus (G״) and storage modulus (Gʹ) of samples increased. However, Gʹ was higher than G״ in all samples. By replacement of FM and FF, the moisture content, water activity, antioxidant capacity, crumb hardness, and cohesiveness of the samples increased while springiness, crust hardness, and specific volume decreased (P < 0.05). Lightness of samples with replacement of FF decreased (P < 0.05). The overall acceptance score was enhanced with an increase in FM substitution while it was decreased with the replacement of FF (P < 0.05). The amounts of fat, calories, and peroxide values were decreased in sponge cake with the incorporation of FF and FM (P < 0.05). In general, the substitution of FM (60%) and FF (28%) in the cake formulation as an optimized sample to make new products with low-calorie content is possible without significant decreases in product quality.