Decreasing oil resources creates the need to search for raw materials in the biosphere, which can be converted into polyols suitable for obtaining polyurethane foams (PUF). One such low-cost and reproducible biopolymer is cellulose. There are not many examples of cellulose-derived polyols due to the sluggish reactivity of cellulose itself. Recently, cellulose and its hydroxypropyl derivatives were applied as source materials to obtain polyols, further converted into biodegradable rigid polyurethane foams (PUFs). Those PUFs were flammable. Here, we describe our efforts to modify such PUFs in order to decrease their flammability. We obtained an ester from diethylene glycol and phosphoric(III) acid and used it as a reactive flame retardant in the synthesis of polyol-containing hydroxypropyl derivative of cellulose. The cellulose-based polyol was characterized by infrared spectra (IR) and proton nuclear magnetic resonance (1H-NMR) methods. Its properties, such as density, viscosity, surface tension, and hydroxyl numbers, were determined. Melamine was also added to the foamed composition as an additive flame retardant, obtaining PUFs, which were characterized by apparent density, water uptake, dimension stability, heat conductance, compressive strength, and heat resistance at 150 and 175 °C. Obtained rigid PUFs were tested for flammability by determining oxygen index, horizontal flammability test, and calorimetric analysis. Obtained rigid PUFs showed improved flammability resistance in comparison with non-modified PUFs and classic PUFs.