A new type of organic/inorganic form-stable composite phase change material (FSCPCM) was prepared via direct impregnation of eutectic mixture of octadecane (C18) and tetracosane (C24) into gypsum. The maximum mass percentage of C18–C24 absorbed in gypsum was determined as 18% without the leakage of C18–C24 in liquid state. The Fourier Transform Infrared (FT-IR) spectroscopy and Differential Scanning Calorimetry (DSC) techniques were used to characterize the FSCPCM. The FT-IR spectra confirmed that any chemical interaction did not occur among the components of the composite during the impregnation. Besides, DSC analysis showed that the FSCPCM had reversible melting and freezing behavior and quite satisfying thermal properties for building applications. The composite maintained its chemical structure and thermal energy storage characteristics after 1000 melting/freezing cycles. In addition, the gypsum plaster with FSCPCM exhibited excellent thermal regulating performance when compared to that of only gypsum plaster under the same conditions. From all results, it can be deduced that the obtained gypsum based composite plaster is a promising material to control room temperature and to conserve energy in buildings.