To overcome the safety hazard of the liquid electrolytes used in traditional lithium batteries, solid electrolytes have drawn more attention because of their advantages such as non-volatility, easy processing, good mechanical properties, and stability. In this paper, sodium alginate (SA) nanofiber membranes were used as the backbone of PEO-based solid electrolytes. SA nanofiber membranes were prepared by electrospinning with assistance from PEO and cross-linked with calcium ions to construct a nanofiber network skeleton, which provided a guarantee for the stability of the subsequent electrolyte preparation process. The effects of spinning conditions and crosslinking time on the structure and performances of the nanofiber membranes were investigated. Meanwhile, the relationship between the skeleton of nanofiber membranes cross-linked with calcium ions and ion conductivity was investigated. The optimal parameters of the electrospinning process including concentration, voltage, distance, and SA content were discussed, and the fiber diameter and its distribution were analyzed. Furthermore, Fourier transform infrared (FTIR) spectrometer, thermal gravimetric analyzer analysis (TGA), X-ray diffraction (XRD), and energy dispersive spectrometer (EDS) maps were used to characterize the nanofiber membranes and electrolytes. The results showed that the thermal performance of cross-linked nanofiber membranes improved and the crystallinity of the PEO matrix decreased. The ion conductivity of the electrolytes was characterized by electrochemical impedance spectroscopy (EIS) testing, and the results showed that the assembled lithium symmetric battery had a good ion conductivity of 6.82 × 10−5 S/cm at 30 °C.