Context. A rich inventory of complex organic molecules (COMs) has been observed in high abundances in the gas phase toward Class 0 protostars. It has been suggested that these molecules are formed in ices and sublimate in the warm inner envelope close to the protostar. However, only the most abundant COM, methanol (CH3OH), had been firmly detected in ices before the era of the James Webb Space Telescope (JWST). Now, it is possible to detect the interstellar ices of other COMs and constrain their ice column densities quantitatively. Aims. We aim to determine the column densities of several oxygen-bearing COMs (O-COMs) in both gas and ice for two low-mass protostellar sources, NGC 1333 IRAS 2A (hereafter IRAS 2A) and B1-c, as case studies in our JWST Observations of Young proto-Stars (JOYS+) program. By comparing the column density ratios with respect to CH3OH between both phases measured in the same sources, we can probe the evolution of COMs from ice to gas in the early stages of star formation. Methods. The column densities of COMs in gas and ice were derived by fitting the spectra observed by the Atacama Large Millimeter/submillimeter Array (ALMA) and the JWST/Mid-InfraRed Instrument-Medium Resolution Spectroscopy (MIRI-MRS), respectively. The gas-phase emission lines were fit using local thermal equilibrium models, and the ice absorption bands were fit by matching the infrared spectra measured in laboratories. The column density ratios of four O-COMs (CH3CHO, C2H5OH, CH3OCH3, and CH3OCHO) with respect to CH3OH were compared between ice and gas in IRAS 2A and B1-c. Results. We were able to fit the fingerprint range of COM ices between 6.8 and 8.8 μm in the JWST/MIRI-MRS spectra of B1-c using similar components to the ones recently used for NGC 1333 IRAS 2A. We claim detection of CH4, OCN−, HCOO−, HCOOH, CH3CHO, C2H5OH, CH3OCH3, CH3OCHO, and CH3COCH3 in B1-c, and upper limits have been estimated for SO2, CH3COOH, and CH3CN. The total abundance of O-COM ices is constrained to be 15% with respect to H2O ice, 80% of which is dominated by CH3OH. The comparison of O-COM ratios with respect to CH3OH between ice and gas shows two different cases. On the one hand, the column density ratios of CH3OCHO and CH3OCH3 match well between the two phases, which may be attributed to a direct inheritance from ice to gas or strong chemical links with CH3OH. On the other hand, the ice ratios of CH3CHO and C2H5OH with respect to CH3OH are higher than the gas ratios by 1–2 orders of magnitude. This difference can be explained by gas-phase reprocessing following sublimation, or different spatial distributions of COMs in the envelope, which is an observational effect resulting from ALMA and JWST tracing different components in a protostellar system. Conclusions. The firm detection of COM ices other than CH3OH is reported in another well-studied low-mass protostar, B1-c, following the recent detection in NGC 1333 IRAS 2A. The column density ratios of four O-COMs with respect to CH3OH show both similarities and differences between gas and ice. Although the straightforward explanations would be the direct inheritance from ice to gas and the gas-phase reprocessing, respectively, other possibilities such as different spatial distributions of molecules cannot be excluded.