CO2 sequestration through the carbonation of cementitious materials has drawn the interest of researchers as a strategy for mitigating climate change. Accordingly, methods for monitoring the carbonation state of cement were developed to determine the capacity of cementitious materials to capture CO2. However, current monitoring approaches require destructive sampling, limiting their practical application in the field. Therefore, we developed embedded-type optical sensors for non-destructive in situ monitoring of carbonation of cementitious materials. The sensor consists of a dialysis membrane, a pH indicator, optical fibers, and an acrylic reactor, and this can be embedded to cementitious materials. Carbonation monitoring experiments and numerical modeling were conducted to investigate the performance of developed sensor for monitoring the carbonation of cementitious materials. A significant linear relationship was found between the color change of the pH indicator, pH of the detection solution, and the concentration of CaCO3 that has precipitated as a result of CO2 sequestration by cement samples (R2 = 0.90, P < 0.05). These results show that our developed sensor efficiently monitored the carbonation of cementitious materials. Our developed sensor may serve as an important tool in monitoring the carbonation of built structures employing cementitious materials.