Organic matter (OM) maturity is closely related to organic pores in shales. Quantitative characterization of organic and inorganic pores in shale is crucial for rock-physics modeling and reservoir porosity and permeability evaluation. Focused ion beam-scanning electron microscopy (FIB-SEM) can capture high-precision three-dimensional (3D) images and directly describe the types, shapes, and spatial distribution of pores in shale gas reservoirs. However, due to the high scanning cost, wide 3D view field, and complex microstructure of FIB-SEM, more efficient segmentation for the FIB-SEM images is required. For this purpose, a multiphase segmentation workflow in conjunction with a U-net is developed to segment pores from the matrix and distinguish organic pores from inorganic pores simultaneously in the entire 3D image stack. The workflow is repeated for FIB-SEM data sets of 17 organic-rich shales with various characteristics. The analysis focuses on improving the efficiency and relevance of the workflow, that is, quantifying the minimum number of training slices while ensuring accuracy and further combining the fractal dimension (FD) and lacunarity to study a simple and objective method of selection. Meanwhile, the computational efficiency, accuracy, and robustness to noise of the 2D U-net model are discussed. The intersection over the union of automatic segmentation can amount to 80%–95% in all data sets with manual labels as ground truth. In addition, calculated by the FIB-SEM multiphase segmentation, the organic porosity is used to quantitatively evaluate the OM decomposition level. Deep-learning-based segmentation shows great potential for characterizing shale pore structures and quantifying OM maturity.