Marbling characteristics are important traits for the genetic improvement of pork quality. Accurate marbling segmentation is the prerequisite for the quantification of these traits. However, the marbling targets are small and thin with dissimilar sizes and shapes and scattered in pork, complicating the segmentation task. Here, we proposed a deep learning-based pipeline, a shallow context encoder network (Marbling-Net) with the usage of patch-based training strategy and image up-sampling to accurately segment marbling regions from images of pork longissimus dorsi (LD) collected by smartphones. A total of 173 images of pork LD were acquired from different pigs and released as a pixel-wise annotation marbling dataset, the pork marbling dataset 2023 (PMD2023). The proposed pipeline achieved an IoU of 76.8%, a precision of 87.8%, a recall of 86.0%, and an F1-score of 86.9% on PMD2023, outperforming the state-of-art counterparts. The marbling ratios in 100 images of pork LD are highly correlated with marbling scores and intramuscular fat content measured by the spectrometer method (R2 = 0.884 and 0.733, respectively), demonstrating the reliability of our method. The trained model could be deployed in mobile platforms to accurately quantify pork marbling characteristics, benefiting the pork quality breeding and meat industry.