Potato (Solanum tuberosum L.) is the most widely grown vegetable in the world. Consumers and processors evaluate potatoes based on quality traits such as shape and skin color, making these traits important targets for breeders. Achieving and evaluating genetic gain is facilitated by precise and accurate trait measures. Historically, quality traits have been measured using visual rating scales, which are subject to human error and necessarily lump individuals with distinct characteristics into categories. Image analysis offers a method of generating quantitative measures of quality traits. In this study, we use TubAR, an image-analysis R package, to generate quantitative measures of shape and skin color traits for use in genomic prediction. We developed and compared different genomic models based on additive and additive plus non-additive relationship matrices for two aspects of skin color, redness, and lightness, and two aspects of shape, roundness, and length-to-width ratio for fresh market red and yellow potatoes grown in Minnesota between 2020 and 2022. Similarly, we used the much larger chipping potato population grown during the same time to develop a multi-trait selection index including roundness, specific gravity, and yield. Traits ranged in heritability with shape traits falling between 0.23 and 0.85, and color traits falling between 0.34 and 0.91. Genetic effects were primarily additive with color traits showing the strongest effect (0.47), while shape traits varied based on market class. Modeling non-additive effects did not significantly improve prediction models for quality traits. The combination of image analysis and genomic prediction presents a promising avenue for improving potato quality traits.