Brown rice traits are critical to both grain yield and quality. In the present study, the chromosome segment substitution lines (CSSLs) population derived from a cross between japonica Koshihikari and indica Nona Bokra was used to analyze the brown rice length (BRL), brown rice width (BRW), length–width ratio of brown rice (BLWR), brown rice thickness (BRT), brown rice perimeter (BRP), brown rice area (BRA), thousand-grain weight of brown rice (BRGW), brown rice ratio (BRR), taste value of brown rice (BTV), and water content of brown rice (BWC). Correlations analysis showed that most of the brown rice traits had significant correlations with each other, except for BRR, BTV, and BWC. A total of sixty-one QTLs for these traits were identified under three environments, which mapped to chromosomes 1, 2, 3, 5, 6, 7, 8, 10, 11, and 12, with the LOD ranging from 2.52 to 16.68 and accounting for 2.60 to 25.38% of the total phenotypic variations. Moreover, thirty pairs of epistatic interactions for BRL, BRW, BLWR, BRP, and BRA were estimated and distributed on all chromosomes except 10. These findings will provide a further understanding of the genetic basis of brown rice traits and facilitate the genetic improvement of rice yield and quality through breeding.