Seed oil content is an important trait of nut seeds, and it is affected by the import of carbon from photosynthetic sources. Although green leaves are the main photosynthetic organs, seed sarcotesta photosynthesis also supplies assimilates to seed development. Understanding the relationship between seed photosynthesis and seed development has theoretical and practical significance in the cultivation of Torreya grandis cv. “Merrillii.” To assess the role of seed sarcotesta photosynthesis on the seed development, anatomical and physiological traits of sarcotesta were measured during two growing seasons in the field. Compared with the attached current-year leaves, the sarcotesta had higher gross photosynthetic rate at the first stage of seed development. At the late second stage of seed development, sarcotesta showed down-regulation of PSII activity, as indicated by significant decrease in the following chlorophyll fluorescence parameters: the maximum PSII efficiency (Fv/Fm), the PSII quantum yield (ΦPSII), and the photosynthetic quenching coefficient (qP). The ribulose 1, 5—bisphosphate carboxylase (Rubisco) activity, the total chlorophyll content (Chl(a+b)) and nitrogen content in the sarcotesta were also significantly decreased during that period. Treatment with DCMU [3-(3,4-dichlorophenyl)-1,1-dimethylurea] preventing seed photosynthesis decreased the seed dry weight and the oil content by 25.4 and 25.5%, respectively. We conclude that seed photosynthesis plays an important role in the dry matter accumulation at the first growth stage. Our results also suggest that down-regulation of seed photosynthesis is a plant response to re-balance the source-sink ratio at the second growth stage. These results suggest that seed photosynthesis is important for biomass accumulation and oil synthesis of the Torreya seeds. The results will facilitate achieving higher yields and oil contents in nut trees by selection for higher seed photosynthesis cultivars.