A heat and mass transfer and stress model in Camellia oleifera fruit was built based on the theory of heat and mass transfer and solid mechanics. The splitting process by hot-air drying was simulated using COMSOL Multiphysical software. The simulation results showed that the moisture diffusion in the splitting process by hot-air drying of Camellia oleifera fruit gradually promoted from the center to the surface, moisture near the surface first migrated and moisture far away from the surface lagged. The internal and external moisture did not move outward simultaneously with different diffusion rates, and the temperature promoted the moisture diffusion. The moisture gradient in the peel caused the stress difference, which shrank and cracked under the action of stress, and its shape gradually changed from arch to flat. With the continuous moisture loss from the peel under the effect of temperature, the shrinkage and deformation of the peel become more severe, increasing the distance between the peels continuously; finally, the peel was removed. The peel’s arc length and width directions were contracted during the splitting process by hot-air drying of Camellia oleifera fruit. The maximum deviation between the simulated and experimental values in the arc width direction was 8.9%, and the average deviation was 5.7%. The maximum deviation of the arc length between the simulated and experimental values was 7.4%, and the average deviation was 4.2%. The results showed that the simulation was consistent with the actual splitting law of Camellia oleifera fruit, which could be used to optimize the splitting process by the hot-air drying of Camellia oleifera fruit.