During the subsoiling operation of clayey soil in sugarcane fields in tropical areas, there are key limitations such as high resistance caused by high adhesion and unsatisfactory subsoiling effects of machines and tools. To address these issues, this study first explored the machine's structural characteristics using a mechanism analysis method to identify key optimization parameters. Then, simulation tests were conducted based on the root-soil complex model. Single factor analysis was conducted to evaluate the effects of backsweep angle, plow spacing, and penetration angle on tillage resistance and soil disturbance, determining their optimal ranges. Finally, a multi-factor orthogonal rotation combination experiment was designed using Box-Behnken theory to optimize the subsoiler's structural parameters. The regression analysis results show that the optimized model significantly improves the reliability of simulation results. Field verification test results show that under the conditions of a sweep angle of 50°, a plow spacing of 60 cm, and a soil penetration angle of 45°, the average tillage resistance of subsoiling operations is significantly reduced to 76.2 kN, and the amount of soil disturbance is reduced to 1780cm². The optimization reduced tillage resistance and soil disturbance by 34.42% and 30.50% respectively, significantly improving the subsoiling performance. To our knowledge, this is the first work applying discrete element methods to subsoiling operations in red clay soils in sugarcane growing areas. The proposed machine structure has higher efficiency and better performance, providing an effective reference for the design and application of sugarcane field subsoiling machinery.