A decomposed dynamic dispatching and preventive maintenance (D-DDPM) framework is presented to overcome the computational challenges of solving large DDPM problems. Stochastic deterioration of machine health is explicitly considered. By efficient allocation of individual machines capacity under different machine health, the D-DDPM framework decomposes the global DDPM problems into single-machine DDPM problems while ensuring throughput optimality. Intelligent machine agents are then designed to solve the single-machine DDPM problems efficiently and to perform real-time decisions for individual machines. After the decomposition, the overall computational complexity grows only linearly with the number of machines and the proposed framework could be adopted in large systems. In the numerical study, the D-DDPM framework is near-optimal in small systems that have optimal DDPM solutions. In which, the computation time is reduced by 90.72% and the average cycle time is within 6% of optimal. For large systems without optimal DDPM solutions, numerical results show that the D-DDPM maintains throughput optimality and reduces average cycle time by up to 90% against other control policies.