Tomato yield prediction plays an important role in agricultural production planning and management, market supply and demand balance, and agricultural risk management. To solve the problems of low accuracy and high uncertainty of tomato yield prediction methods in solar greenhouses, based on experimental data for water and fertilizer consumption by greenhouse tomatoes in different regions over many years, this paper investigated the prediction models of greenhouse tomato yields under three different soil fertility conditions (low, medium, and high). Under these three different soil fertility conditions, greenhouse tomato yields were predicted using the neural network prediction model (NN), the neural network prediction model based on particle swarm optimization (PSO–NN), the neural network prediction model based on an adaptive inertia weight particle swarm optimization algorithm (AIWPSO–NN), and the neural network prediction model based on the improved particle swarm optimization algorithm (IPSO–NN). The experimental results demonstrate that the evaluation indexes (mean square error, mean absolute error, and R2) of the IPSO–NN prediction model proposed in this paper were superior to the other three prediction models (i.e., NN prediction model, AIWPSO–NN prediction model, and IPSO–NN prediction model) under three different soil fertility conditions. Among them, compared with the NN prediction model, the MSE of the other three prediction models under high soil fertility decreased to 0.0082, 0.0041, and 0.0036; MAE decreased to 0.0759, 0.0511, and 0.0489; R2 decreased to 0.8641, 0.9323, and 0.9408. These results indicated that the IPSO–NN prediction model had a higher predictive ability for greenhouse tomato yields under three different soil fertility conditions. In view of the important role of tomato yield prediction in greenhouses, this technology may be beneficial to agricultural management and decision support.