Abstract To improve the performance of traditional solar power generation systems, a new solar organic Rankine cycle system that can generate electricity and heat is proposed. The system incorporates the separation-flash process, regenerator, and ejector to enhance its efficiency. The optimization of the working fluid, pinch point temperature difference, evaporator outlet dryness, flash dryness, and entrainment ratio is conducted to achieve optimal performance. Aiming at maximum exergy efficiency and minimum levelized energy cost, the operating parameters are further optimized using a multi-objective optimization algorithm. R245fa is the optimal working fluid for the system, offering maximum net output power and thermal efficiency. The optimal performance can be achieved when the pinch point temperature difference is 1 K, evaporator outlet dryness is 0.6, flash dryness is 0.44, and entrainment ratio is 0.29. Moreover, the photovoltaic subsystem can further increase the net output power and thermal efficiency by 15.52% and 15.45%, achieving a maximum net output power and thermal efficiency of 33.95 kW and 10.61%. Additionally, when the solar hot water temperature is 100 °C, pinch point temperature difference is 1.8 K, evaporator outlet dryness is 0.6, flash dryness is 0.65, and entrainment ratio is 0.16, the system can achieve the optimal state of both performance and economy, exhibiting an optimal exergy efficiency and levelized energy cost of 64.1% and 0.294 $/kWh, respectively. Finally, the payback period of system is 3.43 years, indicating the potential for significant economic benefits.