Abstract The radial outflow liquid turbine expander (LTEROF) draws increasing attention for enhancing the efficiency of the liquid CO2 energy storage (LCES) system. However, the detrimental cavitation deteriorates the flow behavior, which demands an in-depth study of the flow physics and then effective attenuation. This study aims to effectively mitigate thermosensitive fluid cavitation and reduce energy dissipation. First, a preliminary expander design methodology taking into account the large specific volume variation of working fluid is implemented. Next, the entropy production analysis method (EPAM) is proposed to characterize energy dissipation and cavitation. Furthermore, the improved cavitation and turbulence models are validated through simulating Hord's liquid hydrogen hydrofoil. To suppress the cavitation and energy dissipation, the optimization design method based on the particle swarm optimization (PSO) algorithm together with the Kriging-based adaptive surrogate model is developed. Among them, the nonuniform relational B-splines and free form deformation (NURBS-FFD) method is applied to flexibly deform the profiles of nozzle and rotor, and a novel objective function incorporating vapor volume fraction and local entropy production rate (LEPR) is constructed to capture the cavitation and energy dissipation. During optimization, the optimizer is driven by the objective function to search globally toward the cavitation-resistance and low-dissipation geometry. With the optimization, the LEPR region shrinks and the cavitation is obviously weakened, the performance significantly improves both under design condition and under off-design conditions.