This study proposes the mixed neural operator (MNO) learning framework, which further combines with the particle swarm optimization (PSO) to address challenges of solitary wave propagation over topography. The forward problem is defined as the evolution prediction of the solitary wave propagating over topography, while the inverse problem is defined as an optimization to identify the topography parameter based on the solitary wave elevation. Both the forward and inverse problems can be considered within a single framework and the dataset are provided by the classical Korteweg–de Vries (KdV) equation. The MNO framework is shown to simulate the evolution of solitary waves over topography, accurately capturing the wave elevation under different topographical conditions. By comparing with different neural operators, it is found that the U-shape neural operator is the most suitable for the KdV equation simulation. The coefficient of determination for the inverse problem based on the combination of MNO and PSO can reach 0.992, showing great potential of the approach in topography recognition. Finally, the proposed learning framework is preliminary applied to the prediction of the tsunami runup onto a complex beach, and a good agreement is also achieved between the direct simulation and the learning framework prediction.