In this study, a progressive optimization method combining machine learning and optimization method is proposed and applied to seal structure design. The method is divided into two stages to select the optimal design gradually. So as to find the best design scheme meeting the design requirements. For optimal design, numerical calculation method is commonly used, but hard to evaluate the optimal solution. In this work, a series of numerical model considering the effect of super elastic material about O-ring study the waterproof performance behavior of a rubber seal. K nearest neighbors (KNN) of machine learning algorithms applied to the simulation data to predict the appropriate bolt pretension classification. Furthermore, use TOPSIS method to optimize the groove depth of 30 N bolt pretension classification. By using the TOPSIS method to consider the stress of the rubber component, optimization analysis is conducted to find the optimal design. Results show that the dual optimization method can quickly predict the best design scheme. Through the experiment, a prototype test under the condition of IPX7 verify the method. The design scheme selected by this method meets the waterproof grade requirements. There are no water stains on the surface of the O-ring and inside the motor. This paper provides a fast optimization design method for the design of sealing structure.