In this study, a hydraulic servo system for mold oscillating mechanism was investigated. In order to check the efficiency and the stability of system, its geometrical designs, which change the natural characteristic of system, including a spool volume, area of piston, and so on were analyzed by a non-linear model. The model was composed of the equation of orifice, the continuity equation, and the force equilibrium including a variable effective bulk modulus belonging to operating oil, gas, and cylinder. All simulations were conducted by the MATLAB SIMULINK, and simulated as specific conditions corresponding real operation conditions in the steel industry. According to the analysis, the relation between a controller, servo design, and system performance was investigated with physical means. And, the fact that not compared to other design variables, the entrance shape on spool was dominant to determine the stability and the performance of system, was found. Furthermore, a fault detection method, and optimization problem including this work were discussed.