Orbital drilling is an alternative hole-making machining process that combines drilling and milling, usually performed with specific devices and industrial robots to make holes in aircraft composite structures. In order to eliminate the adverse effects on the diameter precision of the tool deflection in the orbital drilling process of CFRP/Ti stacks, a novel method that fuses cutting force model prediction and feed-axis disturbance observation is proposed to estimate the radial cutting force. The tool deflection caused by the radial cutting force is calculated based on the equivalent diameter cantilever beam model. Then the tool path is corrected to compensate for the tool deflection and prevent borehole diameter deviation by adjusting the tool eccentricity in real-time. The orbital drilling experiments demonstrate that the geometrical error of the hole was reduced by 50%, and the manufacturing precision requirements of IT9 level can be satisfied after one single shot even the cutting force was increased to twice the initial value caused by tool wear, the proposed method effectively improved the hole-making precision of CFRP/Ti stacks.