This paper presents an altitude and attitude control system for a newly designed rocket-type unmanned aerial vehicle (UAV) propelled by a gimbal-based coaxial rotor system (GCRS) enabling thrust vector control (TVC). The GCRS is the only means of actuation available to control the UAV’s orientation, and the flight dynamics identify the primary control difficulty as the highly nonlinear and tightly coupled control distribution problem. To address this, the study presents detailed derivations of attitude flight dynamics and a control strategy to track the desired attitude trajectory. First, a Proportional-Integral-Derivative (PID) control algorithm is developed based on the formulation of linear matrix inequality (LMI) to ensure robust stability and performance. Second, an optimization algorithm using the Levenberg–Marquardt (LM) method is introduced to solve the nonlinear inverse mapping problem between the control law and the actual actuator outputs, addressing the nonlinear coupled control input distribution problem of the GCRS. In summary, the main contribution is the proposal of a new TVC UAV system based on GCRS. The PID control algorithm and LM algorithm were designed to solve the distribution problem of the actuation model and confirm altitude and attitude tracking missions. Finally, to validate the flight properties of the rocket-type UAV and the performance of the proposed control algorithm, several numerical simulations were conducted. The results indicate that the tightly coupled control input nonlinear inverse problem was successfully solved, and the proposed control algorithm achieved effective attitude stabilization even in the presence of disturbances.