It is shown by a prototype experiment that the deformation of a support structure leads to relative deflection of the optical axes of a multiwaveband imaging system and then inconsistency of the fields of view of the subsystems. To solve this issue, a topology optimization method with the objective of minimizing the deflection angles of the optical axes is proposed. The method consists of the establishment of a deflection angle equation, the construction of an objective function, and the achievement of optimization using commercial software. Then, a new optimization structure is extracted from the topology optimization model. The comparative analysis between the original structure and the optimization structure shows that the deflection angles of the optical axes after topology optimization decrease greatly, which proves the effectiveness of the proposed method.