• Nonlinear dynamics analysis of time-varying structure are investigated. • The equations derived by using Hamilton’s principle. • The distinct bifurcation parameters of force & torque inputs have considered. • The effects of excitation frequency, damping coefficient have studied. The analysis of motion equations of flexible-link manipulators indicates that nonlinear terms and behavioral instability exist in their outcomes. In this system, when the differential equations possess structural time-dependent parameters, there is a higher chance for system instability, especially chaos in comparison to time-invariant ones. It is a point for abnormal behavior that is unpredictable and always accompanied by damages. Therefore, the physical parameters and system inputs that cause chaotic responses should be detected and prevented as much as possible. To this aim, the dynamic equations of flexible link manipulator with revolute-prismatic joints as a case of time-variant dynamic system are obtained by Hamilton’s principle with the assumption of nonlinear strains. The discretized motion equations are solved and the results are presented in the forms of bifurcation diagrams (for variation of torque/force amplitude), Poincaré maps, phase-plane portraits, and the largest Lyapunov exponent. Finally, the obtained results are validated with the aid of the experimental setup. The results indicate that although the system response in low torque/force amplitudes is subharmonic (amplitude of 0.01) and quasi-periodic (amplitudes of 0.02–0.03), it becomes quasi-periodic/chaotic with a mild slope with respect to time in high amplitude values. Moreover, since the behavior of modal generalized coordinate is different from the rigid ones, the quasi-periodic and chaotic vibrations do not hurt the joints.