This paper introduces a novel state-space model of nonlinear self-excited forces designed to capture amplitude dependency and unsteady effect in bluff body aeroelasticity. In the present form, this model represents a theoretical extension of the classical linear state-space model into the nonlinear regime, particularly relevant when a bluff section undergoes large amplitude oscillations. The proposed model incorporates additional state variables to approximate nonlinear convolution-based indicial functions, thereby providing a valuable tool for estimating nonlinear wind-induced instabilities, including nonlinear flutter limit cycle oscillation (LCO), vortex-induced vibration (VIV), and unsteady galloping. Furthermore, we establish the analytical foundation for identifying nonlinear parameters through the equivalent linearization of nonlinear transfer functions and the development of a numerical solving algorithm for the nonlinear governing equation. The feasibility of this model is rigorously validated through experimental results pertaining to nonlinear flutter, unsteady galloping, and VIV of typical bluff sections. Additionally, this model serves as the cornerstone for a nonlinear analytical framework in the time domain, facilitating the integration of both aerostatic and structural nonlinearities into comprehensive structural analyses.