A financial agent-based time series model is developed and investigated by the stochastic contact systems. Multicolor contact system, as one of statistical physics systems, is applied to model a random stock price process for investigating the fluctuation dynamics of financial market. The interaction and dispersal of different types of investment attitudes in a financial market is imitated by viruses spreading in a multicolor contact system, and we suppose that the investment attitudes of market participants contribute to the volatilities of financial time series. We introduce a volatility duration analysis to detect the duration and intensity relationship of time series for both SSECI and the financial model. Furthermore, the empirical research is also presented to study the nonlinear behaviors of returns for the actual data and the simulation data.