A two-tier Cooperative Agent-based Traffic Signal control (CATS) is proposed to minimize total delay for independent-learning signal-controlled junctions. For vehicular networks with traffic congestion, a link traffic model is presented to estimate time-varying signal delay under stochastic travel demand. To capture essential features of signal-controlled junctions, an agent-based value function approximator is proposed. For the 1 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">st</sup> tier, common cycle time and offsets are explored to achieve collaboration among control agents. For the 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nd</sup> tier, green splits are exploited to ensure scalability over entire vehicular networks. A stochastic bi-level program is presented to minimize total delays. Numerical experiments are performed at a real-data vehicular network under stochastic flow. Comparisons are made with state-of-the-art traffic signal controls. As reported, the proposed CATS outperforms other alternatives in all cases.