Programmable photonic circuits (PPCs) have garnered substantial interest for their potential in facilitating deep learning accelerations and universal quantum computations. Although photonic computation using PPCs offers ultrafast operation, energy-efficient matrix calculations, and room-temperature quantum states, its poor scalability hinders integration. This challenge arises from the temporally one-shot operation of propagating light in conventional PPCs, resulting in a light-speed increase in device footprints. Here we propose the concept of programmable photonic time circuits, utilizing time-cycle-based computations analogous to gate cycling in the von Neumann architecture and quantum computation. Our building block is a reconfigurable SU(2) time gate, consisting of two resonators with tunable resonances, and coupled via time-coded dual-channel gauge fields. We demonstrate universal U(N) operations with high fidelity using an assembly of the SU(2) time gates, substantially improving scalability from O(N^{2}) to O(N) in terms of both the footprint and the number of gates. This result paves the way for PPC implementation in very large-scale integration.