Negative afterimages are perceptual phenomena that occur after physical stimuli disappear from sight. Their origin is linked to transient post-stimulus responses of visual neurons. The receptive fields (RFs) of these subcortical ON- and OFF-center neurons exhibit antagonistic interactions between central and surrounding visual space, resulting in selectivity for stimulus polarity and size. These two features are closely intertwined, yet their relationship to negative afterimage perception remains unknown. Here we tested whether size differentially affects the perception of bright and dark negative afterimages in humans of both sexes, and how this correlates with neural mechanisms in subcortical ON and OFF cells. Psychophysically, we found a size-dependent asymmetry whereby dark disks produce stronger and longer-lasting negative afterimages than bright disks of equal contrast at sizes >0.8°. Neurophysiological recordings from retinal and relay cells in female cat dorsal lateral geniculate nucleus showed that subcortical ON cells exhibited stronger sustained post-stimulus responses to dark disks, than OFF cells to bright disks, at sizes >1°. These sizes agree with the emergence of center-surround antagonism, revealing stronger suppression to opposite-polarity stimuli for OFF versus ON cells, particularly in dorsal lateral geniculate nucleus. Using a network-based retino-geniculate model, we confirmed stronger antagonism and temporal transience for OFF-cell post-stimulus rebound responses. A V1 population model demonstrated that both strength and duration asymmetries can be propagated to downstream cortical areas. Our results demonstrate how size-dependent antagonism impacts both the neuronal post-stimulus response and the resulting afterimage percepts, thereby supporting the idea of perceptual RFs reflecting the underlying neuronal RF organization of single cells.SIGNIFICANCE STATEMENT Visual illusions occur when sensory inputs and perceptual outcomes do not match, and provide a valuable tool to understand transformations from neural to perceptual responses. A classic example are negative afterimages that remain visible after a stimulus is removed from view. Such perceptions are linked to responses in early visual neurons, yet the details remain poorly understood. Combining human psychophysics, neurophysiological recordings in cats and retino-thalamo-cortical computational modeling, our study reveals how stimulus size and the receptive-field structure of subcortical ON and OFF cells contributes to the parallel asymmetries between neural and perceptual responses to bright versus dark afterimages. Thus, this work provides a deeper link from the underlying neural mechanisms to the resultant perceptual outcomes.