With the rapid development of the chip and metasurface industries, high-speed defect detection of wafer during semiconductor batch manufacturing has become increasingly important for yield control. Here, we propose a high-speed, macro-scale wafer surface defect detection method with enhanced image edges, via optical spatial filtering in serial time-encoded imaging. By using blazed gratings to spatially project different wavelength components of pulsed light into a one-dimensional line, together with dispersive Fourier time stretching technology, a high-speed serial time-encoded imaging system based on a single pixel detector is constructed for testing the wafer surface defects, and optical spatial filtering is introduced to improve the imaging quality. Experiments show high-pass spatial filtering effectively restrains environmental interference and enhances image edge detection. Small target detection is achieved without a complex image algorithm. Our solution extends optical spatial filtering to serial time-encoded imaging, not only permitting wafer macro scale defect detection with edge enhancement, but also having potential significance for imaging lidar, imaging spectrometers, ghost imaging technology, and more.