In this study, we experimentally investigate the terahertz (THz) bandpass and bandstop filters based on the elliptical Babinet complementary structure, including an elliptical hole structure (EHS) and an elliptical ring structure (ERS). The experimental results indicate that the monolayer structure exhibits an obvious polarization selectivity with consistent filtering curves under TE and TM mode excitation, but the spectral line has an apparent shift of 0.1 THz. Based on Babinet's principle, the filtering mode matching is experimentally verified between bandpass and bandstop under a 90° change in polarization. Furthermore, we propose a double-layer filter by integrating EHS and ERS on both sides of the silicon substrate, and realize the bandpass and bandstop filtering simultaneously. The simulation results show that the isolation degree of bandpass and bandstop filtering has a maximum of nearly 40 dB based on the angle-tunable filtering performance, and the Q-value of the filter is further improved than that of the single-layer structure. The filter designed based on the Babinet principle has the advantages of simple structure, easy processing, and sharper filtering effect, and shows great applications in THz communication, imaging, and sensing systems.