Infrared (IR) transparent polymer materials prepared by inverse vulcanization, as a promising candidate to replace inorganic materials, are new materials for constructing key devices in IR optics. However, it is difficult to achieve a balance between infrared optical and thermal properties in polymers due to the intrinsic infrared absorption of organic materials. Herein, our strategy is to construct a high boiling point symmetrical molecular norbornadiene derivative cross‐linking agent (DMMD) which can be inverse vulcanized with molten sulfur, and obtain Poly(S‐r‐DMMD) with different sulfur content by controlling the feed ratio of sulfur. With the rigid core and low IR activity in DMMD, the prepared polymers exhibit tunable thermal properties (Tg: 98.3‐119.8 °C) and high IR transmittance (medium‐wave infrared region (MWIR): 42.9‐52.6 %; long‐wave infrared region (LWIR): 1.5‐5.29 %). In addition, Poly(S‐r‐DMMD) can be used to prepare large‐size free‐standing Fresnel lenses for IR imaging by simple hot‐pressing, which provides flexibility in the design and production of IR fine lenses. This study provides a novel strategy for balancing the thermal and optical properties of IR transparent polymer materials, while providing relevant references for balancing the IR optical and thermal properties of polymer materials.