Controlling the permittivity of dielectric composites is critical for numerous applications dealing with matter/electromagnetic radiation interaction. In this study, we have prepared polymer composites, based on a silicone elastomer matrix and Tuball carbon nanotubes (CNT) via a simple preparation procedure. The as-prepared composites demonstrated record-high dielectric permittivity both in the low-frequency range (102–107 Hz) and in the X-band (8.2–12.4 GHz), significantly exceeding the literature data for such types of composite materials at similar CNT content. Thus, with the 2 wt% filler loading, the permittivity values reach 360 at 106 Hz and >26 in the entire X-band. In similar literature, even the use of conductive polymer hosts and various highly conductive additives had not resulted in such high permittivity values. We attribute this phenomenon to specific structural features of the used Tuball nanotubes, namely their length and ability to form in the polymer matrix percolating network in the form of neuron-shaped clusters. The low cost and large production volumes of Tuball nanotubes, as well as the ease of the composite preparation procedure open the doors for production of cost-efficient, low weight and flexible composites with superior high permittivity.