This paper presents a novel procedure to design a compact and miniaturized unit cell in the substrate integrated plasmonic waveguide (SIPW) structures. The slot of the unit cell is automatically obtained in this method without needing any prior knowledge of its shape. The method is based on topology optimization through pixelization of the slot surface in the unit cell and uses a combination of binary particle swarm optimization (BPSO) and commercial electromagnetic (EM) software. With this method, first, the shapes of the two unit cells are engineered to arbitrarily reduce the high cut-off frequency. Then, two low-pass filters consisting of the proposed unit cells are designed and simulated. To further verify the proposed procedure, one of the filters is fabricated, whose measurement results are in good agreement with the simulation results. This filter provides a pass band of 5.6 to 6.3 GHz and has dimensions of 0.82λg×0.22λg. The proposed method has the potential to minimize microwave and terahertz devices.