Efficient thermoelectric materials require a rare and contraindicated combination of materials properties: large electrical conductivity, large Seebeck coefficient, and low thermal conductivity. One strategy to achieve the first two properties is via low-energy electronic bands containing both flat and dispersive parts in different regions of crystal momentum space, known as a pudding-mold band structure. Here, we illustrate that ${\mathrm{BaPdS}}_{2}$ successfully achieves the pudding-mold band structure for the valence band, contributing to a large $p$-type thermoelectric power factor, due to its anisotropic crystal structure containing zigzag chains of edge-sharing square planar ${\mathrm{PdS}}_{4}$ units; large power factor is achieved for $n$-type doping as well due to band convergence. In addition, ${\mathrm{BaPdS}}_{2}$ exhibits ultralow lattice thermal conductivity, and thus also achieves the third property, due to extremely soft and anharmonic interactions in its transverse acoustic phonon branch. We predict a remarkably large thermoelectric figure of merit, with peak values between 2 and 3 for two of the three crystallographic directions, suggesting ${\mathrm{BaPdS}}_{2}$ warrants experimental investigation.