Single crystals of Bi$_{2} $Rh$ _{3}$S$ _{2}$ and Bi$_{2} $Rh$ _{3.5}$S$ _{2}$ were synthesized by solution growth and the crystal structures, thermodynamic and transport properties of both compounds were studied. In the case of Bi$_{2} $Rh$ _{3}$S$ _{2}$, a structural first-order transition at around 165 K is identified by single crystal diffraction experiments, with clear signatures visible in resistivity, magnetization and specific heat data. No superconducting transition for Bi$_{2} $Rh$ _{3}$S$ _{2}$ was observed down to 0.5 K. In contrast, no structural phase transition at high temperature was observed for Bi$_{2} $Rh$ _{3.5}$S$ _{2}$, however bulk superconductivity with a critical temperature, $T_{c}\approx$ 1.7 K was observed. The Sommerfeld coefficient $\gamma$\ and the Debye temperature($\varTheta_{\textrm{D}}$) were found to be 9.4 mJ mol$^{-1}$ K$^{-2}$ and 209 K respectively for Bi$_{2} $Rh$ _{3}$S$ _{2}$, and 22 mJ mol$^{-1}$ K$^{-2}$ and 196 K respectively for Bi$_{2} $Rh$ _{3.5}$S$ _{2}$. Study of the specific heat in the superconducting state of Bi$_{2} $Rh$ _{3.5}$S$ _{2}$ suggests that Bi$_{2} $Rh$ _{3.5}$S$ _{2}$ is a weakly coupled, BCS superconductor.