Given a differential graded (dg) symmetric Frobenius algebra $A$ we construct an unbounded complex $\mathcal{D}^{}(A,A)$, called the Tate–Hochschild complex, which arises as a totalization of a double complex having Hochschild chains as negative columns and Hochschild cochains as non-negative columns. We prove that the complex $\mathcal{D}^(A,A)$ computes the singular Hochschild cohomology of $A$. We construct a cyclic (or Calabi–Yau) $A$-infinity algebra structure, which extends the classical Hochschild cup and cap products, and an $L$-infinity algebra structure extending the classical Gerstenhaber bracket, on $\mathcal{D}^(A,A)$. Moreover, we prove that the cohomology algebra $H^(\mathcal{D}^\*(A,A))$ is a Batalin–Vilkovisky (BV) algebra with BV operator extending Connes' boundary operator. Finally, we show that if two dg algebras are quasi-isomorphic then their singular Hochschild cohomologies are isomorphic and we use this invariance result to relate the Tate–Hochschild complex to string topology.