For a locally compact Abelian group G, the algebra of Rajchman measures, denoted by $${M}_{0}(G)$$ , is the set of all bounded regular Borel measures on G Fourier transform of which vanish at infinity. In this paper, we investigate the spectral structure of the algebra of Rajchman measures, and illustrate aspects of the residual analytic structure of its maximal ideal space. In particular, we show that $${M}_{0}(G)$$ has a nonzero continuous point derivation, whenever G is a nondiscrete locally compact Abelian group. We then give the definition of the Rajchman algebra for a general (not necessarily Abelian) locally compact group, and prove that for a noncompact connected SIN group, the Rajchman algebra admits a nonzero continuous point derivation. Moreover, we discuss the analytic behavior of the spectrum of $${M}_{0}(G)$$ . Namely, we show that for every nondiscrete metrizable locally compact Abelian group G, the maximal ideal space of $${M}_{0}(G)$$ contains analytic disks.