Let Γ=(G,σ) be a signed graph, where G is the underlying simple graph and σ:E(G)→{+,−} is the sign function on the edges of G. In this paper we consider the spectral characterization problem extended to the adjacency matrix and Laplacian matrix of signed graphs. After giving some basic results, we study the spectral determination of signed lollipop graphs, and we show that any signed lollipop graph is determined by the spectrum of its Laplacian matrix.