State estimation in electric power grid is vulnerable to false data injection attacks, and diagnosing such kind of malicious attacks has significant impacts on ensuring reliable operations for power systems. In this paper, the false data detection problem is viewed as a matrix separation problem. By noticing the intrinsic low dimensionality of temporal measurements of power grid states as well as the sparse nature of false data injection attacks, a novel false data detection mechanism is proposed based on the separation of nominal power grid states and anomalies. Two methods, the nuclear norm minimization and low rank matrix factorization, are presented to solve this problem. It is shown that proposed methods are able to identify proper power system operation states as well as detect the malicious attacks, even under the situation that collected measurement data is incomplete. Numerical simulation results both on the synthetic and real data validate the effectiveness of the proposed mechanism.