This paper proposes a new position tracking algorithm by integrating extended Kalman filter (EKF) and direction-of-arrival (DOA)-based geolocation into one factor graph (FG) framework. A distributed sensor network is assumed for detecting an anonymous target, where the process and observation equations in the state space model (SSM) are unknown. Importantly, the predicted state information can be utilized not only for filtering, but also for enhancing the observation process. To be specific, by taking the prediction into account as the a priori, a new FG scheme is proposed for GEolocation, denoted by FG-GE. The benefits are two-fold, compared with the conventional geolocation scheme which does not rely on the a priori information. First of all, significant performance improvement can be observed, in terms of the root mean square error (RMSE), when severe sensing errors are suddenly encountered. Furthermore, the proposed FG-GE can achieve dramatic reduction of computational complexity. In addition, this paper also proposes the use of a predicted Cramer-Rao lower bound (P-CRLB) to dynamically estimate the observation error variance, which demonstrates more robust tracking performance than that with only fixed average variance approximation.