Let $E\to B$ be a smooth vector bundle of rank $n$, and let $P \in I^p(GL(n,\mathbb{R}))$ be a $GL(n,\mathbb{R})$-invariant polynomial of degree $p$ compatible with a universal integral characteristic class $ u \in H^{2p}(BGL(n,\mathbb{R}),\mathbb{Z})$. Cheeger-Simons theory associates a rigid invariant in $H^{2p-1}(B,\mathbb{R}/\mathbb{Z})$ to any flat connection on this bundle. Generalizing this result, Jaya Iyer (\textit{Letters in Mathematical Physics}, 2016, 106 (1) pp. 131-146) constructed maps $H_r(\mathcal{D}(E)) \to H^{2p-r-1}(B,\mathbb{R}/\mathbb{Z})$ for $p>r+1$ where $\mathcal{D}(E)$ is the simplicial set of relatively flat connections, thereby associating invariants to families of flat connections. In this article we construct such maps for the cases $p<r$ and $p>r+1$ using fiber integration of differential characters. We find that for $p>r+1$ case, the invariants constructed here coincide with those obtained by Jaya Iyer, and that in the $p<r$ case the invariants are trivial. We further compare our construction with other results in the literature.