In his influential paper on quantum modular forms, Zagier developed a conjectural framework describing the behavior of certain quantum knot invariants under the action of the modular group on their arguments. More precisely, when J_{K,0} denotes the colored Jones polynomial of a knot K , Zagier’s modularity conjecture describes the asymptotics of the quotient J_{K,0}(e^{2 \pi i \gamma(x)}) / J_{K,0}(e^{2 \pi i x}) as x \to \infty along rationals with bounded denominators, where \gamma \in \mathrm{SL}(2,\mathbb{Z}) . This problem is most accessible for the figure-eight knot 4_{1} , where the colored Jones polynomial has a simple explicit expression in terms of the q -Pochhammer symbol. Zagier also conjectured that the function h(x) = \log (J_{4_1,0}(e^{2 \pi i x}) / J_{4_1,0}(e^{2 \pi i /x})) can be extended to a function on \mathbb{R} which is continuous at irrationals. In the present paper, we prove Zagier’s continuity conjecture for all irrationals for which the sequence of partial quotients in the continued fraction expansion is unbounded. In particular, the continuity conjecture holds almost everywhere on the real line. We also establish a smooth approximation of h , uniform over all rationals, in accordance with the modularity conjecture. As an application, we find the limit distribution (after a suitable centering and rescaling) of \log J_{4_1,0}(e^{2 \pi i x}) , when x ranges over all reduced rationals in (0,1) with denominator at most N , as N \to \infty , thereby confirming a conjecture of Bettin and Drappeau.