Aims. Accurate distances to evolved stars with high mass loss rates are needed for studies of many of their fundamental properties. However, as these stars are heavily obscured and variable, optical and infrared astrometry is unable to provide enough accuracy. Methods. Astrometry using masers in the circumstellar envelopes can be used to overcome this problem. We have observed the OH masers of a number of Asymptotic Giant Branch (AGB) stars for approximately 1 year with the Very Long Baseline Array (VLBA). We have used the technique of phase referencing with in-beam calibrators to test the improvements this technique can provide to Very Long Baseline Interferometry (VLBI) OH maser astrometric observations. Results. We have significantly improved the parallax and proper motion measurements of the Mira variable stars U Her, S CrB and RR Aql. Conclusions. It is shown that both in-beam phase-referencing and a decrease in solar activity during the observations significantly improves the accuracy of the astrometric observations. The improved distances to S CrB ($418^{+21}_{-18}$ pc) and RR Aql ($633^{+214}_{-128}$ pc) are fully consistent with published P – L relations, but the distance to U Her ($266^{+32}_{-28}$ pc) is significantly smaller. We conclude that for sources that are bright and have a nearby in-beam calibrator, VLBI OH maser astrometry can be used to determine distances to OH masing stars of up to ~2 kpc.