A reliable and valid method for the automatic in situ measurement of hearing thresholds is a prerequisite for the feasibility of a self-fitting hearing aid, whether such a device becomes an automated component of an audiological management program or is fitted by the user independently of a clinician. Issues that must be addressed before implementation of the procedure into a self-fitting hearing aid include the role of real-ear-to-dial difference correction factors in ensuring accurate results and the ability of potential users to successfully self-direct the procedure. The purpose of this study was to evaluate the reliability and validity of an automatic audiometry algorithm that is fully implemented in a wearable hearing aid, to determine to what extent reliability and validity are affected when the procedure is self-directed by the user, and to investigate contributors to a successful outcome. Design was a two-phase correlational study. A total of 60 adults with mild to moderately severe hearing loss participated in both studies: 20 in Study 1 and 40 in Study 2. Twenty-seven participants in Study 2 attended with a partner. Participants in both phases were selected for inclusion if their thresholds were within the output limitations of the test device. In both phases, participants performed automatic audiometry through a receiver-in-canal, behind-the-ear hearing aid coupled to an open dome. In Study 1, the experimenter directed the task. In Study 2, participants followed a set of written, illustrated instructions to perform automatic audiometry independently of the experimenter, with optional assistance from a lay partner. Standardized measures of hearing aid self-efficacy, locus of control, cognitive function, health literacy, and manual dexterity were administered. Statistical analysis examined the repeatability of automatic audiometry; the match between automatically and manually measured thresholds; and contributors to successful, independent completion of the automatic audiometry procedure. When the procedure was directed by an audiologist, automatic audiometry yielded reliable and valid thresholds. Reliability and validity were negatively affected when the procedure was self-directed by the user, but the results were still clinically acceptable: test-retest correspondence was 10 dB or lower in 97% of cases, and 91% of automatic thresholds were within 10 dB of their manual counterparts. However, only 58% of participants were able to achieve a complete audiogram in both ears. Cognitive function significantly influenced accurate and independent performance of the automatic audiometry procedure; accuracy was further affected by locus of control and level of education. Several characteristics of the automatic audiometry algorithm played an additional role in the outcome. Average transducer- and coupling-specific correction factors are sufficient for a self-directed in situ audiometry procedure to yield clinically reliable and valid hearing thresholds. Before implementation in a self-fitting hearing aid, however, the algorithm and test instructions should be refined in an effort to increase the proportion of users who are able to achieve complete audiometric results. Further evaluation of the procedure, particularly among populations likely to form the primary audience of a self-fitting hearing aid, should be undertaken.