Journal clubs are a common element of many residency programs. However, there is little data showing that they improve residents' ability to interpret and use the scientific literature. The authors developed a structured journal club curriculum intended to improve scientific literacy skills among residents and increase acceptability and participation in meaningful discussion of high-quality articles among both residents and facilitators. This study examined the impact of the curriculum on objective skills, subjective skills, and engagement. Psychiatry residents from three academic medical centers completed 10-12 virtual journal clubs during academic year 2022-2023. At the start and end of that year, residents completed a knowledge assessment and a self-rating of scientific literacy skills; at the end of the year, residents self-rated participation in, preparation for, and attitudes towards journal club sessions and assessment of scientific literacy skills. Twenty-seven residents completed self-reports at both time points. Performance on the objective skills assessment did not improve over the course of the year. Subjectively, residents reported increased understanding of common statistical techniques, ability to determine if techniques used in an article are appropriate, and the extent to which information learned from articles informs clinical practice. In the sample of 62 resident engagement surveys at follow-up, the journal club curriculum was well-received. Use of a structured journal club curriculum was associated with improvements in self-rated scientific literacy skills among residents and had high acceptability. More work is needed to understand the impact of teaching methodology on journal club success.