Vaccination has been implemented as a useful measure to combat the COVID-19 pandemic. However, there is a tendency for individuals to avoid vaccination due to the possibility of adverse events, making it important to investigate the relationship between COVID-19 vaccines and their adverse events. This study explored longitudinal adverse event patterns and factors that influence adverse events following the second to fourth doses of the COVID-19 vaccine through a latent class analysis. Participants were recruited from the Fukushima Prefecture and included individuals who had completed four doses of the COVID-19 mRNA vaccine. This study utilized data from questionnaire surveys and blood collection conducted between September 2021 and November 2022. In the questionnaire, factors such as sex, age, medical history, medication, type of vaccine administered, and adverse events following vaccination were recorded. Additionally, in the blood data, serological tests [IgG(S)] and cellular immune responses (T-spot) were measured. Descriptive statistics, latent class analysis, multivariable logistic regression, and multiple regression analyses were performed to identify the longitudinal adverse event patterns and influencing factors. By analyzing adverse events over time, we identified two distinct groups: those less prone to experiencing adverse events (Group 1) and those more susceptible (Group 2) to latent class analysis. A total of 1,175 participants were included after excluding those without any adverse events. The median age of the participants in Group 1 was 70 years, and in Group 2 it was 51 years. The proportion of female participants was 298 in Group 1 and 353 in Group 2. Patients in Group 2 were significantly younger (p < 0.001) and more likely to be female (p < 0.001) than those in Group 1. Furthermore, the median IgG(S) value after the fourth vaccination was 3,233 AU/mL in Group 1 and 4,059.39 AU/mL in Group 2. The median T-spot value was 15.4 in Group 1 and 28.5 in Group 2. Group 2 showed significantly higher IgG(S) and T-spot values after the fourth vaccination (p < 0.001). Our findings suggest that factors other than age, particularly sex and a history of allergies, significantly influence the likelihood of experiencing adverse events. Groups categorized by latent class analysis for longitudinal adverse events are expected to be valuable for optimizing vaccination strategies and formulating public health measures.