Although rapid cognitive decline (RCD) is an important unfavorable prognostic factor, not much is known about it, especially in amyloid-negative individuals. The purpose of this study was to investigate risk factors for RCD in amyloid-negative individuals. We retrospectively enrolled 741 individuals who were either cognitively unimpaired or had early-stage cognitive ability loss and who underwent 18F-florbetaben (FBB) (n = 402) or 18F-flutemetamol (FMM) (n = 339) PET/CT. Based on visual and semiquantitative (SUV ratio [SUVR]-based) analysis, the following amyloid-negative groups were established: visual-negative FBB (n = 232), visual-negative FMM (n = 161), SUVR-negative FBB (n = 104), and SUVR-negative FMM (n = 101). Univariable and multivariable logistic regression analyses were performed for RCD using 5 SUVRs, 5 cortical thicknesses, and 5 neuropsychological domains and clinico-demographic factors. In the amyloid-negative groups, a decline in language function was commonly identified as a significant risk factor for RCD (P = 0.0044 in the visual-negative FBB group, P = 0.0487 in the visual-negative FMM group, P = 0.0031 in the SUVR-negative FBB group, and P = 0.0030 in the SUVR-negative FMM group). In addition, declines in frontal/executive function, frontal SUVR, and parietal SUVR; a longer duration of education; and mild cognitive decline in the amyloid-negative groups were also significant risk factors for RCD. Even in amyloid-negative individuals without cognitive impairment or with early-stage cognitive ability loss, those with decreased language and frontal/executive functions on neuropsychological testing are at risk of progression to RCD.