Bichiral plasmonic nanoparticles exhibited intriguing geometry-dependent circular dichroism (CD) reversal; however, the crucial factor that dominates the plasmonic CD is still unclear. Combined with CD spectroscopy and theoretical multipole analysis, we demonstrate that plasmonic CD originates from the excitation of electric quadrupolar plasmons. Moreover, a comparative study of two distinct quadrupolar modes reveals the correlation between the sign of the CD and the local geometric handedness at the plasmonic hotspots, thereby establishing a structure-property relationship in bichiral nanoparticles. The reverse CD is attributed to the opposite directions of the wavelength shift of the two plasmon modes upon changing the particle geometry. By finely tuning the size of bichiral nanoparticles, we can further reveal that the dependence of plasmonic CD on the electric quadrupolar plasmons. Our work sheds light on the physical origin of plasmonic CD and provides important guidelines for the design of chiral plasmonic nanoparticles toward chirality-dependent applications.