Individual differences in electrical impedance spectroscopy (EIS) parameters is a technical bottleneck in fruit quality detection. To achieve non-destructive fruit quality testing and minimize the impact of individual differences, this paper proposes a frequency selection + support vector machine strategy based on the frequency response characteristics of biological tissues and the path of current flow. This study found obvious differences in the EIS parameters of different samples. The measurement stability was affected by the electrode type and frequency. The destructive needle electrode was much less affected by individual differences than the non-destructive patch electrode. The effect of low frequencies was greater than that of high frequencies. The heterogeneity of the fruit peel and pulp and the contact area between the electrode and peel were the main reasons for individual differences. When detecting banana maturity, the highest precision of the models was 98.9% by proposed methods; however, the generalizability of the model and its application for other fruit quality tests still require further verification and optimization. The results promote the practical application of EIS in the field of fruit quality assessment and provide new insights for the quality evaluation of other agricultural products.