Class imbalance is a case in which the proportion of training data between one class and another is not balanced, the larger data are called “major class”, conversely known as the “minor class”. It is believed that accuracy of data mining algorithms can be affected by an imbalance problem. Nowadays, researchers distinguish three main factors of class imbalance that affect the accuracy of data mining algorithm such as overlap, small disjuncts and outliers. A general solution to the problem is the modification of data level or algorithm level. To overcome imbalance problems, we propose a new algorithm called RANDSHUFF(Random Shuffle Oversampling Techniques for Qualitative Data), oversampling synthetic data generation for qualitative data type. RANDSHUFF algorithm uses the concept of neighborhood with IVDM (Interpolated Value Difference Metric) distance calculation and crossovers of the original attribute values and their neighbor’s attribute values using the random shuffle technique. Our experimental results showed that RANDSHUFF, combined with Borderline and ADASYN concepts, provides the best results against seven imbalanced public qualitative data type (best minor class Recall on hepatitis, breast cancer and German data and best F-Measure of minor class on hepatitis, abalone and German data).