Breast cancer has become the most common cancer worldwide, and early screening improves the patient’s survival rate significantly. Although pathology with needle-based biopsy is the gold standard for breast cancer diagnosis, it is invasive, painful, and expensive. Meanwhile it makes patients suffer from misplacement of the needle, resulting in misdiagnosis and further assessment. Ultrasound imaging is non-invasive and real-time, however, benign and malignant tumors are hard to differentiate in grayscale B-mode images. We hypothesis that breast tumors exhibit characteristic properties, which generates distinctive spectral patterns not only in scattering, but also during propagation. In this paper, we propose a breast tumor classification method that evaluates the spectral pattern of the tissues both inside the tumor and beneath it. First, quantitative ultrasonic parameters of these spectral patterns were calculated as the representation of the corresponding tissues. Second, parameters were classified by the K-Nearest Neighbor machine learning model. This method was verified with an open access dataset as a reference, and applied to our own dataset to evaluate the potential for tumors assessment. With both datasets, the proposed method demonstrates accurate classification of the tumors, which potentially makes it unnecessary for certain patients to take the biopsy, reducing the rate of the painful and expensive procedure.