Pain sensitization and neuropathic pain-like symptoms are some of the common pain symptoms in patients with lower limbs, including hip and knee, osteoarthritis (HOA/KOA). Exercise therapy has been the first-line treatment; however, the effects differ for each patient. This prospective cohort study investigated the relationship between the effectiveness of exercise therapy and pretreatment characteristics (radiologic severity, pain sensitization, and neuropathic pain-like symptoms) of patients with HOA/KOA. We assessed the pain intensity using a numerical rating scale (NRS) before and after 12 weeks of exercise therapy in patients with HOA/KOA (n = 101). Before treatment, the Kellgren–Lawrence (K-L) grade; minimum joint space width (mJSW); pressure pain threshold (PPT) and temporal summation of pain (TSP) at the affected joint, tibia, and forearm; Central Sensitization Inventory-9; and painDETECT questionnaire (PDQ) were assessed. Cluster analysis was based on the pretreatment NRS and change in NRS with exercise therapy to identify the subgroups of pain reduction. The pretreatment characteristics of each cluster were compared. According to the results of the cluster analyses, patients in cluster 1 had severe pain that did not improve after exercise therapy, patients in cluster 2 had severe pain that improved, and those in cluster 3 had mild pain that improved. The patients in cluster 1 exhibited lower PPT at all measurement sites, higher TSP at the affected joint, and higher PDQ scores than those in other clusters. There was no difference in the K-L grade and mJSW among the clusters. The subgroup with severe pain and pain sensitization or neuropathic pain-like symptoms at pretreatment, even with mild joint deformity, may have difficulty in achieving improvement in pain after 12 weeks of exercise therapy. These findings could be useful for prognosis prediction and for planning exercise therapy and combining with other treatment.