Literatures revealed that the cognitive and affective components are the factors affecting problem solving. In this article we identified factors considered by the students in learning mathematical problem solving. Using a descriptive phenomenological research we explored the lived experiences of forty-five (45) student’s in solving a mathematics problem. Following the Colaizzi method for data analysis, four themes emerged: emotions and self- efficacy as affective factors, and group learning activity and teacher- student relationship as social factors. Sixty items from these four themes were further explored in using an Exploratory Factor Analysis (EFA) for a new set of 200 students. These four-factor structures of the student’s experiences in mathematics problem solving explained 66% of the variance in the pattern of relationships among the items. All four-factor structures had high reliabilities (all at or above Cronbach’s α > .904). The study exemplified that teacher- student interaction relationship during learning activities, which is a social factor, provides the highest correlated factor that influences the mathematical performance of the students.