Factor analysis is a multivariate analysis method widely used to extract latent factors from high dimensional data and interpret multidimensional data through dimension reduction. Additionally, factor scores of individual observations for the extracted factors are calculated and used in a subsequent regression analysis. This factor model has the characteristics of parameter indeterminacy which means parameters such as the factor loading matrix or unique variance are not uniquely determined, and factor indeterminacy which represents that latent factors are not uniquely determined even in situations where the parameters of the factor model are determined. In this paper, we examine the indeterminacy of factor scores and propose a new method of calculating factor scores that maximize the correlation coefficient between a response variable and a factor in a factor score regression model. Simulations and real data analysis were conducted to compare the existing factor scores using the regression method and the proposed factor scores utilizing the indeterminate part of factors in factor score regression setting. As a result of data analysis, it was confirmed that the estimation accuracy of regression model was higher when using the proposed factor scores.