Normalization of count data is an essential step of in the analysis of RNA-sequencing data. Its statistical analysis has been mostly addressed in the context of differential expression analysis, that is in the univariate setting. However, relationships between genes and samples are better explored and quantified using multivariate exploratory data analysis tools like Principal Component Analysis (PCA). In this study we investigate how normalization impacts PCA models and interpretation, considering twelve different widely used normalization methods that were applied on simulated and experimental data. Correlation patterns in the normalized data were explored using both summary statistics and Covariance Simultaneous Component Analysis. The impact of normalization on the PCA solutions was assessed by exploring the model complexity, the quality of sample clustering in the low-dimensional PCA space and gene ranking in the model fit to normalized data. PCA models upon normalization were interpreted in the context gene enrichment pathway analysis. We found that although PCA score plots are often similar independently form the normalization used, biological interpretation of the models can depend heavily on the normalization method applied.