Conventionally, passive sonar classification has been conducted to predict a ship size for given sample. Recent studies have shown high accuracy, far exceeding the performance of early baselines. However, the assumption that manifolds of passive sonar dataset are formed according to the ship size hasn't been studied deeply. In this paper, we investigate the assumption and attempt to analyze the dataset to observe the manifolds. We consider various attributes of passive sonar signals including, which could form the manifold. We employ unsupervised and supervised learning methods with various extracted features. In our analysis, it is difficult to find an attributes dominantly forming the manifolds because passive signals significantly changed by various facts including ocean environment and moving status. Our study wil provide a deeper understanding of the dataset, leading to a realistic and robust passive sonar classifier.