As a major marine pollution source, oil spills largely threaten the sustainability of the coastal environment. Polarimetric synthetic aperture radar remote sensing has become a promising approach for marine oil spill detection since it could effectively separate crude oil and biogenic look-alikes. However, on the sea surface, the signal to noise ratio of Synthetic Aperture Radar (SAR) backscatter is usually low, especially for cross-polarized channels. In practice, it is necessary to combine the refined detail of slick-sea boundary derived from the co-polarized channel and the highly accurate crude slick/look-alike classification result obtained based on the polarimetric information. In this paper, the architecture for oil spill detection based on polarimetric SAR is proposed and analyzed. The performance of different polarimetric SAR filters for oil spill classification are compared. Polarimetric SAR features are extracted and taken as the input of Staked Auto Encoder (SAE) to achieve high accurate classification between crude oil, biogenic slicks, and clean sea surface. A post-processing method is proposed to combine the classification result derived from SAE and the refined boundary derived from VV channel power image based on special density thresholding (SDT). Experiments were conducted on spaceborne fully polarimetric SAR images where both crude oil and biogenic slicks were presented on the sea surface.