Marine pollution is becoming ubiquitous in the environment. Observations of pollution on beaches, in the coastal ocean, and in organisms in the Antarctic are becoming distressingly common. Increasing human activity, growing tourism, and an expanding krill fishing industry along the West Antarctic Peninsula all represent potential sources of plastic pollution and other debris (collectively referred to as debris) to the region. However, the sources of these pollutants from point (pollutants released from discrete sources) versus non-point (pollutants from a large area rather than a specific source) sources are poorly understood. We used buoyant simulated particles released in a high-resolution physical ocean model to quantify pollutant loads throughout the region. We considered non-point sources of debris from the Antarctic Circumpolar Current, Bellingshausen Sea, Weddell Sea, and point source pollution from human activities including tourism, research, and fishing. We also determined possible origins for observed debris based on data from the Southern Ocean Observing System and Palmer Long-Term Ecological Research program. Our results indicate that point source pollution released in the coastal Antarctic is more likely to serve as a source for observed debris than non-point sources, and that the dominant source of pollution is region-specific. Penguin colonies in the South Shetland and Elephant Islands had the greatest debris load from point sources whereas loads from non-point sources were greatest around the southernmost colonies. Penguin colonies at Cornwallis Island and Fort Point were exposed to the highest theoretical debris loads. While these results do not include physical processes such as windage and Stokes Drift that are known to impact debris distributions and transport in the coastal ocean, these results provide critical insights to building an effective stratified sampling and monitoring effort to better understand debris distributions, concentrations, and origins throughout the West Antarctic Peninsula.