Graptolite-rich shale is the main layer of shale gas resources in the southern marine sedimentary basin. Recently, shale gas resources were discovered in the Ordovician marine graptolite-rich strata in the Ordos Basin. The graptolite shale in the study area is different from the marine graptolite shale in the Yangtze plate in southern China, and further exploration is needed. This paper presents core samples of the graptolite-rich shale of the Pingliang Formation in the southwest Ordos Basin as research objects. The graptolite genus and graptolite shale characteristics were studied using core observation, electron microscopy, scanning electron microscopy, and geochemical analysis. We determined the role of the sedimentary environment and thermal maturation of graptolite shale in hydrocarbon formation and explored the possibility of hydrocarbon generation. Many graptolite epidermises provide buried organic matter. The quiet sea and low-energy marine environment create favorable conditions for preserving organic matter. The tectonic process resulted in the evolution stage in the oil generation window. Different types of pores formed the spaces of hydrocarbons. Therefore, the shale of the Pingliang Formation has shale oil exploration potential, which complements the shale gas in the northwestern margin of the basin, and provides new venues for shale oil and gas exploration in northern China.