Olibanum, a golden oleo-gum resin from species in the Boswellia genus (Burseraceae family), is a famous traditional herbal medicine widely used around the world. Previous phytochemical studies mainly focused on the non-polar fractions of olibanum. In this study, nine novel diterpenoids, boswellianols A-I (1-9), and three known compounds were isolated from the polar methanolic fraction of the oleo-gum resin of Boswellia carterii. Their structures were determined through comprehensive spectroscopic analysis as well as experimental and calculated electronic circular dichroism (ECD) data comparison. Compound 1 is a novel diterpenoid possessing an undescribed prenylmaaliane-type skeleton with a 6/6/3 tricyclic system. Compounds 2-4 were unusual prenylaromadendrane-type diterpenoids, and compounds 5-9 were new highly oxidized cembrane-type diterpenoids. Compounds 1 and 5 showed significant transforming growth factor β (TGF-β) inhibitory activity via inhibiting the TGF-β-induced phosphorylation of Smad3 and the expression of fibronectin and N-cadherin (the biomarker of the epithelial-mesenchymal transition) in a dose-dependent manner in LX-2 human hepatic stellate cells, indicating that compounds 1 and 5 should be potential anti-fibrosis agents. These findings give a new insight into the chemical constituents of the polar fraction of olibanum and their inhibitory activities on the TGF-β/Smad signaling pathway.