Despite recent advances in understanding the biological behavior of osteosarcoma (OS), OS is still the most common primary bone sarcoma that endangers the health of children and adolescents. High-temperature requirement A (HTRA) protease family plays an important regulatory role in numerous malignancies and acts as a prognostic biomarker. However, the function and underlying mechanisms of the HTRA family in OS development remain unknown. Through analyzing the GSE126209 dataset obtained from different Gene Expression Omnibus (GEO) databases, we found that HTRA3 as a member of the HTRA family was downregulated in OS tissues compared with that in normal tissues. Functional experiments indicated that HTRA3 overexpression suppressed malignant behaviors of OS cells in vitro and tumor growth in vivo. Mechanistically, we found that HTRA3 co-localized with the X-linked inhibitor of apoptosis protein (XIAP) and decreased XIAP stability. Further investigation showed that XIAP knockdown inhibited the degradation of phosphatase and tensin homolog (PTEN) and that HTRA3 caused the blockage of PTEN/phosphoinositide 3-kinase (PI3K)/AKT pathway, characterized as the reverse of cell function caused by HTRA3 overexpression after PTEN inhibitor BpV (HOpic) treatment. Detailed investigations showed that forkhead box protein 1 (FOXP1), an oncogene in OS progression, downregulated HTRA3 expression and inhibited the transcriptional activity of HTRA3, suggesting that HTRA3 was regulated negatively by FOXP1. In conclusion, our study demonstrates that HTRA3 is a repressor involved in OS development via the PTEN/PI3K/AKT pathway under the modulation of transcription factor FOXP1, and it may provide a therapeutic direction for OS patients.