The role of senescence of cancer-associated fibroblasts (CAFs) in the development of cancer is controversial. In this study, we investigated whether cellular senescence of CAFs, represented by CAV1 expression, affects tumor progression in pancreatic cancers (PC). Because CAV1 plays a major role in cellular senescence, we used CAV1 expression to monitor cellular senescence. A total of 157 consecutive patients with PC who underwent curative resection were enrolled in the study. Patients were divided into two groups according to CAV1 expression in CAFs by immunohistochemistry. We investigated the relationship between the CAV1 expression in CAFs and the patients' clinicopathological characteristics, including survival. We also established ten CAFs cell lines using PC clinical samples and chose one of them to knock down CAV1 expression. Finally, we cultured a PC cell line (MIAPaCa-2) in CAF-conditioned medium (CM). Regarding patients' clinicopathological characteristics, the serum levels of carbohydrate antigen 19-9 and the rate of advanced tumor stage (pT2, 3, and 4) were significantly higher in the high-CAV1 group. The high-CAV1 group had significantly worse outcomes in both overall and disease-free survival (p < 0.01). Additionally, in co-culture assays using CAFs-CM and MIAPaCa-2 cells, we found that knockdown of CAV1 in CAFs negatively affected the invasion of PC cells. In PC, CAV1 expression in CAFs is associated with patients' poor prognosis and the downregulation of CAV1 in CAFs reduces the invasiveness of PC cells. Therefore, CAV1 of CAFs might be a new target for the treatment of PC.