Upper lumbar herniated intervertebral disc (HIVD), defined as L1-2 and L2-3 levels, presents with a lower incidence and more unfavorable surgical outcomes than lower lumbar levels. There are very few reports onthe appropriate surgical interventions for treating upper lumbar HIVD. This study aimed to evaluate the surgical outcome of decompression alone, when compared with spinal fusion surgery. A retrospective study involving a total of 7592 patients who underwent surgery due to HIVD in our institution was conducted. A total of 49 patients were included in this study: 33 patients who underwent decompression-only surgery and 16 patients who underwent fusion surgery. Demographic data, perioperative information, and functional outcomes were recorded. The visual analog scale (VAS) scores showed improvement in both groups postoperatively. The three-month postoperative Oswestry Disability Index score was significantly better in the fusion group. Additionally, 10 patients (76.9%) in the decompression group and 5 patients (83.3%) in the fusion group reported improvement in preoperative motor weakness. The final “satisfactory” rate was 66.7% in the decompression group and 93.8% in the fusion group (p = 0.034). The overall surgical outcomes of patients with upper lumbar HIVD were satisfactory in this study without any major complications. More reliable satisfactory rates and better functional scores at the three-month postoperative follow-up were reported in the fusion group.