Introduction: COVID-19 can be associated with a variety of longer-lasting impairments that can have a significant impact on patients’ quality of life (QoL). While this is well described in the literature for limitations in lung capacity or permanent headaches, there is little research on the impact of olfactory dysfunction in the context of COVID-19 on patients’ QoL. Methods: In 65 patients with a history of COVID-19, the present olfactory ability was assessed using the Sniffin’ Sticks test. In addition, olfactory QoL was assessed by the Questionnaire of Olfactory Disorders. Self-assessment was performed with visual analogue scales. The data were compared with the results obtained in healthy individuals and in patients with hyposmia due to other viral infections. Results: The QoL of COVID-19 patients was significantly lower compared to the healthy control group. Even recovered subjects whose olfaction had already returned to the normal range still had a reduced QoL. The severity of the olfactory impairment correlated with the reduction in QoL. However, the olfactory QoL of COVID-19 patients was not worse than that of patients’ olfactory loss due to other viral infections. Patients with parosmia had reduced QoL and rated their situation worse than patients without parosmia. Conclusion: QoL appears to be impaired in patients with long-lasting COVID-19 olfactory disorders several months after overcoming acute symptoms, even if olfaction has normalized. However, the impairment is not more pronounced than in patients with other postviral olfactory disorders of the same duration.