One of the major global health concerns today is cardiovascular disease. A cardiovascular disease is a disease that involves the heart and blood vessels, its most common causes being smoking, alcohol, high blood pressure and blood cholesterol, etc. Added to the fact that it is a major global health issue, the fact that it is incurable once diagnosed makes the disease significantly more severe. Hence, an early diagnosis is crucial in preventing the disease and, if detected, mitigating it. Traditionally, detection for cardiovascular diseases was usually done by computerized tomography scans; however, these methods are too costly and not easily accessible for the general population. Because of these shortcomings of the traditional diagnosis method, there is a need for a more automated, accessible early diagnosis system, and that is the center of this research. In this study, we propose a novel machine learning-based approach for the detection of cardiovascular diseases using retinal images. The proposed method comprises three modules: a feature extractor, an age prediction network, and a cardiovascular disease diagnosis network. We consider the diagnosis of cardiovascular diseases as a classification task, aiming to assess the severity level of the condition based on the retinal image. The proposed method classifies retinal images into one of four categories, each corresponding to a specific level of disease severity. Through extensive experimentation, the results demonstrate that the proposed method outperforms state-of-the-art approaches. We expect that the proposed method holds the potential to become a novel machine learning-powered biomarker for diagnosing cardiovascular diseases.