Background and Objective: Age-related macular degeneration (AMD) is a condition of the eye that affects the aged people. Optical coherence tomography (OCT) is a diagnostic tool capable of analyzing and identifying the disease affected retinal layers with high resolution. The objective of this work is to extract the retinal pigment epithelium (RPE) layer and the baseline (natural eye curvature, particular to every patient) from retinal spectral-domain OCT (SD-OCT) images. It uses them to find the height of drusen (abnormalities) in the RPE layer and classify it as AMD or normal.Methods: In the proposed work, the contrast enhancement based adaptive denoising technique is used for speckle elimination. Pixel grouping and iterative elimination based on the knowledge of typical layer intensities and positions are used to obtain the RPE layer. Using this estimate, randomization techniques are employed, followed by polynomial fitting and drusen removal to arrive at a baseline estimate. The classification is based on the drusen height obtained by taking the difference between the RPE and baseline levels. We have used a patient, wise classification approach where a patient is classified diseased if more than a threshold number of patient images have drusen of more than a certain height. Since all slices of an affected patient will not show drusen, we are justified in adopting this technique.Results: The proposed method is tested on a public data set of 2130 images/slices, which belonged to 30 patient volumes (15 AMD and 15 Normal) and achieved an overall accuracy of 96.66%, with no false positives. In comparison with existing works, the proposed method achieved higher overall accuracy and a better baseline estimate.Conclusions: The proposed work focuses on AMD/normal classification using a statistical approach. It does not require any training. The proposed method modifies the motion restoration paradigm to obtain an application-specific denoising algorithm. The existing RPE detection algorithm is modified significantly to make it robust and applicable even to images where the RPE is not very evident/there is a significant amount of perforations (drusen). The baseline estimation algorithm employs a powerful combination of randomization, iterative polynomial fitting, and pixel elimination in contrast to mere fitting techniques. The main highlight of this work is, it achieved an exact estimation of the baseline in the retinal image compared to the existing methods.