Terahertz time-domain spectroscopy (THz-TDS) is an emerging optical technique that has potential applications in the characterization of (bio)materials. However, the complicated extraction of optical parameters from multi-layered and optically thin samples is a barrier towards its acceptance by applied scientists. Therefore, the aim of this work is to provide a straightforward approach for the extraction of the THz absorption coefficient and index of refraction profiles of aqueous thin films in a window-sample-window configuration, which is ubiquitous in many laboratories (i.e., sample in a cuvette). A numerical approach-based methodology that accounts for multiple layers, Fabry–Pérot effect, and sample thickness is elaborated which involves an optical interference model based on a tri-layer structure and a simple thickness estimation technique. This method was validated on water samples where a good agreement was found with the THz optical parameters of water reported in the literature, while the use of a commercial software resulted in erroneous optical parameters estimates when used without due regard to its limitations. A case study was then performed to demonstrate the ability of the proposed method to characterize agarose hydrogels with varying degree of sulfation. It was demonstrated that THz-TDS can provide insight into the hydration state of the agarose hydrogels, including the relative number of the hydrogen bonds between the hydroxyl moieties of water and the polysaccharide network which is perturbed by the presence of sulfate. The trend in the index of refraction profiles suggested microstructural differences between the agarose hydrogels, which were confirmed by visualizing the agarose network morphology using cryo-SEM imaging.