AbstractThe influence of hexagonal boron nitride (hBN) nanosheet surface modification on epoxy nanocomposites' mechanical and thermal properties were investigated. An innovative approach was developed for synthesizing hBN nanosheets (hBNNs) via optimized ultrasonic‐assisted liquid‐phase exfoliation (UALPE), followed by functionalization with 3‐aminopropyltriethoxysilane (APTS) and polyethylene glycol (PEG600) for nanocomposite preparation. Compared to unmodified epoxy, nanocomposites containing 0.3 wt% of pristine hBNNs exhibited significant enhancements in flexural strength (125.28 MPa) and tensile strength (53.35 MPa), with respective increases of 40.57% and 43.23%. PEG‐hBNNs improved flexural and tensile strengths to 136.83 and 58.57 MPa, respectively. The most substantial enhancements were observed with APTS‐hBNNs, yielding flexural and tensile strengths of 174.13 and 63.53 MPa, reflecting increases of 95.37% and 70.50%, attributed to better interfacial interactions and a more effective crack deflection mechanism. Thermal stability analyses revealed superior performance of modified hBNNs epoxy nanocomposites, increasing 8.2°C and 6.3°C at 50 wt% degradation as compared to the unmodified system, owing to enhanced physical barrier properties. Structural and morphological analyses validated the successful exfoliation and modification of hBNNs. Predictive models utilizing machine and deep learning (DL) techniques, particularly DL with the adaptive moment estimation (ADAM) optimizer, effectively forecasted mechanical properties, achieving high R2 values.HIGHLIGHTS 0.3 wt% modified hBNNs epoxy nanocomposite showed the highest mechanical strength Improved thermal stability of epoxy nanocomposite by modification of hBNNs Improved crack deflection mechanism in the presence of surface‐modified hBNNs Deep learning with ADAM optimizer showed high R2 and low prediction loss