The dynamics of quantum systems coupled to baths are typically studied using the Nakajima-Zwanzig memory kernel (K\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${{{\\bf{{{{\\mathcal{K}}}}}}}}$$\\end{document}) or the influence functions (I), particularly when memory effects are present. Despite their significance, formal connections between the two have not been explicitly known. We establish their connections by examining the system propagator for a N-level system linearly coupled to Gaussian baths with various types of system-bath coupling. For a certain class of problems, we devised a non-perturbative, diagrammatic approach to construct K\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${{{\\bf{{{{\\mathcal{K}}}}}}}}$$\\end{document} from I for (driven) systems interacting with Gaussian baths, bypassing conventional projection-free dynamics inputs. Our work provides a way to interpret approximate path integral methods in terms of approximate memory kernels. Moreover, it offers a Hamiltonian learning procedure to extract the bath spectral density from reduced system trajectories, opening new avenues in quantum sensing and engineering. The insights we provide advance our understanding of non-Markovian dynamics and will serve as a stepping stone for future theoretical and experimental developments in this area.