Vibrational energy relaxation of molecules adsorbed at single crystal metal surfaces is measured by populating excited vibrational levels with picosecond infrared pump pulses and probing the population by infrared-visible sum frequency generation. Energy relaxation of the v=1 levels of the C-O stretching mode in CO/Cu(100) and of the symmetric C-H stretching mode of CH 3S/Ag(111) are discussed.