In this study the coordination chemistry of three ligands, C5H4NOCONRR′ (where, R, R′ = iC3H7 (L1); R, R′ = iC4H9 (L2); and R = H, R′ = tC4H9 (L3) composed of N-oxide and carboxamide groups have been explored with uranyl nitrate and some selected lanthanide (La, Sm, and Eu) nitrates. All the synthesized ligands as well as their complexes (1–12) of type UO2(NO3)2L (where, L = L1, L2, and L3 for 1, 2, and 3 respectively) and Ln(NO3)3(H2O)L2 (where, Ln = La, L = L1 for 4, L = L2 for 5, and L = L3 for 6; Ln = Sm, L = L1 for 7, L = L2 for 8, and L = L3 for 9; Ln = Eu, L = L1 for 10, L = L2 for 11, and L = L3 for 12) have been characterized by elemental analysis, spectroscopic analyses such as FTIR, 1H NMR, and electrospray ionization mass spectrometry (ESI-MS). Solid-state structural analysis of L1, 3, and 10 is carried out by X-ray crystallographic technique. The CO and NO groups of L1 are placed almost mutually perpendicular to each other in the crystal structure of L1. The X-ray data show that in [UO2(NO3)2{C5H4NOCONH(tC4H9)}] (3), the ligand acts as a bidentate chelating ligand and is bonded through both the N-oxo and amide oxygen atoms, whereas, in [Eu(NO3)3(H2O){C5H4NOCON(iC3H7)2}2] (10), the ligands show monodentate behavior and are bonded only through N-oxo oxygen atoms. Quantum mechanical calculation at DFT level corroborates the possibility of various bonding modes of these ligands towards uranium and europium nitrate with the preference of bonding as observed in the synthesized complexes. Solvent extraction studies using N,N-dioctyl N-oxo pyridine 2-carboxamide ligand (L4) in n-dodecane with UO22+, Pu4+, Am3+ and Eu3+ indicate the trend Pu4+ ˃ UO22+ > Am3+ > Eu3+ at acidity range from 0.01 M to 6 M HNO3. The ligands show good radiation stability at gamma dose up to 500 kGy and chemical stability at 3 M HNO3 for up to 200 h without much affecting the metal ion extraction. Theoretical calculations show the possibility of presence of different metal species in the organic phase, other than the products obtained from dichloromethane during the solvent extraction of UO22+ and Eu3+ in water/dodecane biphasic media. Energy decomposition analysis supports the higher extraction coefficient of UO22+ than Eu3+ with an evidence of higher orbital interaction of the ligands with UO22+.