The progress of transition metal complexes having anticancer activity has greater influence in cancer therapy. With the anticipation of investigating the anticancer activity and property of the transition metal complexes, Ni(II), Zn(II), and Cd(II) metal complexes of 1-hydroxy-2-naphthoic acid (1H2NA)/2-hydroxy-1-naphthoic acid (2H1NA) with aminoguanidine of ratio 1:1:2 were synthesized and characterized by spectroscopic studies like IR, UV–Vis, Thermal analysis, powder XRD, molar conductivity measurements, antimicrobial, and anticancer studies. Simple salts of 1-hydroxy-2-naphthoic acid (1H2NA)/2-hydroxy-1-naphthoic acid (2H1NA) with aminoguanidine were also prepared in different ratios viz, 1:1 and 1:2. It is characterized by spectroscopic measurements IR, UV, TG-DTA, powder XRD, molar conductivity measurements, and cytotoxic effects were also studied. Nanometal oxides of zinc complexes were prepared and surface morphological studies were investigated. The functional groups were confirmed from IR spectroscopy data. Band gap energies of all the synthesized compounds were calculated from UV–Visible spectroscopy. Various decomposition steps and the thermal stability of the compounds were determined from thermal analysis. From the molar conductivity measurements, it is confirmed that the compounds are electrolytes. SEM and TEM images confirm the rod and hexagonal shape of the respective nanometal oxides. The complexes and nanometal oxides show significant activity against antimicrobials like Staphylococcus aureus, Escherichia coli, and Aspergillus fumigates. The results of present research show that, the synthesized ligands and metal complexes at lower concentration predominantly shows higher cytotoxic activity against cancer cellular lines like MCF-7 and A-549 (human breast cancer cells and human lung cancer cells, respectively). DFT computational studies were carried out using GAUSSIAN 09 W software to understand the bonding mode and chemical parameters of the complexes were calculated using frontier molecular orbital studies.