The nuclear security architecture in India is three-fold: the infrastructure security including physical security of the nuclear plant; port and border security including training and capacity building to prevent any illicit trafficking of nuclear material into the country and; the inter-institutional coordination at the Centre and State level during radiological emergencies. However, there exist structural weaknesses that need to be accounted for in all these three areas. It is imperative to develop capacities not only for safe handling, transport and disposal of nuclear material but also instituting stringent cyber-security laws, border security measures and functional Centre-State coordination in crisis situation for the safety and security of the general population. The following paper seeks to address these challenges and provide recommendations for strengthening the nuclear security and disaster response framework in India. The paper draws recommendations from the 2019 IPCS workshop on Smuggling and Radiation Detection, on the illicit trafficking of radioactive materials supported by the Office of Nuclear Smuggling Detection and Deterrence (NSDD) of the United States Department of Energy (DOE).