Agricultural extension is a communication network linking different stakeholders in agriculture to improve their productivity and Information Communication Technology (ICT) has been utilised as an extension tool for enhancing information flow between agricultural extension services and their clients. The application of ICT in agricultural extension and rural development has significantly increased in several countries where it has provided an adequate access to agricultural information. Efforts are, therefore, needed to scale up investments in physical ICT infrastructure and services across developing country. This could be realised through the implementation of interventions aimed at speeding up assimilation and adoption of improved agricultural technology and management practices of the less productive smallholder farmers. Drawing on relevant published works, this paper argues that agricultural knowledge and information management within an extension system can improve productivity of smallholder farmers. The role of ICTs in agricultural extension is discussed together with its challenges towards the improvement of productivity among smallholder farmers. Smallholder farmers need to develop and utilize ICT based knowledge management techniques to implement strategies and interventions to transform the agricultural sector and improve their productivity. Agricultural extension is the ideal mechanism that can facilitate the introduction and subsequent adoption of ICTs for effective knowledge and information management.