Tamil character recognition is an important field of research in pattern recognition and it is a technical challenge than other languages due to similarity and complexity of characters. Stone inscriptions reveal the details of lavishness, lifestyle, economic conditions, culture, and also of the managerial regulations followed by various rulers and dynasties particular to those regions. However, due to the long history of ancient stone inscription, natural erosion and lack of early protection measures, there are lot of noise in the existing ancient stone inscriptions, which create adverse effects on reading these stone inscriptions and their aesthetic appreciation. The research challenge in recognizing Tamil characters is mainly because of the characters with a number of holes, loops and curves. The number of letters in Tamil language is higher when compared to other languages. Even though there are various approaches provided by the researchers, challenges and issues still prevail in recognition of tamil text in stone inscriptions. In the existing systems, detection algorithms fail to produce desired accuracy and hence stone inscription recognition using transfer learning, a promising method is proposed in this paper. Lion Optimization Algorithm (LOA) is applied to optimize brightness and contrast and then stone inscription images are pre-processed for noise removal and then each character is separated by identifying contours. Characters are recognized using Transfer Learning (TL), a Deep Convolution Neural Network-based multi classification approach. The proposed hybrid model Self-Adaptive Lion Optimization Algorithm with Transfer Learning (SLOA-TL) when implemented in images of stone inscriptions achieves better accuracy and speed than other existing methods. It serves as an efficient design for recognition of tamil characters in stone inscriptions and preserving tamil traditional knowledge.