Efficient parameter tuning of neural foundation models for drug perspective prediction from unstructured socio-medical data

Abstract

The phenomenal popularity of social media platforms over the past decade has accelerated the development of intelligent applications that leverage social media data for informed decision-making in diverse domains like finance, education, public policy and healthcare management practices. While understanding the colloquial language of users on social media remains a challenging problem, access to users’ medical perspectives that conversationally divulge healthcare-related experiences and insights can help reshape healthcare ecosystems like chronic disease management, pandemics, public health, pharmacovigilance and more. Most existing models are constrained to a particular dataset while neglecting model adaptability across data sources and domains. Model generalization across variable data sizes also has received very little research attention. Conventional foundation models can be fine-tuned by adding additional model heads or by appending contributing network layers, however, there has been very little focus on effective parameter calibration for adapting neural foundation models to a specific task. In this study, an Adaptive Learning mechanism for Socio-Medical data (AL4SM) built on generic foundation neural models with efficient parameter learning is proposed, to categorize users’ perspectives on prescription drug-related experiences and adapt to diverse socio-medical data sources of variable sizes. AL4SM aims to lighten the over-parameterized mechanisms adopted by existing foundational techniques by efficiently learning latent medical information based on optimized parameter calibration and weight reinitialization techniques. Comprehensive cross-domain and cross-data analyses are undertaken to explore specific user perspectives related to prescription effectiveness and side effects. Validation experiments conducted on standard datasets obtained from Drugs.com and Druglib.com revealed that the proposed AL4SM outperformed state-of-the-art models, achieving an improvement of 6.06% in accuracy and 7.62% in F1-score for 3-class and 2% in F1-score for 10-class drug perspective categorization. The cross-data experiments further emphasized the superiority of the proposed model, with improved accuracy of 17% on Drugs.com and 9% on Druglib.com datasets, respectively.