Model-Agnostic Meta-Learning (MAML) is a widely used few-shot learning (FSL) technique that reduces reliance on large, labeled datasets in deep learning for medical imaging analysis. However, MAML requires backpropagating through all feature layers for task adaptation, leading to suboptimal computational efficiency. We propose IMAML-IDCG (ImageNet Model-Agnostic Meta-Learning in Invasive Ductal Carcinoma Grading), which enhances computational efficiency for few-shot grading of Invasive Ductal Carcinoma (IDC) through three key techniques: (1) ImageNet feature reusing, (2) ImageNet partial freezing strategy, and (3) adaptive inner learning rate. IMAML-IDCG is initialized with ImageNet pre-trained weights. During the inner optimization loop, only the model’s classifier head layer is optimized, leveraging prior ImageNet knowledge (ImageNet feature reusing) and employing an adaptive learning rate for improved task adaptation. In the outer optimization loop, IMAML-IDCG selectively fine-tunes the last few model layers to enhance efficiency and reduce overfitting (ImageNet partial freezing strategy). We evaluated IMAML-IDCG using the BreaKHis dataset (7,909 images) as the base dataset, and the BCHI (282 images) and PathoIDCG (3,744 images) datasets as the novel datasets. Our empirical results demonstrate that IMAML-IDCG outperforms MAML and other FSL methods in few-shot IDC grading tasks across various cross-magnification domain settings. Notably, IMAML-IDCG achieves a 14.64% improvement over MAML on the BCHI dataset and a 6.04% improvement on the PathoIDCG 40X dataset when meta-trained with the BreaKHis 40X dataset in the 3-way 5-shot scenario.