Dr. LLaMA: Improving Small Language Models on PubMedQA via Generative Data Augmentation

Add code
May 17, 2023
Zhen Guo, Peiqi Wang, Yanwei Wang, Shangdi Yu

Figure 1 for Dr. LLaMA: Improving Small Language Models on PubMedQA via Generative Data Augmentation
Figure 2 for Dr. LLaMA: Improving Small Language Models on PubMedQA via Generative Data Augmentation
Figure 3 for Dr. LLaMA: Improving Small Language Models on PubMedQA via Generative Data Augmentation
Figure 4 for Dr. LLaMA: Improving Small Language Models on PubMedQA via Generative Data Augmentation

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon

Large Language Models (LLMs) have made remarkable strides in natural language processing, but their expanding size poses challenges in terms of computational expense and inefficiency. Conversely, Small Language Models (SLMs) are known for their efficiency but often encounter difficulties in tasks with limited capacity and training data, particularly in domain-specific scenarios. In this paper, we introduce Dr. LLaMA, a method that improves SLMs in the medical domain through generative data augmentation utilizing LLMs. The objective is to develop more efficient and capable models tailored for specialized applications. Our preliminary results on the PubMedQA dataset demonstrate that LLMs effectively refine and diversify existing question-answer pairs, leading to improved performance of a significantly smaller model after fine-tuning. The best SLM surpasses few-shot GPT-4 with under 1.6 billion parameters on the PubMedQA. Our code and generated data are publicly available to facilitate further explorations.

View paper onarxiv icon

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon