Attempt to reproduce Mixture-of-LoRAs classifier
Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models
https://arxiv.org/pdf/2403.03432
Datasets
We evenly sample about 10k training data and 2k validation data on each dataset.
From laion/OIG was taken only:
- unified_merged_code_xp3.jsonl
- unified_grade_school_math_instructions.jsonl
- unified_mathqa_flanv2_kojma_cot.jsonl
- Downloads last month
- 6