Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMs
Paper
•
2402.12030
•
Published
LLMs Distillation
-
-
mistralai/Mistral-7B-Instruct-v0.2
Text Generation • Updated • 3.61M • • 2.66k -
meta-llama/Llama-2-7b-chat-hf
Text Generation • Updated • 1.3M • • 4.23k -
EleutherAI/pythia-160m-deduped
Text Generation • Updated • 45.5k • 3 -
EleutherAI/pythia-410m-deduped
Text Generation • Updated • 21.3k • 20 -
EleutherAI/pythia-1b-deduped
Text Generation • Updated • 23.2k • 19 -
bigscience/bloomz-560m
Text Generation • Updated • 388k • 117 -
bigscience/mt0-base
Text2Text Generation • Updated • 3.69k • • 30 -
facebook/opt-350m
Text Generation • Updated • 1.01M • • 134 -
rajpurkar/squad
Viewer • Updated • 98.2k • 55.3k • 289 -
google-research-datasets/qed
Updated • 162 • 3 -
GEM/FairytaleQA
Viewer • Updated • 10.6k • 258 • 8 -
qiaojin/PubMedQA
Viewer • Updated • 274k • 3.86k • 170 -
knkarthick/dialogsum
Viewer • Updated • 14.5k • 10.9k • 192