|
--- |
|
language: |
|
- en |
|
- zh |
|
license: apache-2.0 |
|
library_name: transformers |
|
datasets: |
|
- EleutherAI/pile |
|
- togethercomputer/RedPajama-Data-1T |
|
- p208p2002/wudao |
|
widget: |
|
- text: <s> 4 + 3 = |
|
--- |
|
## MiniLoong-3B |
|
|
|
📑 [arXiv](https://arxiv.org/abs/2311.07052) | 👻 [GitHub](https://github.com/GeneZC/MiniMA) | 🤗 [HuggingFace-MiniMA-3B](https://huggingface.co/GeneZC/MiniMA-3B) | 🤗 [HuggingFace-MiniChat-3B](https://huggingface.co/GeneZC/MiniChat-3B) | 🤖 [ModelScope-MiniMA-3B](https://modelscope.cn/models/GeneZC/MiniMA-3B) | 🤖 [ModelScope-MiniChat-3B](https://modelscope.cn/models/GeneZC/MiniChat-3B) | 🤗 [HuggingFace-MiniChat-1.5-3B](https://huggingface.co/GeneZC/MiniChat-1.5-3B) | 🤗 [HuggingFace-MiniMA-2-3B](https://huggingface.co/GeneZC/MiniMA-2-3B) | 🤗 [HuggingFace-MiniChat-2-3B](https://huggingface.co/GeneZC/MiniChat-2-3B) | 🤗 [HuggingFace-MiniMA-2-1B](https://huggingface.co/GeneZC/MiniMA-2-1B) | 🤗 [HuggingFace-MiniLoong-3B](https://huggingface.co/GeneZC/MiniLoong-3B) | 🤗 [HuggingFace-MiniMix-2/4x3B](https://huggingface.co/GeneZC/MiniMix-2_4x3B) |
|
|
|
❗ Must comply with LICENSE of LLaMA-2 since it is derived from LLaMA-2. |
|
|
|
<img src="./teaser_d.jpg" alt="teaser_d" width="700" /> |
|
|
|
## Bibtex |
|
|
|
```bibtex |
|
@article{zhang2023law, |
|
title={Towards the Law of Capacity Gap in Distilling Language Models}, |
|
author={Zhang, Chen and Song, Dawei and Ye, Zheyu and Gao, Yan}, |
|
year={2023}, |
|
url={https://arxiv.org/abs/2311.07052} |
|
} |
|
``` |
