| license: apache-2.0 | |
| datasets: | |
| - wikipedia | |
| # Model details | |
| `minimoe-4L-384H` distilled from `bert-large-uncased` on `Wikipedia`. | |
| repository: [https://github.com/GeneZC/MiniMoE](https://github.com/GeneZC/MiniMoE) | |
| arXiv: [https://arxiv.org/abs/2305.12129](https://arxiv.org/abs/2305.12129) |