| license: apache-2.0 | |
| datasets: | |
| - HuggingFaceTB/smollm-corpus | |
| - mittagessen/oscar_subset | |
| base_model: | |
| - mittagessen/bytellama_random | |
| This is a [ByteLlama](https://github.com/mittagessen/bytellama) 101M model pretrained on the Cosmopedia v2 portion of the SmolLM corpus for 2 epochs, followed by training on a subset of OSCAR for another epoch. |