GPT2-124M-TinyStories

This is a prototype / proof of concept model to see what the results of pretraining GPT2 exclusively on narrative texts would look like. That's right-- this isn't a finetune, it's entirely pretrained on TinyStories.

The GPT2 config and tokenizer is however unmodified from the original.

Downloads last month
28
Safetensors
Model size
124M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train DarwinAnim8or/gpt2-124M-tinystories