| license: mit | |
| datasets: | |
| - oscar | |
| language: | |
| - uk | |
| library_name: transformers | |
| pipeline_tag: text-generation | |
| # GPT2 Ukrainian | |
| A generative language model for the Ukrainian language follows the [GPT-2 architecture](https://huggingface.co/gpt2) (124M parameters). | |
| - hidden size: 768 | |
| - number of heads: 12 | |
| - number of layers: 12 | |
| - seq length: 1024 | |
| - tokens: 11238113280 (3 epochs) | |
| - steps: 57167 | |
| ## Training data | |
| - OSCAR | |
| - Wikimedia dumps | |
| ## License | |
| MIT | |