File size: 494 Bytes
6595fb3 4cefaa9 0c44282 d1599f9 0c44282 bcc9d40 c11ef27 bcc9d40 53ac067 bcc9d40 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
---
license: mit
language:
- ru
metrics:
- perplexity
pipeline_tag: text-generation
---
This model was created by [ilnikolaev](https://huggingface.co/ilnikolaev)
Trained from scratch using Tensorflow Keras
[200mb Russian Comments from 2ch](https://www.kaggle.com/datasets/fizzzgen/65mb-of-dvach-conversations) dataset used
- Type: decoder-only
- Tokenizer: BPE
- Vocabulary size: 32000
- Max sequence length: 120
- Hidden size: 768
- FFN size: 3072
- Attention heads: 24
- Decoder layers: 4 |