Update README.md

README.md

@@ -3,5 +3,27 @@ language:
 - fi
 pipeline_tag: text-generation
 ---
+Generative Pretrained Transformer with 8B parameteres for Finnish.
 
-GPT-3 small for Finnish. Documentation coming soon!
+TurkuNLP Finnish GPT-3-models are a model family of pretrained monolingual GPT-style language models that are based on BLOOM-architecture.
+Note that the models are pure language models, meaning that they are not [instruction finetuned](https://arxiv.org/abs/2203.02155) for dialogue
+or answering questions.
+
+These models are intended to be used as foundational models that can be e.g. instruction finetuned to serve as modern chat-models.
+
+
+
+**Parameters**
+| Model | Layers | Dim  | Heads | Params |
+|--------|--------|------|-------|--------|
+| Small  | 12     | 768  | 12    | 186M   |
+| Medium | 24     | 1024 | 16    | 437M   |
+| Large  | 24     | 1536 | 16    | 881M   |
+| XL     | 24     | 2064 | 24    | 1.5B   |
+| ”3B”   | 32     | 2560 | 32    | 2.8B   |
+| ”8B”   | 32     | 4096 | 32    | 7.5B   |
+| "13B"  | 40     | 5120 | 40    | 13.3B  |
+
+
+
+More documentation coming soon!