Update README.md
Browse files
README.md
CHANGED
|
@@ -11,6 +11,8 @@ language:
|
|
| 11 |
This is a finetune of https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T trained on a small 50mb subset of the Kobble Dataset.
|
| 12 |
Training was done in under 2 hours on a single Nvidia RTX 2060 Mobile GPU with qLora (LR 1.5e-4, rank 8, alpha 16, batch size 1, gradient acc. 4, 1024 ctx).
|
| 13 |
|
|
|
|
|
|
|
| 14 |
## Dataset and Objectives
|
| 15 |
|
| 16 |
The Kobble Dataset is a semi-private aggregated dataset made from multiple online sources and web scrapes.
|
|
|
|
| 11 |
This is a finetune of https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T trained on a small 50mb subset of the Kobble Dataset.
|
| 12 |
Training was done in under 2 hours on a single Nvidia RTX 2060 Mobile GPU with qLora (LR 1.5e-4, rank 8, alpha 16, batch size 1, gradient acc. 4, 1024 ctx).
|
| 13 |
|
| 14 |
+
You can obtain the GGUF quantization of this model here: https://huggingface.co/concedo/KobbleTiny-GGUF
|
| 15 |
+
|
| 16 |
## Dataset and Objectives
|
| 17 |
|
| 18 |
The Kobble Dataset is a semi-private aggregated dataset made from multiple online sources and web scrapes.
|