Update README.md
Browse files
README.md
CHANGED
@@ -9,7 +9,7 @@ language:
|
|
9 |
</div>
|
10 |
|
11 |
This is a finetune of https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T trained on a small 50mb subset of the Kobble Dataset.
|
12 |
-
Training was done in under 2 hours on a single Nvidia RTX 2060 Mobile GPU with qLora (LR 1.5e-4, rank 8, alpha 16, batch size
|
13 |
|
14 |
You can obtain the GGUF quantization of this model here: https://huggingface.co/concedo/KobbleTinyV2-GGUF
|
15 |
|
|
|
9 |
</div>
|
10 |
|
11 |
This is a finetune of https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T trained on a small 50mb subset of the Kobble Dataset.
|
12 |
+
Training was done in under 2 hours on a single Nvidia RTX 2060 Mobile GPU with qLora (LR 1.5e-4, rank 8, alpha 16, batch size 2, gradient acc. 4, 2048 ctx).
|
13 |
|
14 |
You can obtain the GGUF quantization of this model here: https://huggingface.co/concedo/KobbleTinyV2-GGUF
|
15 |
|