Commit
·
2dc02d8
1
Parent(s):
d39f7b9
update readme
Browse files
README.md
CHANGED
@@ -18,8 +18,8 @@ Finetuned with ORCA-GPT4 (chatml format)
|
|
18 |
This is a fine-tuned version of [TinyLlama-1.1B](https://github.com/jzhang38/TinyLlama) using the [sam-mosaic/orca-gpt4-chatml](https://huggingface.co/datasets/sam-mosaic/orca-gpt4-chatml) dataset.
|
19 |
|
20 |
### Training
|
21 |
-
- Method
|
22 |
-
- Quantization
|
23 |
-
- Time
|
24 |
-
- Cost
|
25 |
-
-
|
|
|
18 |
This is a fine-tuned version of [TinyLlama-1.1B](https://github.com/jzhang38/TinyLlama) using the [sam-mosaic/orca-gpt4-chatml](https://huggingface.co/datasets/sam-mosaic/orca-gpt4-chatml) dataset.
|
19 |
|
20 |
### Training
|
21 |
+
- **Method**: QLORA
|
22 |
+
- **Quantization**: 16bit
|
23 |
+
- **Time**: 20h on a RTX 4090 (from runpod.io)
|
24 |
+
- **Cost**: About $15
|
25 |
+
- **Based on**: [https://colab.research.google.com/drive/1Zmaceu65d7w4Tcd-cfnZRb6k_Tcv2b8g](https://colab.research.google.com/drive/1Zmaceu65d7w4Tcd-cfnZRb6k_Tcv2b8g)
|