Commit
•
39f6fc8
1
Parent(s):
934bda9
Update README.md
Browse files
README.md
CHANGED
|
@@ -6,6 +6,7 @@ license: apache-2.0
|
|
| 6 |
|
| 7 |
This is an intermediate model used as base-model for further pythia 12b SFT-8 experiments.
|
| 8 |
It was trained on a wider set of instruction-tuning datasets for >12.5k steps with batch-size 128 and a context size of 2048.
|
|
|
|
| 9 |
|
| 10 |
|
| 11 |
- wandb: https://wandb.ai/open-assistant/supervised-finetuning/runs/sytsyhrp
|
|
|
|
| 6 |
|
| 7 |
This is an intermediate model used as base-model for further pythia 12b SFT-8 experiments.
|
| 8 |
It was trained on a wider set of instruction-tuning datasets for >12.5k steps with batch-size 128 and a context size of 2048.
|
| 9 |
+
The gpt4all dataset had "as a language model" *contamination* (>1.8k entries). We added filtering later, but this model (pre-v8) was trained on the raw unfildered gpt4all dataset.
|
| 10 |
|
| 11 |
|
| 12 |
- wandb: https://wandb.ai/open-assistant/supervised-finetuning/runs/sytsyhrp
|