What dataset was this model finetuned on?
#3
by
saattrupdan
- opened
Hi and thanks for this fantastic model!
I was wondering about your finetuning dataset. Did you use the OASST1 dataset as the original Guanaco, or Alpaca, as seems to be the default in the QLoRA repo?
Thanks!
I trained this model using the finetune_guanaco_65b.sh script found in this part of the repo. That script was specifically added to reproduce the original Guanaco model. And as such use the same OASST1 dataset as Guanaco did. My goal was to be quite faithful to the original model, so the only part of the script I changed was the model it was pointed at. Everything else remained unchanged. So you can look at the script and see exactly what parameter the model was trained with.
Perfect, thanks! 😊
saattrupdan
changed discussion status to
closed