Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
ggml version (post-PR #1405)
|
| 2 |
+
|
| 3 |
+
As a base model used https://huggingface.co/eachadea/vicuna-13b-1.1
|
| 4 |
+
|
| 5 |
+
Finetuned on Teknium's GPTeacher dataset, unreleased Roleplay v2 dataset, GPT-4-LLM dataset, and Nous Research Instruct Dataset
|
| 6 |
+
|
| 7 |
+
Approx 180k instructions, all from GPT-4, all cleaned of any OpenAI censorship/"As an AI Language Model" etc.
|
| 8 |
+
|
| 9 |
+
Base model still has OpenAI censorship. Soon, a new version will be released with cleaned vicuna from https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltere
|
| 10 |
+
|
| 11 |
+
Trained on 8 A100-80GB GPUs for 5 epochs following Alpaca deepspeed training code.
|
| 12 |
+
|
| 13 |
+
Nous Research Instruct Dataset will be released soon.
|
| 14 |
+
|
| 15 |
+
GPTeacher, Roleplay v2 by https://huggingface.co/teknium
|
| 16 |
+
|
| 17 |
+
Wizard LM by https://github.com/nlpxucan
|
| 18 |
+
|
| 19 |
+
Nous Research Instruct Dataset by https://huggingface.co/karan4d and https://huggingface.co/huemin
|
| 20 |
+
|
| 21 |
+
Compute provided by our project sponsor https://redmond.ai/
|