Commit
•
0eccac0
1
Parent(s):
a90048d
Update README.md
Browse files
README.md
CHANGED
|
@@ -33,14 +33,16 @@ dtype: float16
|
|
| 33 |
```
|
| 34 |
|
| 35 |
First exllama quantization pass:
|
| 36 |
-
|
| 37 |
-
|
|
|
|
| 38 |
|
| 39 |
Second exllama quantization pass:
|
|
|
|
|
|
|
|
|
|
| 40 |
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
Both are 200K context models with Vicuna syntax, so:
|
| 44 |
|
| 45 |
# Prompt Format:
|
| 46 |
|
|
|
|
| 33 |
```
|
| 34 |
|
| 35 |
First exllama quantization pass:
|
| 36 |
+
```
|
| 37 |
+
python convert.py --in_dir /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K -o /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K-exl2 -om /home/alpha/FastModels/capytessmes.json --cal_dataset /home/alpha/Documents/smol.parquet -l 2048 -r 80 -ml 2048 -mr 40 -gr 40 -ss 4096 -nr -b 3.5 -hb 6
|
| 38 |
+
```
|
| 39 |
|
| 40 |
Second exllama quantization pass:
|
| 41 |
+
```
|
| 42 |
+
python convert.py --in_dir /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K -o /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K-exl2 -m /home/alpha/FastModels/capytessmes.json --cal_dataset /home/alpha/Documents/medium.parquet -l 2048 -r 200 -ml 2048 -mr 40 -gr 200 -ss 4096 -b 3.1 -hb 6 -cf /home/alpha/FastModels/Capybara-Tess-Yi-34B-200K-exl2-31bpw -nr
|
| 43 |
+
```
|
| 44 |
|
| 45 |
+
Both models have Vicuna syntax, so:
|
|
|
|
|
|
|
| 46 |
|
| 47 |
# Prompt Format:
|
| 48 |
|