Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
BF16:
|
| 2 |
```
|
| 3 |
|-------------|------:|--------|-----:|---|-----:|
|
|
|
|
| 1 |
+
## Perplexity (ctx 8192)
|
| 2 |
+
|
| 3 |
+
BF16:
|
| 4 |
+
```
|
| 5 |
+
I1005 10:12:54.337877 967332 eval_ppl.py:66] wikitext2 perplexity: 6.49992036819458
|
| 6 |
+
I1005 10:17:19.718732 967332 eval_ppl.py:66] c4 perplexity: 8.022844314575195
|
| 7 |
+
```
|
| 8 |
+
4 Bit QTIP (this model):
|
| 9 |
+
```
|
| 10 |
+
I1005 10:13:22.272425 967335 eval_ppl.py:66] wikitext2 perplexity: 6.610454082489014
|
| 11 |
+
I1005 10:19:33.215519 967335 eval_ppl.py:66] c4 perplexity: 8.128130912780762
|
| 12 |
+
```
|
| 13 |
+
|
| 14 |
+
## Zeroshot Results with lm_eval 0.3.0
|
| 15 |
+
|
| 16 |
BF16:
|
| 17 |
```
|
| 18 |
|-------------|------:|--------|-----:|---|-----:|
|