Added PPL results with new-eval

README.md

@@ -40,12 +40,30 @@ Refs:
 
 # Benchmarks
 
-Best results in **bold**.
+Using https://github.com/qwopqwop200/GPTQ-for-LLaMa/. Best results in **bold**.
 
-| Model  | wikitext2 PPL  | c4 PPL  | ptb PPL  | C4-VRAM  |
+*`--benchmark 2048 --check` results:*
+
+| Model  | wikitext2 PPL  | ptb PPL  | c4 PPL  | VRAM Utilization  |
 |---|---|---|---|---|
-| 4bit-GPTQ - TheBloke/vicuna-13B-1.1-GPTQ-4bit-128g  | 8.517391204833984  | **7.058407783508301**  | 20.888103485107422  | **8670.26953125**  |
-| 8bit-GPTQ - Thireus/Vicuna13B-v1.1-8bit-128g  | **8.508771896362305**  | 7.105874538421631  | **20.75649070739746**  | 14840.26171875  |
+| 4bit-GPTQ - TheBloke/vicuna-13B-1.1-GPTQ-4bit-128g  | 8.517391204833984  | 20.888103485107422  | **7.058407783508301**  | **8670.26953125**  |
+| 8bit-GPTQ - Thireus/Vicuna13B-v1.1-8bit-128g  | **8.508771896362305**  | **20.75649070739746**  | 7.105874538421631  | 14840.26171875  |
+
+*`--eval` results (pending):*
+
+| Model  | wikitext2 PPL  | ptb PPL  | c4 PPL  |
+|---|---|---|---|
+| 4bit-GPTQ - TheBloke/vicuna-13B-1.1-GPTQ-4bit-128g  |  |  |   |
+| 8bit-GPTQ - Thireus/Vicuna13B-v1.1-8bit-128g  |   |   |   |
+
+*`--new-eval --eval` results:*
+
+| Model  | wikitext2 PPL  | ptb-new PPL  | c4-new PPL  |
+|---|---|---|---|
+| 4bit-GPTQ - TheBloke/vicuna-13B-1.1-GPTQ-4bit-128g  | 7.119165420532227  | 35.637290954589844  | 9.550592422485352  |
+| 8bit-GPTQ - Thireus/Vicuna13B-v1.1-8bit-128g  | **6.988043308258057**  | **34.264320373535156**  | **9.426002502441406**  |
+
+PPL = Perplexity - https://huggingface.co/docs/transformers/perplexity
 
 <br>
 <br>