Update README.md

README.md

@@ -8,11 +8,9 @@ pipeline_tag: text-generation
 ---
 ## Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-3bit-metaoffload-HQQ
 This is a version of the 
-<a href="https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1"> Mixtral-8x7B-Instruct-v0.1 model</a> quantized with a mix of 4-bit and 3-bit via Half-Quadratic Quantization (HQQ).
+<a href="https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1"> Mixtral-8x7B-Instruct-v0.1 model</a> quantized with a mix of 4-bit and 3-bit via Half-Quadratic Quantization (HQQ). More specifically, the attention layers are quantized to 4-bit and the experts are quantized to 3-bit. 
 
-More specifically, the attention layers are quantized to 4-bit and the experts are quantized to 3-bit. 
-
-Contrary to the <a href="https://huggingface.co/mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-2bitgs8-metaoffload-HQQ"> 2bitgs8 model </a> that was designed to use less GPU memory, this one uses about 22GB for the folks who want to get better quality and use the maximum VRAM available on 24GB GPUs. 
+Contrary to the <a href="https://huggingface.co/mobiuslabsgmbh/Mixtral-8x7B-Instruct-v0.1-hf-attn-4bit-moe-2bitgs8-metaoffload-HQQ"> 2bitgs8 model </a> that was designed to use less GPU memory, this one uses about ~22GB for the folks who want to get better quality and use the maximum VRAM available on 24GB GPUs. 
 
 ![image/gif](https://cdn-uploads.huggingface.co/production/uploads/636b945ef575d3705149e982/-gwGOZHDb9l5VxLexIhkM.gif)
 
@@ -25,10 +23,10 @@ Contrary to the <a href="https://huggingface.co/mobiuslabsgmbh/Mixtral-8x7B-Inst
 | Runtime VRAM      | 94 GB            | <b> 22.3 GB</b>     |  
 | ARC (25-shot)     | 70.22            |    69.62        |
 | Hellaswag (10-shot)|     87.63       |            |
-| MMLU (5-shot)      |      71.16      |           |
+| MMLU (5-shot)      |      71.16      |     69.46     |
 | TruthfulQA-MC2    | 64.58            |  62.63     |
 | Winogrande (5-shot)| 81.37           |  81.06       |
-| GSM8K (5-shot)|      60.73      |            |
+| GSM8K (5-shot)|      60.73      |         57.77   |
 | Average|   72.62         |           |
 
 ### Basic Usage
@@ -50,7 +48,6 @@ model     = HQQModelForCausalLM.from_quantized(model_id)
 from hqq.core.quantize import *
 HQQLinear.set_backend(HQQBackend.ATEN_BACKPROP)
 
-
 def chat_processor(chat, max_new_tokens=100, do_sample=True):
     tokenizer.use_default_system_prompt = False
     streamer = transformers.TextIteratorStreamer(tokenizer, timeout=10.0, skip_prompt=True, skip_special_tokens=True)
@@ -81,7 +78,6 @@ def chat_processor(chat, max_new_tokens=100, do_sample=True):
 outputs = chat_processor("How do I build a car?", max_new_tokens=1000, do_sample=False)
 ```
 
-
 ### Quantization
 
 You can reproduce the model using the following quant configs: