| library_name: gguf | |
| pipeline_tag: text-generation | |
| base_model: jondurbin/bagel-dpo-34b-v0.5 | |
| license: apache-2.0 | |
| * GGUF importance matrix (imatrix) quants for https://huggingface.co/jondurbin/bagel-dpo-34b-v0.5 | |
| * The importance matrix was trained for 100K tokens (200 batches of 512 tokens) using wiki.train.raw. | |
| * The [imatrix is being used on the K-quants](https://github.com/ggerganov/llama.cpp/pull/4930) as well (below Q6_K). | |
| * Generated with llama.cpp commit `f87f7b89` | |
| | Layers | Context | [Template](https://huggingface.co/jondurbin/bagel-dpo-34b-v0.5#prompt-formatting) | | |
| | --- | --- | --- | | |
| | <pre>60</pre> | <pre>200000</pre> | <pre>[INST] \<\<SYS\>\><br>{instructions}<br>\<\</SYS\>\><br><br>{prompt} [/INST]<br>{response}</pre> | |