--- license: other license_name: tongyi-qianwen license_link: https://huggingface.co/Qwen/Qwen1.5-72B-Chat/blob/main/LICENSE pipeline_tag: text-generation library_name: gguf base_model: Qwen/Qwen1.5-72B-Chat --- **NOTE**: You will need a recent build of llama.cpp to run these quants (i.e. at least commit `494c870`). **2024-03-19**: Uploading new quants retrained on `wiki.train.raw` for ~100K tokens. **2024-03-07**: Refreshing quants using latest build as things seem to have stabilized a bit now. GGUF importance matrix (imatrix) quants for https://huggingface.co/Qwen/Qwen1.5-72B-Chat * The importance matrix was trained for ~50K tokens (105 batches of 512 tokens) using a [general purpose imatrix calibration dataset](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384). * The [imatrix is being used on the K-quants](https://github.com/ggerganov/llama.cpp/pull/4930) as well. | Layers | Context | [Template](https://huggingface.co/Qwen/Qwen1.5-72B-Chat/blob/main/tokenizer_config.json#L31) | | --- | --- | --- | |

<\|im_start\|>system
{instructions}<\|im_end\|>
<\|im_start\|>user
{prompt}<\|im_end\|>
<\|im_start\|>assistant
{response}