Quantization made by Richard Erkhov. [Github](https://github.com/RichardErkhov) [Discord](https://discord.gg/pvy7H8DZMG) [Request more models](https://github.com/RichardErkhov/quant_request) Dracarys2-72B-Instruct - GGUF - Model creator: https://huggingface.co/abacusai/ - Original model: https://huggingface.co/abacusai/Dracarys2-72B-Instruct/ | Name | Quant method | Size | | ---- | ---- | ---- | | [Dracarys2-72B-Instruct.Q2_K.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.Q2_K.gguf) | Q2_K | 27.76GB | | [Dracarys2-72B-Instruct.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.IQ3_XS.gguf) | IQ3_XS | 30.59GB | | [Dracarys2-72B-Instruct.IQ3_S.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.IQ3_S.gguf) | IQ3_S | 32.12GB | | [Dracarys2-72B-Instruct.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.Q3_K_S.gguf) | Q3_K_S | 32.12GB | | [Dracarys2-72B-Instruct.IQ3_M.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.IQ3_M.gguf) | IQ3_M | 33.07GB | | [Dracarys2-72B-Instruct.Q3_K.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.Q3_K.gguf) | Q3_K | 35.11GB | | [Dracarys2-72B-Instruct.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.Q3_K_M.gguf) | Q3_K_M | 29.28GB | | [Dracarys2-72B-Instruct.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.Q3_K_L.gguf) | Q3_K_L | 36.79GB | | [Dracarys2-72B-Instruct.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.IQ4_XS.gguf) | IQ4_XS | 7.05GB | | [Dracarys2-72B-Instruct.Q4_0.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q4_0 | 38.4GB | | [Dracarys2-72B-Instruct.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | IQ4_NL | 38.9GB | | [Dracarys2-72B-Instruct.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q4_K_S | 40.88GB | | [Dracarys2-72B-Instruct.Q4_K.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q4_K | 44.16GB | | [Dracarys2-72B-Instruct.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q4_K_M | 44.16GB | | [Dracarys2-72B-Instruct.Q4_1.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q4_1 | 42.56GB | | [Dracarys2-72B-Instruct.Q5_0.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q5_0 | 46.72GB | | [Dracarys2-72B-Instruct.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q5_K_S | 47.85GB | | [Dracarys2-72B-Instruct.Q5_K.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q5_K | 50.71GB | | [Dracarys2-72B-Instruct.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q5_K_M | 50.71GB | | [Dracarys2-72B-Instruct.Q5_1.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.Q5_1.gguf) | Q5_1 | 34.61GB | | [Dracarys2-72B-Instruct.Q6_K.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q6_K | 59.93GB | | [Dracarys2-72B-Instruct.Q8_0.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q8_0 | 71.96GB | Original model description: --- language: - en license: other tags: - chat license_name: tongyi-qianwen license_link: https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/main/LICENSE pipeline_tag: text-generation --- # Dracarys2-72B-Instruct # Introduction We introduce the latest in the Smaug series, the Dracarys family of finetunes targeting coding performance improvements across a variety of base models. This variant is a finetune of [Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct) Compared to Qwen2.5-72B-Instruct, Dracarys has better LiveCodeBench scores (see evaluation results below). ### Model Description - **Developed by:** [Abacus.AI](https://abacus.ai) - **License:** https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/main/LICENSE - **Finetuned from model:** [Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct). ## How to use The prompt format is unchanged from Qwen2.5-72B-Instruct (see evaluations for prompt details for LCB) ### Use with transformers See the snippet below for usage with Transformers: ```python import transformers import torch model_id = "abacusai/Dracarys2-72B-Instruct" pipeline = transformers.pipeline( "text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto", ) messages = [ {"role": "system", "content": "You are data science coding assistant that generates Python code using Pandas and Numpy."}, {"role": "user", "content": "Write code to select rows from the dataframe `df` having the maximum `temp` for each `city`"}, ] prompt = pipeline.tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) terminators = [ pipeline.tokenizer.eos_token_id, pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>") ] outputs = pipeline( prompt, max_new_tokens=256, eos_token_id=terminators, do_sample=True, temperature=0.6, top_p=0.9, ) print(outputs[0]["generated_text"][len(prompt):]) ``` # Evaluation Results ## LiveCodeBench | Model | Code Generation | Code Execution (COT) |Test Output Prediction | |----------------------------|-----------------|----------------------|-----------------------| | **Dracarys2-72B-Instruct** | **53.80** | **89.12** | **59.61** | | Qwen2.5-72B-Instruct | 53.03 | 88.72 | 46.28 | ## Breakdown of LiveCodeBench CodeGeneration | Model | Easy | Medium | Hard | |---------------------------|-----------------|----------------|---------------| | **Dracarys2-72B-Instruct**| **88.79** | **50.28** | 9.47 | | Qwen2.5-72B-Instruct | 86.99 | 49.59 | 9.99 | ## Breakdown of LiveCodeBench TestOutputPrediction | Model | Easy | Medium | Hard | |---------------------------|-----------------|----------------|-----------------------| | **Dracarys2-72B-Instruct**| **79.25** | **53.76** | **37.63** | | Qwen2.5-72B-Instruct | 68.43 | 39.46 | 22.22 |