Hugging Face's logo

RichardErkhov
/
abacusai_-_Dracarys2-72B-Instruct-gguf

GGUF
Inference Endpoints
conversational
Model card Files Community
1
abacusai_-_Dracarys2-72B-Instruct-gguf
File size: 7,183 Bytes 
5271eb4
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
 
Quantization made by Richard Erkhov.

[Github](https://github.com/RichardErkhov)

[Discord](https://discord.gg/pvy7H8DZMG)

[Request more models](https://github.com/RichardErkhov/quant_request)


Dracarys2-72B-Instruct - GGUF
- Model creator: https://huggingface.co/abacusai/
- Original model: https://huggingface.co/abacusai/Dracarys2-72B-Instruct/


| Name | Quant method | Size |
| ---- | ---- | ---- |
| [Dracarys2-72B-Instruct.Q2_K.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.Q2_K.gguf) | Q2_K | 27.76GB |
| [Dracarys2-72B-Instruct.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.IQ3_XS.gguf) | IQ3_XS | 30.59GB |
| [Dracarys2-72B-Instruct.IQ3_S.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.IQ3_S.gguf) | IQ3_S | 32.12GB |
| [Dracarys2-72B-Instruct.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.Q3_K_S.gguf) | Q3_K_S | 32.12GB |
| [Dracarys2-72B-Instruct.IQ3_M.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.IQ3_M.gguf) | IQ3_M | 33.07GB |
| [Dracarys2-72B-Instruct.Q3_K.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.Q3_K.gguf) | Q3_K | 35.11GB |
| [Dracarys2-72B-Instruct.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.Q3_K_M.gguf) | Q3_K_M | 29.28GB |
| [Dracarys2-72B-Instruct.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.Q3_K_L.gguf) | Q3_K_L | 36.79GB |
| [Dracarys2-72B-Instruct.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.IQ4_XS.gguf) | IQ4_XS | 7.05GB |
| [Dracarys2-72B-Instruct.Q4_0.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q4_0 | 38.4GB |
| [Dracarys2-72B-Instruct.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | IQ4_NL | 38.9GB |
| [Dracarys2-72B-Instruct.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q4_K_S | 40.88GB |
| [Dracarys2-72B-Instruct.Q4_K.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q4_K | 44.16GB |
| [Dracarys2-72B-Instruct.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q4_K_M | 44.16GB |
| [Dracarys2-72B-Instruct.Q4_1.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q4_1 | 42.56GB |
| [Dracarys2-72B-Instruct.Q5_0.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q5_0 | 46.72GB |
| [Dracarys2-72B-Instruct.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q5_K_S | 47.85GB |
| [Dracarys2-72B-Instruct.Q5_K.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q5_K | 50.71GB |
| [Dracarys2-72B-Instruct.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q5_K_M | 50.71GB |
| [Dracarys2-72B-Instruct.Q5_1.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/blob/main/Dracarys2-72B-Instruct.Q5_1.gguf) | Q5_1 | 34.61GB |
| [Dracarys2-72B-Instruct.Q6_K.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q6_K | 59.93GB |
| [Dracarys2-72B-Instruct.Q8_0.gguf](https://huggingface.co/RichardErkhov/abacusai_-_Dracarys2-72B-Instruct-gguf/tree/main/) | Q8_0 | 71.96GB |




Original model description:
---
language:
- en
license: other
tags:
- chat
license_name: tongyi-qianwen
license_link: https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/main/LICENSE
pipeline_tag: text-generation
---

# Dracarys2-72B-Instruct

# Introduction

We introduce the latest in the Smaug series, the Dracarys family of finetunes targeting coding performance improvements
across a variety of base models.

This variant is a finetune of [Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct)

Compared to Qwen2.5-72B-Instruct, Dracarys has better LiveCodeBench scores (see evaluation results below).

### Model Description

- **Developed by:** [Abacus.AI](https://abacus.ai)
- **License:** https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/main/LICENSE
- **Finetuned from model:** [Qwen2.5-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct).

## How to use

The prompt format is unchanged from Qwen2.5-72B-Instruct (see evaluations for prompt details for LCB)

### Use with transformers

See the snippet below for usage with Transformers:

```python
import transformers
import torch

model_id = "abacusai/Dracarys2-72B-Instruct"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are data science coding assistant that generates Python code using Pandas and Numpy."},
    {"role": "user", "content": "Write code to select rows from the dataframe `df` having the maximum `temp` for each `city`"},
]

prompt = pipeline.tokenizer.apply_chat_template(
		messages, 
		tokenize=False, 
		add_generation_prompt=True
)

terminators = [
    pipeline.tokenizer.eos_token_id,
    pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = pipeline(
    prompt,
    max_new_tokens=256,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)
print(outputs[0]["generated_text"][len(prompt):])
```

# Evaluation Results


## LiveCodeBench

| Model                      | Code Generation | Code Execution (COT) |Test Output Prediction |
|----------------------------|-----------------|----------------------|-----------------------|
| **Dracarys2-72B-Instruct** | **53.80**       | **89.12**            | **59.61**             |
| Qwen2.5-72B-Instruct       | 53.03           | 88.72                | 46.28                 |

## Breakdown of LiveCodeBench CodeGeneration

| Model                     | Easy            | Medium         | Hard          |
|---------------------------|-----------------|----------------|---------------|
| **Dracarys2-72B-Instruct**| **88.79**       | **50.28**      | 9.47          |
| Qwen2.5-72B-Instruct      |  86.99          | 49.59          | 9.99          |

## Breakdown of LiveCodeBench TestOutputPrediction

| Model                     | Easy            | Medium         | Hard                  |
|---------------------------|-----------------|----------------|-----------------------|
| **Dracarys2-72B-Instruct**| **79.25**       | **53.76**      | **37.63**             |
| Qwen2.5-72B-Instruct      |  68.43          |  39.46         |  22.22                |