Hugging Face's logo

tensorblock
/
Qwen2.5-Math-7B-Instruct-GGUF

Transformers
GGUF
English
unsloth
TensorBlock
GGUF
Inference Endpoints
conversational
Model card Files Community
morriszms commited on
Commit
7e7ef1c
•
1 Parent(s): ad5c137

Upload folder using huggingface_hub

Browse files
Files changed (13) hide show
  1. Qwen2.5-Math-7B-Instruct-Q2_K.gguf +2 -2
  2. Qwen2.5-Math-7B-Instruct-Q3_K_L.gguf +2 -2
  3. Qwen2.5-Math-7B-Instruct-Q3_K_M.gguf +2 -2
  4. Qwen2.5-Math-7B-Instruct-Q3_K_S.gguf +2 -2
  5. Qwen2.5-Math-7B-Instruct-Q4_0.gguf +2 -2
  6. Qwen2.5-Math-7B-Instruct-Q4_K_M.gguf +2 -2
  7. Qwen2.5-Math-7B-Instruct-Q4_K_S.gguf +2 -2
  8. Qwen2.5-Math-7B-Instruct-Q5_0.gguf +2 -2
  9. Qwen2.5-Math-7B-Instruct-Q5_K_M.gguf +2 -2
  10. Qwen2.5-Math-7B-Instruct-Q5_K_S.gguf +2 -2
  11. Qwen2.5-Math-7B-Instruct-Q6_K.gguf +2 -2
  12. Qwen2.5-Math-7B-Instruct-Q8_0.gguf +2 -2
  13. README.md +19 -22
Qwen2.5-Math-7B-Instruct-Q2_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c669c66582ac13bf98ee221b1108b218eeda6ba06692e2dd6d420228c2bc83bc
3
- size 3015940544
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee20eaf9014337ce37801176344aed16b9afcaea5448f0f8ea29ca88355dcbd2
3
+ size 3015940480
Qwen2.5-Math-7B-Instruct-Q3_K_L.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0a564160a77d956bdac3d6726e96c3613ee8a15f5d6cb5538bf06f5583bcf1e1
3
- size 4088459712
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e25b667bf968ccb6bb566a596abb681e9326b2ad75f99d438441fdef7a4dfbcf
3
+ size 4088459648
Qwen2.5-Math-7B-Instruct-Q3_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:00d6822db5f3e03d0689e692ee63e5decead696b41b102d5fd2733dac494f98c
3
- size 3808391616
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1fe4d4ea793e1d92b9853ce573cabf3becea7354d54e4455513d64b2a8fe94d0
3
+ size 3808391552
Qwen2.5-Math-7B-Instruct-Q3_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3f39b6cded222d895acf13bc0b3bb8a4da1a59bb4bef4bcb9f8895ba2bdd2ca6
3
- size 3492368832
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:451911a1e24e786a84fa2796d371818243f71972b80ec0574037dd1dd4b8cf8e
3
+ size 3492368768
Qwen2.5-Math-7B-Instruct-Q4_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:075756d645005a01ac731ca9dae949c7d29c3a69f45d51527c5d02c62c5150a7
3
- size 4431391168
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:da011599d042d748c37cea92bc9c3c88c32ffa177282f38ca9d7a0ba7be1403b
3
+ size 4431391104
Qwen2.5-Math-7B-Instruct-Q4_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:eede280728071b35293c3631aeb960ad74f4725647f0e1a20ab2685633311f24
3
- size 4683073984
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:62b4bb6e48d5606b3abf4f85b054ef3a75798f3e61e8634ea8fce7c1b31a9ad7
3
+ size 4683073920
Qwen2.5-Math-7B-Instruct-Q4_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0d0d9a59ea6f421c352fbc10571713f85de89aac1cdbd0ec6e24bb1399619161
3
- size 4457769408
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5122eeadb5b802c5cb2c8442272dfbaaf81ba77a6581f01701e1fc85e70937bf
3
+ size 4457769344
Qwen2.5-Math-7B-Instruct-Q5_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4372d8e7e852341e34199125d272890435c65f0e256ebfa6e885d43e194b3118
3
- size 5315176896
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f49af74cfd6b4edd32693a56674af5e852f50a2671071699acfcb8b4c278f12a
3
+ size 5315176832
Qwen2.5-Math-7B-Instruct-Q5_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:80b3708e8751c0e7f13e1fb74ff92b7ea3b71e5fb267a35229db178c33ca6165
3
- size 5444831680
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b49c8699d3f54e3f3fbaf21c8e6da7170ccf2c7c569e074720d890ac93249829
3
+ size 5444831616
Qwen2.5-Math-7B-Instruct-Q5_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:80d360907a96dbc498b7c561d68aabbcd512dd1aded812560d39f0f454739639
3
- size 5315176896
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:548cd34f6ebeee339d1ca13627cc5f7cfcde8ad9972ef037b28f001e47baa01e
3
+ size 5315176832
Qwen2.5-Math-7B-Instruct-Q6_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fcd0aaa57764f3530cd3ae1ebbad87a68d47aea0bf90866093d4817e515a47b4
3
- size 6254199232
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:16aa7375fe470b6c907b3c4968e268edddc8948c51b0d4c27fffe3db5a9b47d2
3
+ size 6254199168
Qwen2.5-Math-7B-Instruct-Q8_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:473541672e34edca345ae54b1bb7852ce1f52fecf5d8035de583754adc5855a3
3
- size 8098525632
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bbd0dafa82e91b9b3eecd46f277ad4ae8627480e8379b4c7b837400e9d9b3977
3
+ size 8098525568
README.md CHANGED
@@ -1,15 +1,14 @@
1
  ---
2
- base_model: Qwen/Qwen2.5-Math-7B-Instruct
3
  language:
4
  - en
5
- pipeline_tag: text-generation
 
6
  tags:
7
- - chat
 
8
  - TensorBlock
9
  - GGUF
10
- library_name: transformers
11
- license: apache-2.0
12
- license_link: https://huggingface.co/Qwen/Qwen2.5-Math-7B-Instruct/blob/main/LICENSE
13
  ---
14
 
15
  <div style="width: auto; margin-left: auto; margin-right: auto">
@@ -23,13 +22,12 @@ license_link: https://huggingface.co/Qwen/Qwen2.5-Math-7B-Instruct/blob/main/LIC
23
  </div>
24
  </div>
25
 
26
- ## Qwen/Qwen2.5-Math-7B-Instruct - GGUF
27
 
28
- This repo contains GGUF format model files for [Qwen/Qwen2.5-Math-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Math-7B-Instruct).
29
 
30
  The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
31
 
32
-
33
  <div style="text-align: left; margin: 20px 0;">
34
  <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
35
  Run them on the TensorBlock client using your local machine ↗
@@ -38,7 +36,6 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
38
 
39
  ## Prompt template
40
 
41
-
42
  ```
43
  <|im_start|>system
44
  {system_prompt}<|im_end|>
@@ -51,18 +48,18 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
51
 
52
  | Filename | Quant type | File Size | Description |
53
  | -------- | ---------- | --------- | ----------- |
54
- | [Qwen2.5-Math-7B-Instruct-Q2_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q2_K.gguf) | Q2_K | 2.809 GB | smallest, significant quality loss - not recommended for most purposes |
55
- | [Qwen2.5-Math-7B-Instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q3_K_S.gguf) | Q3_K_S | 3.253 GB | very small, high quality loss |
56
- | [Qwen2.5-Math-7B-Instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q3_K_M.gguf) | Q3_K_M | 3.547 GB | very small, high quality loss |
57
- | [Qwen2.5-Math-7B-Instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q3_K_L.gguf) | Q3_K_L | 3.808 GB | small, substantial quality loss |
58
- | [Qwen2.5-Math-7B-Instruct-Q4_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q4_0.gguf) | Q4_0 | 4.127 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
59
- | [Qwen2.5-Math-7B-Instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q4_K_S.gguf) | Q4_K_S | 4.152 GB | small, greater quality loss |
60
- | [Qwen2.5-Math-7B-Instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q4_K_M.gguf) | Q4_K_M | 4.361 GB | medium, balanced quality - recommended |
61
- | [Qwen2.5-Math-7B-Instruct-Q5_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q5_0.gguf) | Q5_0 | 4.950 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
62
- | [Qwen2.5-Math-7B-Instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q5_K_S.gguf) | Q5_K_S | 4.950 GB | large, low quality loss - recommended |
63
- | [Qwen2.5-Math-7B-Instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q5_K_M.gguf) | Q5_K_M | 5.071 GB | large, very low quality loss - recommended |
64
- | [Qwen2.5-Math-7B-Instruct-Q6_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q6_K.gguf) | Q6_K | 5.825 GB | very large, extremely low quality loss |
65
- | [Qwen2.5-Math-7B-Instruct-Q8_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q8_0.gguf) | Q8_0 | 7.542 GB | very large, extremely low quality loss - not recommended |
66
 
67
 
68
  ## Downloading instruction
 
1
  ---
2
+ base_model: unsloth/Qwen2.5-Math-7B-Instruct
3
  language:
4
  - en
5
+ library_name: transformers
6
+ license: apache-2.0
7
  tags:
8
+ - unsloth
9
+ - transformers
10
  - TensorBlock
11
  - GGUF
 
 
 
12
  ---
13
 
14
  <div style="width: auto; margin-left: auto; margin-right: auto">
 
22
  </div>
23
  </div>
24
 
25
+ ## unsloth/Qwen2.5-Math-7B-Instruct - GGUF
26
 
27
+ This repo contains GGUF format model files for [unsloth/Qwen2.5-Math-7B-Instruct](https://huggingface.co/unsloth/Qwen2.5-Math-7B-Instruct).
28
 
29
  The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
30
 
 
31
  <div style="text-align: left; margin: 20px 0;">
32
  <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
33
  Run them on the TensorBlock client using your local machine ↗
 
36
 
37
  ## Prompt template
38
 
 
39
  ```
40
  <|im_start|>system
41
  {system_prompt}<|im_end|>
 
48
 
49
  | Filename | Quant type | File Size | Description |
50
  | -------- | ---------- | --------- | ----------- |
51
+ | [Qwen2.5-Math-7B-Instruct-Q2_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q2_K.gguf) | Q2_K | 3.016 GB | smallest, significant quality loss - not recommended for most purposes |
52
+ | [Qwen2.5-Math-7B-Instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q3_K_S.gguf) | Q3_K_S | 3.492 GB | very small, high quality loss |
53
+ | [Qwen2.5-Math-7B-Instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q3_K_M.gguf) | Q3_K_M | 3.808 GB | very small, high quality loss |
54
+ | [Qwen2.5-Math-7B-Instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q3_K_L.gguf) | Q3_K_L | 4.088 GB | small, substantial quality loss |
55
+ | [Qwen2.5-Math-7B-Instruct-Q4_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q4_0.gguf) | Q4_0 | 4.431 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
56
+ | [Qwen2.5-Math-7B-Instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q4_K_S.gguf) | Q4_K_S | 4.458 GB | small, greater quality loss |
57
+ | [Qwen2.5-Math-7B-Instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q4_K_M.gguf) | Q4_K_M | 4.683 GB | medium, balanced quality - recommended |
58
+ | [Qwen2.5-Math-7B-Instruct-Q5_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q5_0.gguf) | Q5_0 | 5.315 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
59
+ | [Qwen2.5-Math-7B-Instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q5_K_S.gguf) | Q5_K_S | 5.315 GB | large, low quality loss - recommended |
60
+ | [Qwen2.5-Math-7B-Instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q5_K_M.gguf) | Q5_K_M | 5.445 GB | large, very low quality loss - recommended |
61
+ | [Qwen2.5-Math-7B-Instruct-Q6_K.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q6_K.gguf) | Q6_K | 6.254 GB | very large, extremely low quality loss |
62
+ | [Qwen2.5-Math-7B-Instruct-Q8_0.gguf](https://huggingface.co/tensorblock/Qwen2.5-Math-7B-Instruct-GGUF/blob/main/Qwen2.5-Math-7B-Instruct-Q8_0.gguf) | Q8_0 | 8.099 GB | very large, extremely low quality loss - not recommended |
63
 
64
 
65
  ## Downloading instruction