Hugging Face's logo

bartowski
/
Hyperion-3.0-Yi-34B-GGUF

Text Generation
Transformers
GGUF
English
code
chemistry
medical
Inference Endpoints
Model card Files Community
bartowski commited on
Commit
92bde18
•
1 Parent(s): 9f7ed86

Llamacpp quants

Browse files
Files changed (17) hide show
  1. .gitattributes +14 -0
  2. Hyperion-3.0-Yi-34B-Q2_K.gguf +3 -0
  3. Hyperion-3.0-Yi-34B-Q3_K_L.gguf +3 -0
  4. Hyperion-3.0-Yi-34B-Q3_K_M.gguf +3 -0
  5. Hyperion-3.0-Yi-34B-Q3_K_S.gguf +3 -0
  6. Hyperion-3.0-Yi-34B-Q4_0.gguf +3 -0
  7. Hyperion-3.0-Yi-34B-Q4_K_M.gguf +3 -0
  8. Hyperion-3.0-Yi-34B-Q4_K_S.gguf +3 -0
  9. Hyperion-3.0-Yi-34B-Q5_0.gguf +3 -0
  10. Hyperion-3.0-Yi-34B-Q5_K_M.gguf +3 -0
  11. Hyperion-3.0-Yi-34B-Q5_K_S.gguf +3 -0
  12. Hyperion-3.0-Yi-34B-Q6_K.gguf +3 -0
  13. Hyperion-3.0-Yi-34B-Q8_0.gguf +3 -0
  14. Hyperion-3.0-Yi-34B-fp16.gguf/Hyperion-3.0-Yi-34B-fp16_part_a +3 -0
  15. Hyperion-3.0-Yi-34B-fp16.gguf/Hyperion-3.0-Yi-34B-fp16_part_b +3 -0
  16. Hyperion-3.0-Yi-34B-fp16.gguf/combine.sh +2 -0
  17. README.md +39 -0
.gitattributes CHANGED
@@ -33,3 +33,17 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Hyperion-3.0-Yi-34B-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Hyperion-3.0-Yi-34B-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Hyperion-3.0-Yi-34B-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Hyperion-3.0-Yi-34B-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Hyperion-3.0-Yi-34B-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Hyperion-3.0-Yi-34B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Hyperion-3.0-Yi-34B-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Hyperion-3.0-Yi-34B-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Hyperion-3.0-Yi-34B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Hyperion-3.0-Yi-34B-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Hyperion-3.0-Yi-34B-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ Hyperion-3.0-Yi-34B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Hyperion-3.0-Yi-34B-fp16.gguf/Hyperion-3.0-Yi-34B-fp16_part_a filter=lfs diff=lfs merge=lfs -text
49
+ Hyperion-3.0-Yi-34B-fp16.gguf/Hyperion-3.0-Yi-34B-fp16_part_b filter=lfs diff=lfs merge=lfs -text
Hyperion-3.0-Yi-34B-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee7b75f87167a20d7f7ff075b2a89f981b2352a14259c4b34324e1018a75c1cf
3
+ size 12825233504
Hyperion-3.0-Yi-34B-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cc92e6c4529febe1f6a104a276d4cee9772bfbebb4c077043099b858e9f37f67
3
+ size 18139445344
Hyperion-3.0-Yi-34B-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95b35884a9951b1bbbc41acdb5057f91cad2a7a5620c5d1e719f73ac3d27c903
3
+ size 16654923872
Hyperion-3.0-Yi-34B-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cdc4252a1343b5534d6eca18c3b9821197de967242ff7a5baba92affdecc22e2
3
+ size 14960293984
Hyperion-3.0-Yi-34B-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:07aa1867a8615c13f9018c5848cf5d122e3c347f19fb9127d60ef6a620abad12
3
+ size 19466528864
Hyperion-3.0-Yi-34B-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:abb7cdc20c862451169e4259177f7764375c31322d8ba75dfe6fce7c34e22346
3
+ size 20658710624
Hyperion-3.0-Yi-34B-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:49ab759b0df22d7a45ad93938eda12b315208565b2eb7a0d0d17d1a5f2430947
3
+ size 19598649440
Hyperion-3.0-Yi-34B-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:50a3efe87a25267f08ad6888812525089bdcf0503a47c8f7912eb66a5485580d
3
+ size 23707691104
Hyperion-3.0-Yi-34B-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8117e60fd5cd9b2b350703599412f73203c4c27051d673b6c8bd96c2adc21cb
3
+ size 24321845344
Hyperion-3.0-Yi-34B-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c64c49a4c168d3d106df6bfbe71a80d861e62f5f6214f7d479a15e8b67e771f8
3
+ size 23707691104
Hyperion-3.0-Yi-34B-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d080a6ad5b36b456ab555c6d180a8d18dcc150bcfc9dfad57e5b70a3bc043f3a
3
+ size 28213925984
Hyperion-3.0-Yi-34B-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c5d4a42f97f963b7cccace43e6fa8095bfea7e4bf11ced3125cf0b5a1dc9a98
3
+ size 36542281824
Hyperion-3.0-Yi-34B-fp16.gguf/Hyperion-3.0-Yi-34B-fp16_part_a ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d10fd300a48f6e1d89b3b13667934fb27f88c4239f7a35ecd35c289c622dacd1
3
+ size 34390539281
Hyperion-3.0-Yi-34B-fp16.gguf/Hyperion-3.0-Yi-34B-fp16_part_b ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9bb72362d31b418911de0ebcd1d6cc61d6204d8db1650eb44c0ee135bf91c06f
3
+ size 34390539279
Hyperion-3.0-Yi-34B-fp16.gguf/combine.sh ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ #!/bin/bash
2
+ cat Hyperion-3.0-Yi-34B-fp16_part_* > "Hyperion-3.0-Yi-34B-fp16.gguf"
README.md ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: other
5
+ library_name: transformers
6
+ tags:
7
+ - code
8
+ - chemistry
9
+ - medical
10
+ datasets:
11
+ - Locutusque/hyperion-v3.0
12
+ quantized_by: bartowski
13
+ pipeline_tag: text-generation
14
+ ---
15
+
16
+ ## Llamacpp Quantizations of Hyperion-3.0-Yi-34B
17
+
18
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2440">b2440</a> for quantization.
19
+
20
+ Original model: https://huggingface.co/Locutusque/Hyperion-3.0-Yi-34B
21
+
22
+ Download a file (not the whole branch) from below:
23
+
24
+ | Filename | Quant type | File Size | Description |
25
+ | -------- | ---------- | --------- | ----------- |
26
+ | [Hyperion-3.0-Yi-34B-Q8_0.gguf](https://huggingface.co/bartowski/Hyperion-3.0-Yi-34B-GGUF/blob/main/Hyperion-3.0-Yi-34B-Q8_0.gguf) | Q8_0 | 36.54GB | Extremely high quality, generally unneeded but max available quant. |
27
+ | [Hyperion-3.0-Yi-34B-Q6_K.gguf](https://huggingface.co/bartowski/Hyperion-3.0-Yi-34B-GGUF/blob/main/Hyperion-3.0-Yi-34B-Q6_K.gguf) | Q6_K | 28.21GB | Very high quality, near perfect, *recommended*. |
28
+ | [Hyperion-3.0-Yi-34B-Q5_K_M.gguf](https://huggingface.co/bartowski/Hyperion-3.0-Yi-34B-GGUF/blob/main/Hyperion-3.0-Yi-34B-Q5_K_M.gguf) | Q5_K_M | 24.32GB | High quality, very usable. |
29
+ | [Hyperion-3.0-Yi-34B-Q5_K_S.gguf](https://huggingface.co/bartowski/Hyperion-3.0-Yi-34B-GGUF/blob/main/Hyperion-3.0-Yi-34B-Q5_K_S.gguf) | Q5_K_S | 23.70GB | High quality, very usable. |
30
+ | [Hyperion-3.0-Yi-34B-Q5_0.gguf](https://huggingface.co/bartowski/Hyperion-3.0-Yi-34B-GGUF/blob/main/Hyperion-3.0-Yi-34B-Q5_0.gguf) | Q5_0 | 23.70GB | High quality, older format, generally not recommended. |
31
+ | [Hyperion-3.0-Yi-34B-Q4_K_M.gguf](https://huggingface.co/bartowski/Hyperion-3.0-Yi-34B-GGUF/blob/main/Hyperion-3.0-Yi-34B-Q4_K_M.gguf) | Q4_K_M | 20.65GB | Good quality, similar to 4.25 bpw. |
32
+ | [Hyperion-3.0-Yi-34B-Q4_K_S.gguf](https://huggingface.co/bartowski/Hyperion-3.0-Yi-34B-GGUF/blob/main/Hyperion-3.0-Yi-34B-Q4_K_S.gguf) | Q4_K_S | 19.59GB | Slightly lower quality with small space savings. |
33
+ | [Hyperion-3.0-Yi-34B-Q4_0.gguf](https://huggingface.co/bartowski/Hyperion-3.0-Yi-34B-GGUF/blob/main/Hyperion-3.0-Yi-34B-Q4_0.gguf) | Q4_0 | 19.46GB | Decent quality, older format, generally not recommended. |
34
+ | [Hyperion-3.0-Yi-34B-Q3_K_L.gguf](https://huggingface.co/bartowski/Hyperion-3.0-Yi-34B-GGUF/blob/main/Hyperion-3.0-Yi-34B-Q3_K_L.gguf) | Q3_K_L | 18.13GB | Lower quality but usable, good for low RAM availability. |
35
+ | [Hyperion-3.0-Yi-34B-Q3_K_M.gguf](https://huggingface.co/bartowski/Hyperion-3.0-Yi-34B-GGUF/blob/main/Hyperion-3.0-Yi-34B-Q3_K_M.gguf) | Q3_K_M | 16.65GB | Even lower quality. |
36
+ | [Hyperion-3.0-Yi-34B-Q3_K_S.gguf](https://huggingface.co/bartowski/Hyperion-3.0-Yi-34B-GGUF/blob/main/Hyperion-3.0-Yi-34B-Q3_K_S.gguf) | Q3_K_S | 14.96GB | Low quality, not recommended. |
37
+ | [Hyperion-3.0-Yi-34B-Q2_K.gguf](https://huggingface.co/bartowski/Hyperion-3.0-Yi-34B-GGUF/blob/main/Hyperion-3.0-Yi-34B-Q2_K.gguf) | Q2_K | 12.82GB | Extremely low quality, *not* recommended.
38
+
39
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski