Hugging Face's logo

DavidAU
/
L3-MOE-4X8B-Grand-Horror-25B-GGUF

Text Generation
Transformers
GGUF
English
mergekit
Mixture of Experts
mixture of experts
Merge
4x8B
Llama3 MOE
creative
creative writing
fiction writing
plot generation
sub-plot generation
story generation
scene continue
storytelling
fiction story
science fiction
romance
all genres
story
writing
vivid prosing
vivid writing
fiction
roleplaying
bfloat16
swearing
rp
horror
Inference Endpoints
conversational
Model card Files Community
DavidAU mradermacher commited on
Commit
27ecbd5
•
0 Parent(s):

Duplicate from mradermacher/L3-MOE-4X8B-Grand-Horror-25B-GGUF

Browse files

Co-authored-by: team mradermacher <mradermacher@users.noreply.huggingface.co>

Files changed (13) hide show
  1. .gitattributes +46 -0
  2. L3-MOE-4X8B-Grand-Horror-25B.IQ4_XS.gguf +3 -0
  3. L3-MOE-4X8B-Grand-Horror-25B.Q2_K.gguf +3 -0
  4. L3-MOE-4X8B-Grand-Horror-25B.Q3_K_L.gguf +3 -0
  5. L3-MOE-4X8B-Grand-Horror-25B.Q3_K_M.gguf +3 -0
  6. L3-MOE-4X8B-Grand-Horror-25B.Q3_K_S.gguf +3 -0
  7. L3-MOE-4X8B-Grand-Horror-25B.Q4_K_M.gguf +3 -0
  8. L3-MOE-4X8B-Grand-Horror-25B.Q4_K_S.gguf +3 -0
  9. L3-MOE-4X8B-Grand-Horror-25B.Q5_K_M.gguf +3 -0
  10. L3-MOE-4X8B-Grand-Horror-25B.Q5_K_S.gguf +3 -0
  11. L3-MOE-4X8B-Grand-Horror-25B.Q6_K.gguf +3 -0
  12. L3-MOE-4X8B-Grand-Horror-25B.Q8_0.gguf +3 -0
  13. README.md +67 -0
.gitattributes ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ L3-MOE-4X8B-Grand-Horror-25B.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
37
+ L3-MOE-4X8B-Grand-Horror-25B.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
38
+ L3-MOE-4X8B-Grand-Horror-25B.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
39
+ L3-MOE-4X8B-Grand-Horror-25B.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
40
+ L3-MOE-4X8B-Grand-Horror-25B.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
41
+ L3-MOE-4X8B-Grand-Horror-25B.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
42
+ L3-MOE-4X8B-Grand-Horror-25B.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
43
+ L3-MOE-4X8B-Grand-Horror-25B.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
44
+ L3-MOE-4X8B-Grand-Horror-25B.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
45
+ L3-MOE-4X8B-Grand-Horror-25B.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ L3-MOE-4X8B-Grand-Horror-25B.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
L3-MOE-4X8B-Grand-Horror-25B.IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:70a82f605ccf57b2afa258c54a1bfa296c0933f8b701eec2b9062e321f2cf2a8
3
+ size 13580763648
L3-MOE-4X8B-Grand-Horror-25B.Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b56700dd7385cac997f3e81ea84dc38de3b3510047bc42a5cd8a2dd0fa0873de
3
+ size 9302819328
L3-MOE-4X8B-Grand-Horror-25B.Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17e64de4c3db05564b87817e02d6a5e44fe71acfa8fb6ce3e038fa62924174ab
3
+ size 13044015616
L3-MOE-4X8B-Grand-Horror-25B.Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:164b5b50bc66bb795f3ac49199a6ed603e40075784136dd0edf2c692fe85f419
3
+ size 12080374272
L3-MOE-4X8B-Grand-Horror-25B.Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4efd7751f1c298def07f7320f09ddc478cbff8c157fef866304a3823855a6e86
3
+ size 10933232128
L3-MOE-4X8B-Grand-Horror-25B.Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f6ff7ed66446647cc95305e15df405f07aeb78fa2bd46944f3244f00b88d8a74
3
+ size 15162180096
L3-MOE-4X8B-Grand-Horror-25B.Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4ce3ff94aff97940b9f6485febbc3c07e2ccff04b85e6fef79e16bf9346d674
3
+ size 14295532032
L3-MOE-4X8B-Grand-Horror-25B.Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b1f62e4b57654352127b7637acf5544bb6932899b30f51dde367ba3c9024cba7
3
+ size 17736040960
L3-MOE-4X8B-Grand-Horror-25B.Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1cc7706ea9e2a177a8c0c2ae0fa62d16c30ccb5bdc636a461d82e1fbc23c1fe4
3
+ size 17228005888
L3-MOE-4X8B-Grand-Horror-25B.Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b33bf7e6fae707d746cec74dc2eac9d9b057ee2d7ed31b337c5694d6a04ba7ba
3
+ size 20470768128
L3-MOE-4X8B-Grand-Horror-25B.Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9784529c0f25da65885dbda183ab955502d671bd9816f77b06a64aa8755201b3
3
+ size 26511270400
README.md ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: DavidAU/L3-MOE-4X8B-Grand-Horror-25B
3
+ language:
4
+ - en
5
+ library_name: transformers
6
+ quantized_by: mradermacher
7
+ tags:
8
+ - mergekit
9
+ - moe
10
+ - mixture of experts
11
+ - merge
12
+ ---
13
+ ## About
14
+
15
+ <!-- ### quantize_version: 2 -->
16
+ <!-- ### output_tensor_quantised: 1 -->
17
+ <!-- ### convert_type: hf -->
18
+ <!-- ### vocab_type: -->
19
+ <!-- ### tags: -->
20
+ static quants of https://huggingface.co/DavidAU/L3-MOE-4X8B-Grand-Horror-25B
21
+
22
+ <!-- provided-files -->
23
+ weighted/imatrix quants seem not to be available (by me) at this time. If they do not show up a week or so after the static ones, I have probably not planned for them. Feel free to request them by opening a Community Discussion.
24
+ ## Usage
25
+
26
+ If you are unsure how to use GGUF files, refer to one of [TheBloke's
27
+ READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
28
+ more details, including on how to concatenate multi-part files.
29
+
30
+ ## Provided Quants
31
+
32
+ (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
33
+
34
+ | Link | Type | Size/GB | Notes |
35
+ |:-----|:-----|--------:|:------|
36
+ | [GGUF](https://huggingface.co/mradermacher/L3-MOE-4X8B-Grand-Horror-25B-GGUF/resolve/main/L3-MOE-4X8B-Grand-Horror-25B.Q2_K.gguf) | Q2_K | 9.4 | |
37
+ | [GGUF](https://huggingface.co/mradermacher/L3-MOE-4X8B-Grand-Horror-25B-GGUF/resolve/main/L3-MOE-4X8B-Grand-Horror-25B.Q3_K_S.gguf) | Q3_K_S | 11.0 | |
38
+ | [GGUF](https://huggingface.co/mradermacher/L3-MOE-4X8B-Grand-Horror-25B-GGUF/resolve/main/L3-MOE-4X8B-Grand-Horror-25B.Q3_K_M.gguf) | Q3_K_M | 12.2 | lower quality |
39
+ | [GGUF](https://huggingface.co/mradermacher/L3-MOE-4X8B-Grand-Horror-25B-GGUF/resolve/main/L3-MOE-4X8B-Grand-Horror-25B.Q3_K_L.gguf) | Q3_K_L | 13.1 | |
40
+ | [GGUF](https://huggingface.co/mradermacher/L3-MOE-4X8B-Grand-Horror-25B-GGUF/resolve/main/L3-MOE-4X8B-Grand-Horror-25B.IQ4_XS.gguf) | IQ4_XS | 13.7 | |
41
+ | [GGUF](https://huggingface.co/mradermacher/L3-MOE-4X8B-Grand-Horror-25B-GGUF/resolve/main/L3-MOE-4X8B-Grand-Horror-25B.Q4_K_S.gguf) | Q4_K_S | 14.4 | fast, recommended |
42
+ | [GGUF](https://huggingface.co/mradermacher/L3-MOE-4X8B-Grand-Horror-25B-GGUF/resolve/main/L3-MOE-4X8B-Grand-Horror-25B.Q4_K_M.gguf) | Q4_K_M | 15.3 | fast, recommended |
43
+ | [GGUF](https://huggingface.co/mradermacher/L3-MOE-4X8B-Grand-Horror-25B-GGUF/resolve/main/L3-MOE-4X8B-Grand-Horror-25B.Q5_K_S.gguf) | Q5_K_S | 17.3 | |
44
+ | [GGUF](https://huggingface.co/mradermacher/L3-MOE-4X8B-Grand-Horror-25B-GGUF/resolve/main/L3-MOE-4X8B-Grand-Horror-25B.Q5_K_M.gguf) | Q5_K_M | 17.8 | |
45
+ | [GGUF](https://huggingface.co/mradermacher/L3-MOE-4X8B-Grand-Horror-25B-GGUF/resolve/main/L3-MOE-4X8B-Grand-Horror-25B.Q6_K.gguf) | Q6_K | 20.6 | very good quality |
46
+ | [GGUF](https://huggingface.co/mradermacher/L3-MOE-4X8B-Grand-Horror-25B-GGUF/resolve/main/L3-MOE-4X8B-Grand-Horror-25B.Q8_0.gguf) | Q8_0 | 26.6 | fast, best quality |
47
+
48
+ Here is a handy graph by ikawrakow comparing some lower-quality quant
49
+ types (lower is better):
50
+
51
+ ![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
52
+
53
+ And here are Artefact2's thoughts on the matter:
54
+ https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
55
+
56
+ ## FAQ / Model Request
57
+
58
+ See https://huggingface.co/mradermacher/model_requests for some answers to
59
+ questions you might have and/or if you want some other model quantized.
60
+
61
+ ## Thanks
62
+
63
+ I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
64
+ me use its servers and providing upgrades to my workstation to enable
65
+ this work in my free time.
66
+
67
+ <!-- end -->