Hugging Face's logo

elinas
/
Meta-Llama-3-120B-Instruct-4.0bpw-exl2

Text Generation
Transformers
Safetensors
llama
Merge
mergekit
lazymergekit
autoquant
exl2
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
1
elinas commited on
Commit
21ab5c0
•
1 Parent(s): fd4a8ba

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +28 -2
  2. cal_data.safetensors +3 -0
  3. hidden_states.safetensors +3 -0
  4. job_new.json +0 -0
  5. measurement.json +0 -0
  6. out_tensor/lm_head.safetensors +3 -0
  7. out_tensor/model.layers.0.mlp.down_proj.safetensors +3 -0
  8. out_tensor/model.layers.0.mlp.gate_proj.safetensors +3 -0
  9. out_tensor/model.layers.0.mlp.up_proj.safetensors +3 -0
  10. out_tensor/model.layers.0.self_attn.k_proj.safetensors +3 -0
  11. out_tensor/model.layers.0.self_attn.o_proj.safetensors +3 -0
  12. out_tensor/model.layers.0.self_attn.q_proj.safetensors +3 -0
  13. out_tensor/model.layers.0.self_attn.v_proj.safetensors +3 -0
  14. out_tensor/model.layers.1.mlp.down_proj.safetensors +3 -0
  15. out_tensor/model.layers.1.mlp.gate_proj.safetensors +3 -0
  16. out_tensor/model.layers.1.mlp.up_proj.safetensors +3 -0
  17. out_tensor/model.layers.1.self_attn.k_proj.safetensors +3 -0
  18. out_tensor/model.layers.1.self_attn.o_proj.safetensors +3 -0
  19. out_tensor/model.layers.1.self_attn.q_proj.safetensors +3 -0
  20. out_tensor/model.layers.1.self_attn.v_proj.safetensors +3 -0
  21. out_tensor/model.layers.10.mlp.down_proj.safetensors +3 -0
  22. out_tensor/model.layers.10.mlp.gate_proj.safetensors +3 -0
  23. out_tensor/model.layers.10.mlp.up_proj.safetensors +3 -0
  24. out_tensor/model.layers.10.self_attn.k_proj.safetensors +3 -0
  25. out_tensor/model.layers.10.self_attn.o_proj.safetensors +3 -0
  26. out_tensor/model.layers.10.self_attn.q_proj.safetensors +3 -0
  27. out_tensor/model.layers.10.self_attn.v_proj.safetensors +3 -0
  28. out_tensor/model.layers.100.mlp.down_proj.safetensors +3 -0
  29. out_tensor/model.layers.100.mlp.gate_proj.safetensors +3 -0
  30. out_tensor/model.layers.100.mlp.up_proj.safetensors +3 -0
  31. out_tensor/model.layers.100.self_attn.k_proj.safetensors +3 -0
  32. out_tensor/model.layers.100.self_attn.o_proj.safetensors +3 -0
  33. out_tensor/model.layers.100.self_attn.q_proj.safetensors +3 -0
  34. out_tensor/model.layers.100.self_attn.v_proj.safetensors +3 -0
  35. out_tensor/model.layers.101.mlp.down_proj.safetensors +3 -0
  36. out_tensor/model.layers.101.mlp.gate_proj.safetensors +3 -0
  37. out_tensor/model.layers.101.mlp.up_proj.safetensors +3 -0
  38. out_tensor/model.layers.101.self_attn.k_proj.safetensors +3 -0
  39. out_tensor/model.layers.101.self_attn.o_proj.safetensors +3 -0
  40. out_tensor/model.layers.101.self_attn.q_proj.safetensors +3 -0
  41. out_tensor/model.layers.101.self_attn.v_proj.safetensors +3 -0
  42. out_tensor/model.layers.102.mlp.down_proj.safetensors +3 -0
  43. out_tensor/model.layers.102.mlp.gate_proj.safetensors +3 -0
  44. out_tensor/model.layers.102.mlp.up_proj.safetensors +3 -0
  45. out_tensor/model.layers.102.self_attn.k_proj.safetensors +3 -0
  46. out_tensor/model.layers.102.self_attn.o_proj.safetensors +3 -0
  47. out_tensor/model.layers.102.self_attn.q_proj.safetensors +3 -0
  48. out_tensor/model.layers.102.self_attn.v_proj.safetensors +3 -0
  49. out_tensor/model.layers.103.mlp.down_proj.safetensors +3 -0
  50. out_tensor/model.layers.103.mlp.gate_proj.safetensors +3 -0
README.md CHANGED
@@ -22,9 +22,35 @@ base_model:
22
 
23
  Meta-Llama-3-120B-Instruct is a self-merge with [meta-llama/Meta-Llama-3-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct).
24
 
25
- It was inspired by large merges like [alpindale/goliath-120b](https://huggingface.co/alpindale/goliath-120b), [nsfwthrowitaway69/Venus-120b-v1.0](https://huggingface.co/nsfwthrowitaway69/Venus-120b-v1.0), [cognitivecomputations/MegaDolphin-120b](https://huggingface.co/cognitivecomputations/MegaDolphin-120b), and [wolfram/miquliz-120b-v2.0](https://huggingface.co/wolfram/miquliz-120b-v2.0).
26
 
27
- No eval yet, but it is approved by Eric Hartford: https://twitter.com/erhartford/status/1787050962114207886
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
 
29
  ## 🧩 Configuration
30
 
 
22
 
23
  Meta-Llama-3-120B-Instruct is a self-merge with [meta-llama/Meta-Llama-3-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct).
24
 
25
+ It was inspired by large merges like:
26
 
27
+ - [alpindale/goliath-120b](https://huggingface.co/alpindale/goliath-120b)
28
+ - [nsfwthrowitaway69/Venus-120b-v1.0](https://huggingface.co/nsfwthrowitaway69/Venus-120b-v1.0)
29
+ - [cognitivecomputations/MegaDolphin-120b](https://huggingface.co/cognitivecomputations/MegaDolphin-120b)
30
+ - [wolfram/miquliz-120b-v2.0](https://huggingface.co/wolfram/miquliz-120b-v2.0).
31
+
32
+ ## 🔍 Applications
33
+
34
+ I recommend using this model for creative writing. It uses the Llama 3 chat template with a default context window of 8K (can be extended with rope theta).
35
+
36
+ Check the examples in the evaluation section to get an idea of its performance.
37
+
38
+ ## ⚡ Quantized models
39
+
40
+ Thanks to [Eric Hartford](https://huggingface.co/ehartford), [elinas](https://huggingface.co/elinas), and the [mlx-community](https://huggingface.co/mlx-community) for providing these models.
41
+
42
+ * **GGUF**: https://huggingface.co/cognitivecomputations/Meta-Llama-3-120B-Instruct-gguf
43
+ * **EXL2**: https://huggingface.co/elinas/Meta-Llama-3-120B-Instruct-4.0bpw-exl2
44
+ * **mlx**: https://huggingface.co/mlx-community/Meta-Llama-3-120B-Instruct-4bit
45
+
46
+ ## 🏆 Evaluation
47
+
48
+ The model looks excellent for creating writing tasks, outperforming GPT-4. Thanks again to [Eric Hartford](https://huggingface.co/ehartford) for noticing this.
49
+
50
+ * **X thread by Eric Hartford (creative writing)**: https://twitter.com/erhartford/status/1787050962114207886
51
+ * **X thread by Daniel Kaiser (creative writing)**: https://twitter.com/spectate_or/status/1787257261309518101
52
+ * **X thread by Simon (reasoning)**: https://twitter.com/NewDigitalEdu/status/1787403266894020893
53
+ * **r/LocalLLaMa**: https://www.reddit.com/r/LocalLLaMA/comments/1cl525q/goliath_lovers_where_is_the_feedback_about/
54
 
55
  ## 🧩 Configuration
56
 
cal_data.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3c2571ae462e0c551a79738031b2b44a7f7bb89b25b67534b452b272f7805390
3
+ size 1884248
hidden_states.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4d56960d819a3b32f27cf880b58b9691f49dcd84e73acb30878e0ca34f66951
3
+ size 3858769856
job_new.json ADDED
The diff for this file is too large to render. See raw diff
 
measurement.json ADDED
The diff for this file is too large to render. See raw diff
 
out_tensor/lm_head.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:40393095a7da94033eaa7d995aade2ee30bf1d4c9e7be6e9bf1ebf6b1781ec57
3
+ size 833184568
out_tensor/model.layers.0.mlp.down_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7829b10db8ebb528164d0b6ce2ee56297f3f09c2b39603d259acd1f370c8bb53
3
+ size 127129376
out_tensor/model.layers.0.mlp.gate_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:422d6fcfe7c926c9fe04b279e7efe3d8ba563b369b284f084db387c6d28fc74c
3
+ size 121145376
out_tensor/model.layers.0.mlp.up_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cd9a925e0577fbece4b308a4d2587e9cc28b02c238e08c0d2db2f61d8b241e95
3
+ size 121145368
out_tensor/model.layers.0.self_attn.k_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3f6cf7eb947000ffb5a91880f8c28a4497647f34190ddd3ebfbe7ad590e9e134
3
+ size 6457376
out_tensor/model.layers.0.self_attn.o_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:248e698f4eb4f3fbbdad0260fa2aef999ae6b420d24101add458485dfbb6a1ee
3
+ size 51415080
out_tensor/model.layers.0.self_attn.q_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b9b794f1fff7f6bc884963a96b328a7d23cfed6df566d4a64e133e30bee4383d
3
+ size 51415080
out_tensor/model.layers.0.self_attn.v_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:307768bb9e34716ad7aebb965e853a7a99c2adc10c39c97ae9182ee2a5c23ce5
3
+ size 8554528
out_tensor/model.layers.1.mlp.down_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:353a1bafcee1141ae8fc5e03c1e1a947f4f26ae3227ad2303e74337037e25604
3
+ size 130078496
out_tensor/model.layers.1.mlp.gate_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f9d74080740d7161dc11609302166329d6e1b17526a480bd5897b45159d98d4d
3
+ size 124127264
out_tensor/model.layers.1.mlp.up_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9998e9878aa5aad5bb31b9d063655b68b26e33a67e5845ad9545322266963cf4
3
+ size 128485400
out_tensor/model.layers.1.self_attn.k_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:91d2587d8c683e8efcb3ff14f14135d1e5af609fbe64a46bb5abf38eabfadd8f
3
+ size 6457376
out_tensor/model.layers.1.self_attn.o_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3248cdb78ad88fc861a90d610b7c47476dbf055f11903b5910011bdfa788bf14
3
+ size 51415080
out_tensor/model.layers.1.self_attn.q_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7ddea7e1c5b5bd066ae2cecc7b0642dbfd7b88026498d07b53dfc765326333e3
3
+ size 51415080
out_tensor/model.layers.1.self_attn.v_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d5feda799722261005cfccd4100032ef6705b03e96c4a31f9dac464dceba854
3
+ size 8554528
out_tensor/model.layers.10.mlp.down_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7b76b3d1ce67f69cfbccca107eec8c2058373666272235be1dbdc0e29a73b491
3
+ size 72564010
out_tensor/model.layers.10.mlp.gate_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:93cc7ede7263d610fd1da1ebd942591937994d0c4b44f6ccaf40ffe2129c2ad5
3
+ size 62194976
out_tensor/model.layers.10.mlp.up_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:51f5788d5c8365d99b4edc1bd0e4eb2589b31881df38638e98b9ca7f9d521d2d
3
+ size 62194968
out_tensor/model.layers.10.self_attn.k_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a91f446db4cd3a6ed5021205de054a1a89317e94e459437e579f9a20281299b5
3
+ size 2303264
out_tensor/model.layers.10.self_attn.o_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:80989e80e64bd23c6d89c383f0bf787d9d4e08d3df0f5848868e80f1d80ad5c3
3
+ size 26379176
out_tensor/model.layers.10.self_attn.q_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f3bae592a814b11b048c99c32d0d95b9a0a50dc512a8ea799094d0ac85f8010c
3
+ size 18187560
out_tensor/model.layers.10.self_attn.v_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:68a026cff4f58fb7f8d9cc66f333dcb549473c918c4b35686564a25c5db4f018
3
+ size 3326880
out_tensor/model.layers.100.mlp.down_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9f5779fc3ff69e30790a6ce800c3bf269479b5d675cc69967b125ae17d32bd00
3
+ size 124512308
out_tensor/model.layers.100.mlp.gate_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:318665906992be42aabb2a1f2c1b9e34f8d1e054c286538c6f97cc8fa17f1cd5
3
+ size 94767136
out_tensor/model.layers.100.mlp.up_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:81e18558c739511eb68d891cf8f599de9457add7392e651aca983dce6879a667
3
+ size 99125272
out_tensor/model.layers.100.self_attn.k_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3c4862592a91a5b4edad50e5fed35fec4f4b6829bd14f70da12fcc4b6cfdca9d
3
+ size 4375464
out_tensor/model.layers.100.self_attn.o_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:777342db63087464405e1dd2dc771be524169c57da3c65178e807b64164e846b
3
+ size 34767792
out_tensor/model.layers.100.self_attn.q_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:001372ae86626228f802e1f1a289d935a88c738b9728233448ee65305bb62e6d
3
+ size 34767792
out_tensor/model.layers.100.self_attn.v_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5f9ea26d79c2003d80615d904bc7a01729cce72235a656821842fae1374747db
3
+ size 4400424
out_tensor/model.layers.101.mlp.down_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:51c6a7b277d28bfddd65c90d3ab29f476b4117b329a3c43fabc94ca542e0aa44
3
+ size 127526964
out_tensor/model.layers.101.mlp.gate_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:666aab82c0b2a69f3734b5657ce2464edd101cdc08788b1db43dc730dce08f83
3
+ size 121602984
out_tensor/model.layers.101.mlp.up_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3b94e1643c294caac6ebfcc32aab6ef50709a253406a8dd8d2ff9c234b4942eb
3
+ size 125731736
out_tensor/model.layers.101.self_attn.k_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:80763a825ede7895a8ae38fe3c1b514c4dbda42a5d46f4906af4fdbaa98728c5
3
+ size 4375464
out_tensor/model.layers.101.self_attn.o_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4bea5c2c986046a4cd784b4676cb07bf0184723ae206fd4b16c6740352f6988c
3
+ size 34767792
out_tensor/model.layers.101.self_attn.q_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e8c5a5ab33fb9c300fd055662dad4d6606c99a0bce53da54a12d12c2aca658ef
3
+ size 34767792
out_tensor/model.layers.101.self_attn.v_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9bf4ce3ff88373b33682f13051d7ad6346c19414f8c8c4db7ac95ec4a7257204
3
+ size 4400424
out_tensor/model.layers.102.mlp.down_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fdca4cc420847d4a969d322a91adc464fccfcdf21ab7481318fa241e43f3ce72
3
+ size 127129392
out_tensor/model.layers.102.mlp.gate_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:78fb8df92be9d18135c34c7f23fd78cec1753f5762df1281937d0e034eb706d4
3
+ size 121145384
out_tensor/model.layers.102.mlp.up_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3f6fb620caac35a8c4849ff78e1d3a3395e52d629bd82d6cfb3bf4ea78dd7a35
3
+ size 121145376
out_tensor/model.layers.102.self_attn.k_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fca695c0f3911681018ff13c59d2b0ea583db1812336e664592fadec90bd2880
3
+ size 4466728
out_tensor/model.layers.102.self_attn.o_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b74a48b30887207be5da1337f2e1641bcca01e22eb3c37c6607f6088ab86c46e
3
+ size 35489840
out_tensor/model.layers.102.self_attn.q_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4eb2a1c0b3b8bef91500c70238709939b0a5e27fece8212cff3898aeb8a48308
3
+ size 35489840
out_tensor/model.layers.102.self_attn.v_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6dadc5bfd9b6e503dee3e258adc7107541d8a00fcc6de3f3022ce7326020cb82
3
+ size 5408808
out_tensor/model.layers.103.mlp.down_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f09001d648d65d98c97af9edce8fc54fe3220079bad8f0237eff8f1093dea48a
3
+ size 127129392
out_tensor/model.layers.103.mlp.gate_proj.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:928370c47cdedfcf2c316a885a12e50cb91ae7a12ca2437834041fe51cb14e2b
3
+ size 121145384