Hugging Face's logo

schnapper79
/
lumikabra-195B_v0.3

Text Generation
Transformers
Safetensors
mistral
mergekit
lumikabra-195B
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
1
schnapper79 commited on
Commit
97fd7da
•
1 Parent(s): 04a1c3f

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +38 -0
  2. config.json +27 -0
  3. mergekit_config.yml +9 -0
  4. model-00001-of-00081.safetensors +3 -0
  5. model-00002-of-00081.safetensors +3 -0
  6. model-00003-of-00081.safetensors +3 -0
  7. model-00004-of-00081.safetensors +3 -0
  8. model-00005-of-00081.safetensors +3 -0
  9. model-00006-of-00081.safetensors +3 -0
  10. model-00007-of-00081.safetensors +3 -0
  11. model-00008-of-00081.safetensors +3 -0
  12. model-00009-of-00081.safetensors +3 -0
  13. model-00010-of-00081.safetensors +3 -0
  14. model-00011-of-00081.safetensors +3 -0
  15. model-00012-of-00081.safetensors +3 -0
  16. model-00013-of-00081.safetensors +3 -0
  17. model-00014-of-00081.safetensors +3 -0
  18. model-00015-of-00081.safetensors +3 -0
  19. model-00016-of-00081.safetensors +3 -0
  20. model-00017-of-00081.safetensors +3 -0
  21. model-00018-of-00081.safetensors +3 -0
  22. model-00019-of-00081.safetensors +3 -0
  23. model-00020-of-00081.safetensors +3 -0
  24. model-00021-of-00081.safetensors +3 -0
  25. model-00022-of-00081.safetensors +3 -0
  26. model-00023-of-00081.safetensors +3 -0
  27. model-00024-of-00081.safetensors +3 -0
  28. model-00025-of-00081.safetensors +3 -0
  29. model-00026-of-00081.safetensors +3 -0
  30. model-00027-of-00081.safetensors +3 -0
  31. model-00028-of-00081.safetensors +3 -0
  32. model-00029-of-00081.safetensors +3 -0
  33. model-00030-of-00081.safetensors +3 -0
  34. model-00031-of-00081.safetensors +3 -0
  35. model-00032-of-00081.safetensors +3 -0
  36. model-00033-of-00081.safetensors +3 -0
  37. model-00034-of-00081.safetensors +3 -0
  38. model-00035-of-00081.safetensors +3 -0
  39. model-00036-of-00081.safetensors +3 -0
  40. model-00037-of-00081.safetensors +3 -0
  41. model-00038-of-00081.safetensors +3 -0
  42. model-00039-of-00081.safetensors +3 -0
  43. model-00040-of-00081.safetensors +3 -0
  44. model-00041-of-00081.safetensors +3 -0
  45. model-00042-of-00081.safetensors +3 -0
  46. model-00043-of-00081.safetensors +3 -0
  47. model-00044-of-00081.safetensors +3 -0
  48. model-00045-of-00081.safetensors +3 -0
  49. model-00046-of-00081.safetensors +3 -0
  50. model-00047-of-00081.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: []
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+
8
+ ---
9
+ # merge_big
10
+
11
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
+
13
+ ## Merge Details
14
+ ### Merge Method
15
+
16
+ This model was merged using the passthrough merge method.
17
+
18
+ ### Models Merged
19
+
20
+ The following models were included in the merge:
21
+ * /workspace/text-generation-webui/models/schnapper79_lumikabra-123B_v0.3
22
+
23
+ ### Configuration
24
+
25
+ The following YAML configuration was used to produce this model:
26
+
27
+ ```yaml
28
+ dtype: bfloat16
29
+ merge_method: passthrough
30
+ slices:
31
+ - sources:
32
+ - layer_range: [0, 70]
33
+ model: /workspace/text-generation-webui/models/schnapper79_lumikabra-123B_v0.3
34
+ - sources:
35
+ - layer_range: [18, 88]
36
+ model: /workspace/text-generation-webui/models/schnapper79_lumikabra-123B_v0.3
37
+
38
+ ```
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "/workspace/text-generation-webui/models/schnapper79_lumikabra-123B_v0.3",
3
+ "architectures": [
4
+ "MistralForCausalLM"
5
+ ],
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 1,
8
+ "eos_token_id": 2,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 12288,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 28672,
14
+ "max_position_embeddings": 131072,
15
+ "model_type": "mistral",
16
+ "num_attention_heads": 96,
17
+ "num_hidden_layers": 140,
18
+ "num_key_value_heads": 8,
19
+ "rms_norm_eps": 1e-05,
20
+ "rope_theta": 1000000.0,
21
+ "sliding_window": null,
22
+ "tie_word_embeddings": false,
23
+ "torch_dtype": "bfloat16",
24
+ "transformers_version": "4.44.2",
25
+ "use_cache": true,
26
+ "vocab_size": 32768
27
+ }
mergekit_config.yml ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ dtype: bfloat16
2
+ merge_method: passthrough
3
+ slices:
4
+ - sources:
5
+ - layer_range: [0, 70]
6
+ model: /workspace/text-generation-webui/models/schnapper79_lumikabra-123B_v0.3
7
+ - sources:
8
+ - layer_range: [18, 88]
9
+ model: /workspace/text-generation-webui/models/schnapper79_lumikabra-123B_v0.3
model-00001-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c15fbee97fd6a1053a0242de4b69fcd18d83486aaefb9f9b206e2e66cd7e5132
3
+ size 4378928504
model-00002-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95a1be3519207a56f51deb85a788013c6081794d2af03b58a60c96bfe93735e5
3
+ size 4907411088
model-00003-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:561bef5b2fdd3d814639c55b761d2f5d6f248acbf06279820b79295d13e6f7e1
3
+ size 4806747904
model-00004-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f3aea6320a7254f1e10b0b1f68d20429a0bb7068882d1e90de125192762d3546
3
+ size 4831938544
model-00005-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:58324e621dec2ae32a2d9cc18d91f29e044d5dc258c9e7f572bab47f64dd6aed
3
+ size 4831938552
model-00006-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8b8320b63a92bba5db3d3855737f5f8eda778d79763328c00012b3fcbac26d85
3
+ size 4907411096
model-00007-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7f2d1437d858deab0d4c4b09d7da4f0d775895a8ab94426b7fd54a8cd3206272
3
+ size 4907435776
model-00008-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:11e2a84dfd1558d3013b5dc1e92762a9bf49c7f037623d667d21992776f0c386
3
+ size 4781557264
model-00009-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fe2aff6a7f55d8e6efe2ad234e08c00075584e7d53d94f3e63e8edf391c27068
3
+ size 4781656640
model-00010-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d9e2590625d59c96761a878209845df1ac33a698817f888e652bbde5833d9187
3
+ size 4882220464
model-00011-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:04194ea711c660b39a8e86f4210e02d64000d9939565f24c63f3ae955819d090
3
+ size 4932601720
model-00012-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dbd864178cc810586ca8516bc5aaf685a77b6899657beae53ba6c928182ed9d6
3
+ size 4781557264
model-00013-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9bf05727f939f688a3f03620d1de4b0246037d133f640a9ec2b613de7214e68a
3
+ size 4831938552
model-00014-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4ce8e23daa2867e180aae75943b94140e41b4f140b83fedd7ca2fdabfcd42922
3
+ size 4831938544
model-00015-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1bbf4bc7e0b3e1eb9414da7ee011df36ee3462de37e227349c8df89727483e4b
3
+ size 4831938552
model-00016-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b481de6a6c8370473a07f3e1b657ac5f0e2f95834d7d8ca3b1d940853c2ee25b
3
+ size 4831938552
model-00017-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f9bd016258d6bfaeeb8fde6700c904f61821ea44814bcacd59813760a6131ab
3
+ size 4831938552
model-00018-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0ec431f39a57ebd6bb2f98a43638032a45809309add095e14f6cc3f43292611
3
+ size 4882220464
model-00019-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4eeb36daf9f56fe5a38536bf17286bb0ee44c8c4aa375cd14dce7bcf66df70ef
3
+ size 4932601720
model-00020-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:128f37bd2d41272d08f407cff0ecf45f93dc58cc4f406bf13e2619c8ade32225
3
+ size 4781557264
model-00021-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:24ab9a97b9cc6b5db2602520be1964e98d662bb83efdc3af5eb473caf1922c71
3
+ size 4781656640
model-00022-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d412855e58ffd78b4fee38c26fac566c338cf4ce66e986de04466cd182ba053a
3
+ size 4882220464
model-00023-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a4b6fe40eb9f9f3b81cc6b8a48e055df5d9995cabbc2105e13147b967e848286
3
+ size 4932601720
model-00024-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3ea0a66ed5efdee19d42cee91e761eb68a7a2ffd39a094ae77e95078c232ce89
3
+ size 4781557264
model-00025-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b1fb4a7c270a439922a7b3725af2c1bc37c7878195120cbef36d41f3893ec2bd
3
+ size 4831938552
model-00026-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:609fed1eeb8c15924df486a6f86e1cbc01e062cc95b6629f816706dc1c801120
3
+ size 4831938544
model-00027-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b607a12d403e3cc756c33d1cf0bd19bcb0158727bb1e345b2bdb42580eceee4
3
+ size 4831938552
model-00028-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8c0cc0237b7188ad5cfc4e886c30a8e5b003fba7dd322e7c2ca94a7ceecb6cd9
3
+ size 4831938552
model-00029-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6aca173fa142c66d1b2d53a67f0f9592f57e5e4d20686830df18490d8b5eef8d
3
+ size 4831938552
model-00030-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:51e99ffc49849d4c8aaaef2b12cc79c90dae07657f6ea000090a406885c07453
3
+ size 4882220464
model-00031-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7c40e13fe6bfa8dd9715c9e9738bde8fe4acc732105fc5b4d4ad89a9b53611c6
3
+ size 4932601720
model-00032-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:887267c30cbc0ad6d1f940d939c4010c23916947b7c28ff3454908cdfd0eb610
3
+ size 4781557264
model-00033-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:27d8eace77c44a0ac431e63d6ad8220030db62425b4ba677a7c55410b636ebd2
3
+ size 4781656640
model-00034-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:926bddeb967097ba17ce1dd3199d87bae513fc1b4e6b9497663d497aed0b0663
3
+ size 4882220464
model-00035-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c5a370038f0757fe1bdb6e597fb1232e46829c85327623cf3a63b9dd78cf73f0
3
+ size 4932601720
model-00036-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:268c98cd3354c47c9caa18813970de716f72d0be659616c2035234533d28cffe
3
+ size 4781557264
model-00037-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1a5fd7edcdd1e019c3cbedbf71e4ad78f07c53011c4adc75f786083430ad9f88
3
+ size 4831938552
model-00038-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f6538a58c41391235c08e5123fa455a3a4db41497df69f2df6ecdb4641528db7
3
+ size 4831938544
model-00039-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3d17b68ed9eaa95164e1d22f1c5cefe986a78ae1b32babff3c65c74edf1215cc
3
+ size 4831938552
model-00040-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9bf2943738d9d92a9a8c76f8bddd97436918b5346ee06948f752dbb125abe418
3
+ size 4831938552
model-00041-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f894c8cbfab158faf9f803fa40b08cbc40659c337405ed2d37ee0257bf2609fc
3
+ size 4831938552
model-00042-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5dac82a3b3ce1a7c80d4726661d67b43216fe70391020ae1c3bf4e5b7aae2e5c
3
+ size 4882220464
model-00043-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:136709b6bb88090a1dd5dc5553a51991dff456ae10db92f33f15e8396b2aade1
3
+ size 4932601720
model-00044-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:72c4f9bdc7dce3fdba731752f8f7b74e605b9537bec2affc51e30f2fae016e89
3
+ size 4781557272
model-00045-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4ce0060670267e88d731250d5d9c0d5c8373c0c70519f2025999f9fe461ea302
3
+ size 4781656632
model-00046-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8026a822cb5e5219de14baa25f36d292f94775c05b390e44b8fbb9c2d88a2ce6
3
+ size 4882220464
model-00047-of-00081.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:381707441d26467c6434cf53ace2dc52f6b23b318b23116bda297ae11ec281b0
3
+ size 4932601728