Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,69 @@
+---
+base_model: []
+library_name: transformers
+tags:
+- mergekit
+- merge
+
+---
+# Cerebrum-1.0-12.25B
+
+This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
+
+## Merge Details
+### Merge Method
+
+This model was merged using the passthrough merge method.
+
+### Models Merged
+
+The following models were included in the merge:
+* /Users/jsarnecki/opt/Workspace/AetherResearch/Cerebrum-1.0-7b
+
+### Configuration
+
+The following YAML configuration was used to produce this model:
+
+```yaml
+dtype: bfloat16
+merge_method: passthrough
+# Layer-Interleaving of AetherResearch/Cerebrum-1.0-7b
+# where n = 32 (The number of layers the model originally had)
+#       m = 8 (The number of layers to interleave per row)
+#       M = 24 = 3m (The number of layers interleaved into the model)
+#       s = 56 = 32 + 24 = n + M (The number of layers the model will have after the layer interleaving)
+#       P = 12.25B parameters = (s/n)*7 (The number of parameters the model will have after the layer interleaving)
+slices:
+- sources:
+  - layer_range: [0, 8]
+    model: /Users/jsarnecki/opt/Workspace/AetherResearch/Cerebrum-1.0-7b
+
+# Interleaved Layer
+- sources:
+  - layer_range: [4, 12]
+    model: /Users/jsarnecki/opt/Workspace/AetherResearch/Cerebrum-1.0-7b
+
+- sources:
+  - layer_range: [8, 16]
+    model: /Users/jsarnecki/opt/Workspace/AetherResearch/Cerebrum-1.0-7b
+
+# Interleaved Layer
+- sources:
+  - layer_range: [12, 20]
+    model: /Users/jsarnecki/opt/Workspace/AetherResearch/Cerebrum-1.0-7b 
+
+- sources:
+  - layer_range: [16, 24]
+    model: /Users/jsarnecki/opt/Workspace/AetherResearch/Cerebrum-1.0-7b
+
+# Interleaved Layer
+- sources:
+  - layer_range: [20, 28]
+    model: /Users/jsarnecki/opt/Workspace/AetherResearch/Cerebrum-1.0-7b 
+
+- sources:
+  - layer_range: [24, 32]
+    model: /Users/jsarnecki/opt/Workspace/AetherResearch/Cerebrum-1.0-7b
+
+
+```