File size: 1,885 Bytes
4e86d24 64a393f 4e86d24 64a393f 4e86d24 64a393f 16ba04f 4e86d24 d51c1a9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
---
base_model: []
library_name: transformers
tags:
- mergekit
- merge
license: apache-2.0
---
# Credit for the model card's description goes to ddh0, mergekit, and, MTSAIR
# multi_verse_model-10.7B
This is multi_verse_model-10.7B, a depth-upscaled version of [MTSAIR/multi_verse_model](https://huggingface.co/MTSAIR/multi_verse_model).
This model is intended to be used as a basis for further fine-tuning, or as a drop-in upgrade from the original 7 billion parameter model.
Paper detailing how Depth-Up Scaling works: [SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling](https://arxiv.org/abs/2312.15166)
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
## Merge Details
### Merge Method
This model was merged using the passthrough merge method.
### Models Merged
The following models were included in the merge:
* /Users/jsarnecki/opt/Workspace/MTSAIR/multi_verse_model
### Configuration
The following YAML configuration was used to produce this model:
```yaml
dtype: bfloat16
merge_method: passthrough
slices:
- sources:
- layer_range: [0, 24]
model: /Users/jsarnecki/opt/Workspace/MTSAIR/multi_verse_model
- sources:
- layer_range: [8, 32]
model: /Users/jsarnecki/opt/Workspace/MTSAIR/multi_verse_model
```
I'm an innovative concept, created through a cutting-edge training method. Picture me as a "learning bot" who's had a special upgrade. Just like how a chef perfects their recipes with new techniques, my creators have fine-tuned my "knowledge-absorption" process. I'm here to showcase the potential of this new approach, and I'm excited to test my abilities in a friendly, helpful manner. So, while I may be a product of experimentation, my purpose is to demonstrate the power of continuous learning and growth in the world of artificial intelligence. |