My First ever attempt at Model Merging. I have zero experience, like zero, none. I don't know how to code, or use linux. Getting the terminal to work on Runpod was a nightmare.
Experimental Model I posted. Usable in RPs but it is not the best, compared to top dog models like Mythomax.
Just proof that a clueless guy let me can work on models.
I'm still learning basics on fine-tuning and training models.
I am seriously reading up on various model merging methods, and what models are best to use. I'll leave this here as my first work.
Base Model: Llama-2-13B-fp16
---Merged with
----- chronos-13b-v2
----- Nous-Hermes-Llama2-13b
----- airoboros-l2-13b-gpt4-m2.0
----- OpenOrca-Platypus2-13B
----- Kimiko-13B
----- StableBeluga-13B
Rationale:
Chronos - Used Chronoboros L1 so I put them here
Nous-Hermes - First ever L2 Model I used (γ₯ α΄ _α΄)γ₯β‘
Airoboros - Used Chronoboros L1 so I put them here
OpenOrca-Platypus - platypus.
Kimiko - Cute sounding name
StableBeluga - favourite animal
Prompt Format? No clue. I've been using Alpaca and It's been working fine.
Script used to Merge here
Thank you for the easy to set up script, Chargoddard !
Command used below:
python ties_merge.py TheBloke/Llama-2-13B-fp16 ./Medusa-13b --merge elinas/chronos-13b-v2 --merge NousResearch/Nous-Hermes-Llama2-13b --merge jondurbin/airoboros-l2-13b-gpt4-m2.0 --merge Open-Orca/OpenOrca-Platypus2-13B --merge TheBloke/Kimiko-13B-fp16 --merge stabilityai/StableBeluga-13B --cuda
Do not let me cook!
Image Credit: https://www.pixiv.net/en/artworks/80169912
Artist: TEL-O
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 49.85 |
| ARC (25-shot) | 58.19 |
| HellaSwag (10-shot) | 81.35 |
| MMLU (5-shot) | 57.39 |
| TruthfulQA (0-shot) | 51.24 |
| Winogrande (5-shot) | 73.32 |
| GSM8K (5-shot) | 6.82 |
| DROP (3-shot) | 20.61 |
- Downloads last month
- 1,640