ChuckMcSneed commited on
Commit
17d638c
1 Parent(s): 781a558

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -6,6 +6,8 @@ tags:
6
  ---
7
  # BETTER THAN GOLIATH?!
8
  I've merged [Euryale-lora that I made](https://huggingface.co/ChuckMcSneed/Euryale-1.3-L2-70B-LORA) with [Xwin](https://huggingface.co/Xwin-LM/Xwin-LM-70B-V0.1) and then merged it with itself in [goliath-style merge](/config.yml) using [mergekit](https://github.com/arcee-ai/mergekit). The resulting model performs better than [goliath](https://huggingface.co/alpindale/goliath-120b) on my tests(note: performance on tests is not necessarily performance in practice). Test it, have fun with it. This is a sister model of [Premerge-EX-EX-123B](https://huggingface.co/ChuckMcSneed/Premerge-EX-EX-123B).
 
 
9
  # Ideas behind it
10
  Since the creation of Goliath I was wondering if it was possible to make something even better. I've tried linear, passthrough, SLERP, TIES-merging models, but I could not recreate the greatness of goliath, at least not in a way that I liked in practical use. I knew about the existence of LORAs but I didn't know how well they performed. I created a model named [Gembo](https://huggingface.co/ChuckMcSneed/Gembo-v1-70b) by merging a shitton of LORAs together, and surprisingly it worked! In fact it worked so well that it was the best model on my benchmarks until now. When I found a tool named [LORD](https://github.com/thomasgauthier/LoRD), which can extract LORA from any model, I knew I could do something even better.
11
 
 
6
  ---
7
  # BETTER THAN GOLIATH?!
8
  I've merged [Euryale-lora that I made](https://huggingface.co/ChuckMcSneed/Euryale-1.3-L2-70B-LORA) with [Xwin](https://huggingface.co/Xwin-LM/Xwin-LM-70B-V0.1) and then merged it with itself in [goliath-style merge](/config.yml) using [mergekit](https://github.com/arcee-ai/mergekit). The resulting model performs better than [goliath](https://huggingface.co/alpindale/goliath-120b) on my tests(note: performance on tests is not necessarily performance in practice). Test it, have fun with it. This is a sister model of [Premerge-EX-EX-123B](https://huggingface.co/ChuckMcSneed/Premerge-EX-EX-123B).
9
+ # Prompt format
10
+ Alpaca.
11
  # Ideas behind it
12
  Since the creation of Goliath I was wondering if it was possible to make something even better. I've tried linear, passthrough, SLERP, TIES-merging models, but I could not recreate the greatness of goliath, at least not in a way that I liked in practical use. I knew about the existence of LORAs but I didn't know how well they performed. I created a model named [Gembo](https://huggingface.co/ChuckMcSneed/Gembo-v1-70b) by merging a shitton of LORAs together, and surprisingly it worked! In fact it worked so well that it was the best model on my benchmarks until now. When I found a tool named [LORD](https://github.com/thomasgauthier/LoRD), which can extract LORA from any model, I knew I could do something even better.
13