Undi95 commited on
Commit
a318754
1 Parent(s): fa8ed18

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +61 -1
README.md CHANGED
@@ -1,4 +1,64 @@
1
  ---
2
  license: other
3
  ---
4
- Helo lol
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: other
3
  ---
4
+ ReMM-Lion is a merge of ReMM (SLERP) with Pygmalion-2.
5
+
6
+ This merge use SLERP method to merge Re:MM and Pygmalion-2.
7
+
8
+ Command useds and explaination :
9
+ ```shell
10
+ Due to hardware limitation, some merge was done in 2 part.
11
+
12
+ - Recreate ReML : Mythologic (v2) (Chronos/Hermes/Airboros)
13
+ => Replacing Chronos by The-Face-Of-Goonery/Chronos-Beluga-v2-13bfp16 (0.30)
14
+ => Replacing Airoboros by jondurbin/airoboros-l2-13b-2.1 (last version) (0.40)
15
+ => Keeping NousResearch/Nous-Hermes-Llama2-13b (0.30)
16
+
17
+ Part 1: python ties_merge.py TheBloke/Llama-2-13B-fp16 ./ReML-L2-13B-part1 --merge The-Face-Of-Goonery/Chronos-Beluga-v2-13bfp16 --density 0.42 --merge jondurbin/airoboros-l2-13b-2.1 --density 0.56 --cuda
18
+
19
+ Part 2: python ties_merge.py TheBloke/Llama-2-13B-fp16 ./ReML-L2-13B --merge NousResearch/Nous-Hermes-Llama2-13b --density 0.30 --merge Undi95/ReML-L2-13B-part1 --density 0.70 --cuda
20
+
21
+ With that :
22
+ - Recreate ReMM : MythoMax (v2) (Mythologic/Huginn v1)
23
+ => Replacing Mythologic by the one above (0.5)
24
+ => Replacing Huginn by The-Face-Of-Goonery/Huginn-13b-v1.2 (hottest) (0.5)
25
+
26
+ Part 3: python slerpmergelm.py "The-Face-Of-Goonery_Huginn-13b-v1.2" "Undi95_ReML-L2-13B" "result"
27
+
28
+ With that :
29
+ - ReMM-Lion-13B
30
+
31
+ Part 4: python slerpmergelm.py "The-Face-Of-Goonery_Huginn-13b-v1.2" "Undi95_ReMM-S-Light" "result"
32
+ ```
33
+
34
+ <!-- description start -->
35
+ ## Description
36
+
37
+ This repo contains fp16 files of ReMM-Lion-13B.
38
+ <!-- description end -->
39
+ <!-- description start -->
40
+ ## Models used
41
+
42
+
43
+ - TheBloke/Llama-2-13B-fp16 (base)
44
+ - The-Face-Of-Goonery/Chronos-Beluga-v2-13bfp16
45
+ - jondurbin/airoboros-l2-13b-2.1
46
+ - NousResearch/Nous-Hermes-Llama2-13b
47
+ - The-Face-Of-Goonery/Huginn-13b-v1.2
48
+ - ReML-L2-13B (Private recreation trial of an updated Mythologic-L2-13B)
49
+ - PygmalionAI/pygmalion-2-13b
50
+ <!-- description end -->
51
+ <!-- prompt-template start -->
52
+ ## Prompt template: Alpaca
53
+
54
+ ```
55
+ Below is an instruction that describes a task. Write a response that appropriately completes the request.
56
+
57
+ ### Instruction:
58
+ {prompt}
59
+
60
+ ### Response:
61
+
62
+ ```
63
+
64
+ Special thanks to Sushi kek