Undi95 commited on
Commit
11d82d5
1 Parent(s): e4b3dcf

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +60 -0
README.md ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ ---
4
+ Re:MythoMax (ReMM) is a recreation trial of the original [MythoMax-L2-B13](https://huggingface.co/Gryphe/MythoMax-L2-13b) with updated models.
5
+
6
+ This merge use SLERP [TESTING] to merge ReML and Huginn v1.2.
7
+
8
+ Command useds and explaination :
9
+ ```shell
10
+ Due to hardware limitation, some merge was done in 2 part.
11
+
12
+ - Recreate ReML : Mythologic (v2) (Chronos/Hermes/Airboros)
13
+ => Replacing Chronos by The-Face-Of-Goonery/Chronos-Beluga-v2-13bfp16 (0.30)
14
+ => Replacing Airoboros by jondurbin/airoboros-l2-13b-2.1 (last version) (0.40)
15
+ => Keeping NousResearch/Nous-Hermes-Llama2-13b (0.30)
16
+
17
+ Part 1: python ties_merge.py TheBloke/Llama-2-13B-fp16 ./ReML-L2-13B-part1 --merge The-Face-Of-Goonery/Chronos-Beluga-v2-13bfp16 --density 0.42 --merge jondurbin/airoboros-l2-13b-2.1 --density 0.56 --cuda
18
+
19
+ Part 2: python ties_merge.py TheBloke/Llama-2-13B-fp16 ./ReML-L2-13B --merge NousResearch/Nous-Hermes-Llama2-13b --density 0.30 --merge Undi95/ReML-L2-13B-part1 --density 0.70 --cuda
20
+
21
+ With that :
22
+ - Recreate ReMM : MythoMax (v2) (Mythologic/Huginn v1)
23
+ => Replacing Mythologic by the one above (0.5)
24
+ => Replacing Huginn by The-Face-Of-Goonery/Huginn-13b-v1.2 (hottest) (0.5)
25
+
26
+ Part 3: python slerpmergelm.py "The-Face-Of-Goonery_Huginn-13b-v1.2" "Undi95_ReML-L2-13B" "result"
27
+ ```
28
+
29
+ Version of SLERP used is different to accept usage on notebook : https://github.com/Undi95/LLM-SLERP-MergeTest/tree/main (Thanks @Vali)
30
+
31
+ <!-- description start -->
32
+ ## Description
33
+
34
+ This repo contains quantize files (Q4_K_S and Q5_K_M) of ReMM-SLERP, a recreation of the original MythoMax, but updated and merged with SLERP.
35
+ <!-- description end -->
36
+ <!-- description start -->
37
+ ## Models used
38
+
39
+
40
+ - TheBloke/Llama-2-13B-fp16 (base)
41
+ - The-Face-Of-Goonery/Chronos-Beluga-v2-13bfp16
42
+ - jondurbin/airoboros-l2-13b-2.1
43
+ - NousResearch/Nous-Hermes-Llama2-13b
44
+ - The-Face-Of-Goonery/Huginn-13b-v1.2
45
+ - ReML-L2-13B (Private recreation trial of an updated Mythologic-L2-13B)
46
+ <!-- description end -->
47
+ <!-- prompt-template start -->
48
+ ## Prompt template: Alpaca
49
+
50
+ ```
51
+ Below is an instruction that describes a task. Write a response that appropriately completes the request.
52
+
53
+ ### Instruction:
54
+ {prompt}
55
+
56
+ ### Response:
57
+
58
+ ```
59
+
60
+ Special thanks to Sushi kek