Delta-Vector commited on
Commit
cb114ba
1 Parent(s): c390ba6

Upload ./README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +69 -0
README.md ADDED
@@ -0,0 +1,69 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3
4
+ - crestf411/gemma2-9B-sunfall-v0.5.2
5
+ - allura-org/G2-9B-Sugarquill-v0
6
+ library_name: transformers
7
+ tags:
8
+ - mergekit
9
+ - merge
10
+ license: gemma
11
+ ---
12
+ ### exl2 quant (measurement.json in main branch)
13
+ ---
14
+ ### check revisions for quants
15
+ ---
16
+
17
+
18
+ <img src="inpaint.png">
19
+ <sub>Image by CalamitousFelicitousness</sub>
20
+
21
+ ---
22
+
23
+ # Gemma-2-9B Aletheia v1
24
+
25
+ A merge of Sugarquill and Sunfall. I wanted to combine Sugarquill's more novel-like writing style with something that would improve it's RP perfomance and make it more steerable, w/o adding superfluous synthetic writing patterns.
26
+
27
+ I quite like Crestfall's Sunfall models and I felt like Gemma version of Sunfall will steer the model in this direction when merged in. To keep more of Gemma-2-9B-it-SPPO-iter3's smarts, I've decided to apply Sunfall LoRA on top of it, instead of using the published Sunfall model.
28
+
29
+ I'm generally pleased with the result, this model has nice, fresh writing style, good charcard adherence and good system prompt following.
30
+ It still should work well for raw completion storywriting, as it's a trained feature in both merged models.
31
+
32
+ ---
33
+ Made by Auri.
34
+
35
+ Thanks to Prodeus, Inflatebot and ShotMisser for testing and giving feedback.
36
+
37
+ ### Format
38
+
39
+ Model responds to Gemma instruct formatting, exactly like it's base model.
40
+
41
+ ```
42
+ <bos><start_of_turn>user
43
+ {user message}<end_of_turn>
44
+ <start_of_turn>model
45
+ {response}<end_of_turn><eos>
46
+ ```
47
+
48
+
49
+ ### Mergekit config
50
+
51
+ The following YAML configuration was used to produce this model:
52
+
53
+ ```yaml
54
+ models:
55
+ - model: allura-org/G2-9B-Sugarquill-v0
56
+ parameters:
57
+ weight: 0.55
58
+ density: 0.4
59
+ - model: UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3+AuriAetherwiing/sunfall-g2-lora
60
+ parameters:
61
+ weight: 0.45
62
+ density: 0.3
63
+ merge_method: ties
64
+ base_model: UCLA-AGI/Gemma-2-9B-It-SPPO-Iter3
65
+ parameters:
66
+ normalize: true
67
+ dtype: bfloat16
68
+
69
+ ```