MarsupialAI commited on
Commit
cb875f2
1 Parent(s): 69d7466

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +37 -30
README.md CHANGED
@@ -1,42 +1,49 @@
1
  ---
2
- base_model: []
3
- library_name: transformers
 
4
  tags:
5
- - mergekit
6
- - merge
7
-
8
  ---
9
- # skunk14
 
 
 
 
10
 
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
 
12
 
13
- ## Merge Details
14
- ### Merge Method
 
 
15
 
16
- This model was merged using the [linear](https://arxiv.org/abs/2203.05482) merge method.
 
 
 
17
 
18
- ### Models Merged
19
 
20
- The following models were included in the merge:
21
- * f:\raw\moist14
22
- * f:\raw\solstice14
23
- * f:\raw\fimbul14
24
 
25
- ### Configuration
26
 
27
- The following YAML configuration was used to produce this model:
 
 
 
 
 
28
 
29
- ```yaml
30
- models:
31
- - model: f:\raw\fimbul14
32
- parameters:
33
- weight: 1.0
34
- - model: f:\raw\moist14
35
- parameters:
36
- weight: 1.0
37
- - model: f:\raw\solstice14
38
- parameters:
39
- weight: 1.0
40
- merge_method: linear
41
- dtype: float16
42
  ```
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: cc-by-nc-4.0
3
+ language:
4
+ - en
5
  tags:
6
+ - solar
 
 
7
  ---
8
+ # Skunk Ape 14b
9
+
10
+ ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/65a531bc7ec6af0f95c707b1/p9tbuezkb2qvf8kWEnO_2.jpeg)
11
+
12
+ This version performs *substantially* better than the 16b version.
13
 
14
+ This model is a merge of three self-merged Solar-based models in a 14b (64 layer) configuration. The result of
15
+ this "frankenmerge" is a medium-sized model that contains what I consider to be the best of the solar finetunes.
16
 
17
+ Mergefuel:
18
+ - Sao10K/Fimbulvetr-11B-v2
19
+ - Sao10K/Solstice-11B-v1
20
+ - TheDrummer/Moistral-11B-v1
21
 
22
+ This model is uncensored and capable of generating objectionable material. However, it is not an explicitely-NSFW model,
23
+ and it has never "gone rogue" and tried to insert NSFW content into SFW prompts in my experience. As with any LLM, no
24
+ factual claims made by the model should be taken at face value. You know that boilerplate safety disclaimer that most
25
+ professional models have? Assume this has it too. This model is for entertainment purposes only.
26
 
 
27
 
28
+ iMatrix GGUFs: https://huggingface.co/MarsupialAI/SkunkApe-14b_iMatrix_GGUF
 
 
 
29
 
 
30
 
31
+ # Sample output
32
+
33
+ ```
34
+ {{[INPUT]}}
35
+ Write a detailed and humorous story about a cute and fluffy bunny that goes to a Gwar concert.
36
+ {{[OUTPUT]}}
37
 
38
+ <<<This goes on for a while. See sample.txt for full output>>>
 
 
 
 
 
 
 
 
 
 
 
 
39
  ```
40
+
41
+
42
+ # Prompt format
43
+ Prefers alpaca.
44
+
45
+
46
+ # Weird merge fuckery
47
+ According to Toasty Pigeon, FM, Akai, and probably others on the KAI discord, this merge method works better than a normal stacked merge.
48
+ I don't pretend to understand why, but the huge PPL improvement (5.96 for this model vs 7.65 for the 16b @ Q4km) indicates that they're right.
49
+ See recipe.txt for all the alchemy.