ludis commited on
Commit
a7da3c1
1 Parent(s): 5e611ab

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -13
README.md CHANGED
@@ -1,16 +1,11 @@
1
- ---
2
- datasets:
3
- - PygmalionAI/PIPPA
4
- - ludis/geepeetee4
5
- ---
6
-
7
  ## GGUF
8
 
9
  gguf quants for ludis/tsukasa-13b-qlora-limarp
10
 
 
11
  ## Prompting
12
 
13
- https://rentry.org/v43eo - reccomended prompts and gen settings
14
 
15
  The current model version has been trained on prompts using three different roles, which are denoted by the following tokens: `<|system|>`, `<|user|>` and `<|model|>`.
16
 
@@ -18,13 +13,12 @@ The `<|system|>` prompt can be used to inject out-of-channel information behind
18
 
19
  ## Training
20
 
21
- base model (llama-2-13b-hf)
22
-
23
- tuned on koishi dataset (commit c83d922) for 1 epoch
24
 
25
- then tuned on pippa dataset (commit 6412b0c) for 1 epoch
 
26
 
27
- then tuned on geepeetee4 dataset (commit c83d922) for 1 epoch
28
 
29
- then tuned on limarp (without ponyville, lolicit, and all the fallen subsets. Version 2023-09-14) for 2 epochs
30
 
 
 
 
 
 
 
 
1
  ## GGUF
2
 
3
  gguf quants for ludis/tsukasa-13b-qlora-limarp
4
 
5
+
6
  ## Prompting
7
 
8
+ https://rentry.org/tsukasa13b - reccomended prompts and gen settings
9
 
10
  The current model version has been trained on prompts using three different roles, which are denoted by the following tokens: `<|system|>`, `<|user|>` and `<|model|>`.
11
 
 
13
 
14
  ## Training
15
 
16
+ base model (mistral-0.1-7b)
 
 
17
 
18
+ [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) was used for training
19
+ on a 4x nvidia a40 gpu cluster.
20
 
21
+ the a40 GPU cluster has been graciously provided by [Arc Compute](https://www.arccompute.io/).
22
 
23
+ rank 8 lora tune of mistralai/Mistral-7B-v0.1, first tuned on koishi commit 6e675d1 for one epoch then on limarp (without ponyville, lolicit, all the fallen, and eka's portal subsets) Version 2023-09-30 for 2 epochs in metharme format
24