sophosympatheia commited on
Commit
9804d59
1 Parent(s): 6179695

Update README.md

Browse files

Updates based on more testing at 16K context

Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -22,9 +22,9 @@ This model was designed for roleplaying and storytelling and I think it does wel
22
 
23
  ### Long Context Tips
24
 
25
- You can run this model past 4096 context with alpha_rope set to 1, but I think it performs better if you set alpha_rope to what you would normally use for a Llama2 model with 4096 context. For example, alpha_rope 2.5 for 8K.
26
- Miqu can go up to 32K context in theory. I would expect performance to degrade as you exceed 8K, which is typical for Llama2 models, but the dropoff may not be as extreme with this merge thanks to Miqu.
27
- **UPDATE:** I was able to test my 5.0 bpw exl2 quant of this model out to 16K context just now using 8-bit cache with alpha_rope 1 and it was okay!
28
 
29
  ### Sampler Tips
30
 
@@ -47,7 +47,7 @@ If you save the below settings as a .json file, you can import them directly int
47
  "epsilon_cutoff": 0,
48
  "eta_cutoff": 0,
49
  "typical_p": 1,
50
- "min_p": 0.15,
51
  "rep_pen": 1.05,
52
  "rep_pen_range": 2800,
53
  "no_repeat_ngram_size": 0,
@@ -64,7 +64,7 @@ If you save the below settings as a .json file, you can import them directly int
64
  "min_temp": 0.8,
65
  "max_temp": 1.35,
66
  "dynatemp_exponent": 1,
67
- "smoothing_factor": 0.4,
68
  "add_bos_token": true,
69
  "truncation_length": 2048,
70
  "ban_eos_token": false,
@@ -92,7 +92,7 @@ If you save the below settings as a .json file, you can import them directly int
92
  "n": 1,
93
  "rep_pen_size": 0,
94
  "genamt": 500,
95
- "max_length": 8192
96
  }
97
  ```
98
 
 
22
 
23
  ### Long Context Tips
24
 
25
+ You can run this model past 4096 context with alpha_rope set to 1.
26
+ I have tested my 5.0bpw exl2 quant of this model out to 16K context using 8-bit cache with alpha_rope 1 and it performs great without any noticable drop in quality as the context size filled from < 4K to the full 16K context.
27
+ Miqu can go up to 32K context, so in theory this merge can too. I will test that theory soon.
28
 
29
  ### Sampler Tips
30
 
 
47
  "epsilon_cutoff": 0,
48
  "eta_cutoff": 0,
49
  "typical_p": 1,
50
+ "min_p": 0.2,
51
  "rep_pen": 1.05,
52
  "rep_pen_range": 2800,
53
  "no_repeat_ngram_size": 0,
 
64
  "min_temp": 0.8,
65
  "max_temp": 1.35,
66
  "dynatemp_exponent": 1,
67
+ "smoothing_factor": 0.35,
68
  "add_bos_token": true,
69
  "truncation_length": 2048,
70
  "ban_eos_token": false,
 
92
  "n": 1,
93
  "rep_pen_size": 0,
94
  "genamt": 500,
95
+ "max_length": 16128
96
  }
97
  ```
98