DavidAU commited on
Commit
59b1520
·
verified ·
1 Parent(s): 099c02d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +57 -11
README.md CHANGED
@@ -36,7 +36,7 @@ tags:
36
 
37
  <h3>Maximizing Model Performance for All Quants Types And Full-Precision using Samplers, Advance Samplers and Parameters Guide</h3>
38
 
39
- (Updated: "INDEX", and added "Generation Steering" section)
40
 
41
  This document includes detailed information, references, and notes for general parameters, samplers and
42
  advanced samplers to get the most out of your model's abilities including notes / settings for the most popular AI/LLM app in use (LLAMACPP, KoboldCPP, Text-Generation-WebUI, LMStudio, Sillytavern, Ollama and others).
@@ -124,7 +124,8 @@ SOURCE FILES for my Models / APPS to Run LLMs / AIs:
124
  - TEXT-GENERATION-WEBUI
125
  - KOBOLDCPP
126
  - SILLYTAVERN
127
- - Lmstudio, Ollama, Llamacpp, and OTHER PROGRAMS
 
128
 
129
  TESTING / Default / Generation Example PARAMETERS AND SAMPLERS
130
  - Basic settings suggested for general model operation.
@@ -438,7 +439,7 @@ In section 1 a,b, and c, below are all the LLAMA_CPP parameters and samplers.
438
 
439
  I have added notes below each one for adjustment / enhancement(s) for specific use cases.
440
 
441
- TEXT-GENERATION-WEBUI
442
 
443
  In section 2, will be additional samplers, which become available when using "llamacpp_HF" loader in https://github.com/oobabooga/text-generation-webui
444
  AND/OR https://github.com/LostRuins/koboldcpp ("KOBOLDCPP").
@@ -449,7 +450,7 @@ The "llamacpp_HF" (for "text-generation-webui") only requires the GGUF you want
449
 
450
  This allows access to very advanced samplers in addition to all the parameters / samplers here.
451
 
452
- KOBOLDCPP:
453
 
454
  Note that https://github.com/LostRuins/koboldcpp also allows access to all LLAMACPP parameters/samplers too as well as additional advanced samplers too.
455
 
@@ -457,7 +458,7 @@ You can use almost all parameters, samplers and advanced samplers using "KOBOLDC
457
 
458
  Note: This program has one of the newest samplers called "Anti-slop" which allows phrase/word banning at the generation level.
459
 
460
- SILLYTAVERN:
461
 
462
  Note that https://github.com/SillyTavern/SillyTavern also allows access to all LLAMACPP parameters/samplers too as well as additional advanced samplers too.
463
 
@@ -480,16 +481,12 @@ Currently, at time of this writing, connecting Silly Tavern via KoboldCPP or Tex
480
 
481
  However for some, connecting to Lmstudio, LlamaCPP, or Ollama may be preferred.
482
 
483
- NOTE:
484
-
485
- It appears that Silly Tavern also supports "DRY" and "XTC" too ; but it is not yet in the documentation at the time of writing.
486
-
487
  You may also want to check out how to connect SillyTavern to local AI "apps" running on your pc here:
488
 
489
  https://docs.sillytavern.app/usage/api-connections/
490
 
491
 
492
- OTHER PROGRAMS:
493
 
494
  Other programs like https://www.LMStudio.ai allows access to most of STANDARD samplers, where as others (llamacpp only here) you may need to add to the json file(s) for a model and/or template preset.
495
 
@@ -515,6 +512,29 @@ Most AI/LLM apps operate on Windows, Mac, and Linux.
515
 
516
  Mobile devices (and O/S) are in many cases also supported.
517
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
518
 
519
  ---
520
 
@@ -552,6 +572,12 @@ You should set these as noted first. I would say these are the minimum settings
552
 
553
  Note for Class 3/Class 4 models settings/samplers (discussed below) "repeat-last-n" is a CRITICAL setting.
554
 
 
 
 
 
 
 
555
 
556
  ---
557
 
@@ -799,12 +825,22 @@ Likewise there may be some "name variation(s)" - in other LLM/AI apps - this is
799
 
800
  </small>
801
 
802
- CLASS 3/4 Models:
803
 
804
  If you are using a class 3 or class 4 model for use case(s) such as role play, multi-turn, chat etc etc, it is suggested to activate / set all samplers for class 3 but may be required for class 4 models.
805
 
806
  Likewise for fine control of a class 3/4 via "DRY" and "Quadratic" samplers is detailed below. These allow you to dial up or dial down the model's raw power directly.
807
 
 
 
 
 
 
 
 
 
 
 
808
  MICROSTAT Sampler - IMPORTANT:
809
 
810
  Make sure to review MIROSTAT sampler settings below, due to behaviour of this specific sampler / affect on parameters/other samplers which varies from app to app too.
@@ -855,6 +891,10 @@ Too much temp can affect instruction following in some cases and sometimes not e
855
 
856
  Newer model archs (L3,L3.1,L3.2, Mistral Nemo, Gemma2 etc) many times NEED more temp (1+) to get their best generations.
857
 
 
 
 
 
858
  <B>top-p</B>
859
 
860
  top-p sampling (default: 0.9, 1.0 = disabled)
@@ -937,6 +977,12 @@ This setting also works in conjunction with all other "rep pens" below.
937
 
938
  This parameter is the "RANGE" of tokens looked at for the samplers directly below.
939
 
 
 
 
 
 
 
940
  <B>SECONDARIES:</B>
941
 
942
  <B>repeat-penalty</B>
 
36
 
37
  <h3>Maximizing Model Performance for All Quants Types And Full-Precision using Samplers, Advance Samplers and Parameters Guide</h3>
38
 
39
+ (Updated: "INDEX", and added "Generation Steering" section ; notes on Roleplay/Simulation added)
40
 
41
  This document includes detailed information, references, and notes for general parameters, samplers and
42
  advanced samplers to get the most out of your model's abilities including notes / settings for the most popular AI/LLM app in use (LLAMACPP, KoboldCPP, Text-Generation-WebUI, LMStudio, Sillytavern, Ollama and others).
 
124
  - TEXT-GENERATION-WEBUI
125
  - KOBOLDCPP
126
  - SILLYTAVERN
127
+ - Lmstudio, Ollama, Llamacpp, Backyard, and OTHER PROGRAMS
128
+ - Roleplay and Simulation Programs/Notes on models.
129
 
130
  TESTING / Default / Generation Example PARAMETERS AND SAMPLERS
131
  - Basic settings suggested for general model operation.
 
439
 
440
  I have added notes below each one for adjustment / enhancement(s) for specific use cases.
441
 
442
+ <B>TEXT-GENERATION-WEBUI</B>
443
 
444
  In section 2, will be additional samplers, which become available when using "llamacpp_HF" loader in https://github.com/oobabooga/text-generation-webui
445
  AND/OR https://github.com/LostRuins/koboldcpp ("KOBOLDCPP").
 
450
 
451
  This allows access to very advanced samplers in addition to all the parameters / samplers here.
452
 
453
+ <B>KOBOLDCPP:</B>
454
 
455
  Note that https://github.com/LostRuins/koboldcpp also allows access to all LLAMACPP parameters/samplers too as well as additional advanced samplers too.
456
 
 
458
 
459
  Note: This program has one of the newest samplers called "Anti-slop" which allows phrase/word banning at the generation level.
460
 
461
+ <B>SILLYTAVERN:</B>
462
 
463
  Note that https://github.com/SillyTavern/SillyTavern also allows access to all LLAMACPP parameters/samplers too as well as additional advanced samplers too.
464
 
 
481
 
482
  However for some, connecting to Lmstudio, LlamaCPP, or Ollama may be preferred.
483
 
 
 
 
 
484
  You may also want to check out how to connect SillyTavern to local AI "apps" running on your pc here:
485
 
486
  https://docs.sillytavern.app/usage/api-connections/
487
 
488
 
489
+ <B>Lmstudio, Ollama, Llamacpp, and OTHER PROGRAMS</B>
490
 
491
  Other programs like https://www.LMStudio.ai allows access to most of STANDARD samplers, where as others (llamacpp only here) you may need to add to the json file(s) for a model and/or template preset.
492
 
 
512
 
513
  Mobile devices (and O/S) are in many cases also supported.
514
 
515
+ <B>Roleplay and Simulation Programs/Notes on models.</B>
516
+
517
+ Text Generation Webui, KoboldCPP, and Silly Tavern (and AI/LLM apps connected via Silly Tavern) can all do roleplay / simulation AS WELL as "chat" and other creative activities.
518
+
519
+ LMStudio (the app here directly), Ollama and other LLM/AI apps are for general usage, however they can be connected to Silly Tavern via API too.
520
+
521
+ Backyard ( https://backyard.ai/ ) is software that is dedicated primarily to Roleplay / Simulation, however it can not be (at time of this writing) connected via API to Silly Tavern at this time.
522
+
523
+ If you are using Backyard app, see special notes for "roleplay / simulation" and where applicable, "BACKYARD APP" for specific notes on using these app.
524
+
525
+ Models that are Class 3/4 :
526
+
527
+ Some of my models that are rated Class 3 or 4 maybe a little more challenging to operate with roleplay, especially if you can not access / control certain samplers.
528
+
529
+ How to handle this issue is addressed in "Generational Steering" section (you control it) as well as Quick Reference, and Detailed Parameters, Samplers and Advanced Samplers Sections (automated control).
530
+
531
+ Also, some of my models are available in multiple "classes", IE Dark Planet, and Grand Gutenberg.
532
+
533
+ In these cases, Dark Planet 8B versions and Grand Gutenberg 12B ("Darkness" / "Madness") are class 1 - any use case, including role play and simulation.
534
+
535
+ Likewise Darkest Planet 16.5B and Grand Gutenberg 23/23.5B are class 3 - great at roleplay/simulation, but need a bit more steering and/or parameter/samplers adjustments to work flawlessly for this use case.
536
+
537
+ Note: Dark Planet 8B (class 1) is also a compressed version of Grand Horror 16B (a full on class 4)
538
 
539
  ---
540
 
 
572
 
573
  Note for Class 3/Class 4 models settings/samplers (discussed below) "repeat-last-n" is a CRITICAL setting.
574
 
575
+ BACKYARD APP:
576
+
577
+ In "Backyard" app, "repetition_penalty_range" is called "Repeat Penalty Tokens" (set on the "character card").
578
+
579
+ For class 3/4 models (if using with Backyard app), set this to 64 OR LESS.
580
+
581
 
582
  ---
583
 
 
825
 
826
  </small>
827
 
828
+ CLASS 3/4 MODELS:
829
 
830
  If you are using a class 3 or class 4 model for use case(s) such as role play, multi-turn, chat etc etc, it is suggested to activate / set all samplers for class 3 but may be required for class 4 models.
831
 
832
  Likewise for fine control of a class 3/4 via "DRY" and "Quadratic" samplers is detailed below. These allow you to dial up or dial down the model's raw power directly.
833
 
834
+ ROLEPLAY / SIMULATION NOTES:
835
+
836
+ If you are using a model (regardless of "class") for these uses cases, you may need to LOWER "temp" to get better instruction following.
837
+
838
+ Instruction following issues can cascade over the "adventure" if the temp is set too high for the specific model(s) you are using.
839
+
840
+ Likewise you may want to set MAXIMUM output tokens (a hard limit how much the model can output) to much lower values such as 128 to 300.
841
+
842
+ (This will assist with steering, and stop the model from endlessly "yapping")
843
+
844
  MICROSTAT Sampler - IMPORTANT:
845
 
846
  Make sure to review MIROSTAT sampler settings below, due to behaviour of this specific sampler / affect on parameters/other samplers which varies from app to app too.
 
891
 
892
  Newer model archs (L3,L3.1,L3.2, Mistral Nemo, Gemma2 etc) many times NEED more temp (1+) to get their best generations.
893
 
894
+ ROLEPLAY / SIMULATION NOTE:
895
+
896
+ If you are using a model (regardless of "class") for these uses cases, you may need to LOWER temp to get better instruction following.
897
+
898
  <B>top-p</B>
899
 
900
  top-p sampling (default: 0.9, 1.0 = disabled)
 
977
 
978
  This parameter is the "RANGE" of tokens looked at for the samplers directly below.
979
 
980
+ BACKYARD APP:
981
+
982
+ In "Backyard" app, "repetition_penalty_range" is called "Repeat Penalty Tokens" (set on the "character card").
983
+
984
+ For class 3/4 models (if using with Backyard app), set this to 64 OR LESS.
985
+
986
  <B>SECONDARIES:</B>
987
 
988
  <B>repeat-penalty</B>