RossAscends commited on
Commit
865d43c
·
1 Parent(s): 129b6fd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -3
README.md CHANGED
@@ -5,11 +5,21 @@ language: en
5
  library_name: adapter-transformers
6
  pipeline_tag: text-generation
7
  ---
8
- This is a simple 0.5 merge of LIMA RP lora with ehartford's Mistral Dolphin 2.1
 
 
 
 
9
 
10
- exllama v2 at 4bpw
11
 
12
- Use SilyTavern's ChatML or Mistral instruct formats for best instruct-style results.
 
 
 
 
 
 
13
 
14
  full weights:
15
  https://huggingface.co/RossAscends/Mistral_7B_Dolphin2.1_LIMA0.5_fp16
 
5
  library_name: adapter-transformers
6
  pipeline_tag: text-generation
7
  ---
8
+ ehartford's merge of Mistral 7B 0.1 with his Dolphin 2.1 dataset
9
+ https://huggingface.co/ehartford/dolphin-2.1-mistral-7b
10
+ +
11
+ LIMA RP dataset applied as a lora at 0.5 weight
12
+ https://huggingface.co/lemonilia/limarp-llama2-v2/
13
 
14
+ Purpose of the model is to be RP-focused, smart, fast, and lightweight for users with low VRAM.
15
 
16
+ I've already built the exl2 4bpw quant (linked below), and it will run 8k ctx at around 6GB VRAM and respond to a full context at roughly 30tps (tested on my 3060) if exl2_hf loader is used with FA2 enabled.
17
+
18
+ Model has been tested by several users on the SillyTavern discord server, and run on Horde for a full day - with good results.
19
+
20
+ https://huggingface.co/RossAscends/Mistral7B_Dolphin2.1_LIMARP0.5_4bpw_exl2
21
+
22
+ Mistral or ChatML context presets both possible.
23
 
24
  full weights:
25
  https://huggingface.co/RossAscends/Mistral_7B_Dolphin2.1_LIMA0.5_fp16