Update README.md
Browse files
README.md
CHANGED
@@ -5,11 +5,17 @@ base_model:
|
|
5 |
tags:
|
6 |
- mergekit
|
7 |
- merge
|
8 |
-
|
|
|
9 |
---
|
10 |
-
#
|
|
|
|
|
|
|
|
|
|
|
11 |
|
12 |
-
This
|
13 |
|
14 |
## Merge Details
|
15 |
### Merge Method
|
@@ -54,3 +60,20 @@ merge_method: passthrough
|
|
54 |
dtype: float16
|
55 |
|
56 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
5 |
tags:
|
6 |
- mergekit
|
7 |
- merge
|
8 |
+
- solar
|
9 |
+
license: apache-2.0
|
10 |
---
|
11 |
+
# Nous-Hermes-2-SOLAR-18B
|
12 |
+
|
13 |
+
This is a SOLAR-like model upscaled to 18B.
|
14 |
+
It is a frankenmerge model created using mergekit, alternating layers of Nous-Hermes-2-SOLAR-10.7B and SOLAR-10.7B-Instruct.
|
15 |
+
|
16 |
+
Evaluations coming soon!
|
17 |
|
18 |
+
This model has very good writing capabilities (compared to SOLAR-10.7B), specially for role-playing.
|
19 |
|
20 |
## Merge Details
|
21 |
### Merge Method
|
|
|
60 |
dtype: float16
|
61 |
|
62 |
```
|
63 |
+
|
64 |
+
|
65 |
+
### Usage
|
66 |
+
|
67 |
+
You can use the provided template:
|
68 |
+
|
69 |
+
```
|
70 |
+
tokenizer = AutoTokenizer.from_pretrained("vicgalle/Nous-Hermes-2-SOLAR-18B")
|
71 |
+
model = AutoModelForCausalLM.from_pretrained("vicgalle/Nous-Hermes-2-SOLAR-18B", torch_dtype=torch.float16, load_in_4bit=True)
|
72 |
+
|
73 |
+
conversation = [ {'role': 'system', 'content': SYSTEM_PROMPT}, {'role': 'user', 'content': USER_PROMPT} ]
|
74 |
+
prompt = tokenizer.apply_chat_template(conversation, tokenize=False, add_generation_prompt=True)
|
75 |
+
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
76 |
+
|
77 |
+
outputs = model.generate(**inputs, use_cache=True, max_new_tokens=1024, do_sample=True, temperature=0.8)
|
78 |
+
output_text = tokenizer.decode(outputs[0])
|
79 |
+
```
|