Update README.md
Browse files
README.md
CHANGED
@@ -34,6 +34,13 @@ GGML files are for CPU + GPU inference using [llama.cpp](https://github.com/gger
|
|
34 |
* [4-bit, 5-bit, and 8-bit GGML models for CPU+GPU inference](https://huggingface.co/TheBloke/Karen_theEditor_13B-GGML)
|
35 |
* [Unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/FPHam/Karen_theEditor_13b_HF)
|
36 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
37 |
## THE FILES IN MAIN BRANCH REQUIRES LATEST LLAMA.CPP (May 19th 2023 - commit 2d5db48)!
|
38 |
|
39 |
llama.cpp recently made another breaking change to its quantisation methods - https://github.com/ggerganov/llama.cpp/pull/1508
|
@@ -57,7 +64,7 @@ I have quantised the GGML files in this repo with the latest version. Therefore
|
|
57 |
I use the following command line; adjust for your tastes and needs:
|
58 |
|
59 |
```
|
60 |
-
./main -t 10 -ngl 32 -m Karen-The-Editor.ggmlv3.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "
|
61 |
```
|
62 |
Change `-t 10` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
|
63 |
|
|
|
34 |
* [4-bit, 5-bit, and 8-bit GGML models for CPU+GPU inference](https://huggingface.co/TheBloke/Karen_theEditor_13B-GGML)
|
35 |
* [Unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/FPHam/Karen_theEditor_13b_HF)
|
36 |
|
37 |
+
## Prompt template
|
38 |
+
|
39 |
+
```
|
40 |
+
USER: Edit the following for spelling and grammar mistakes:
|
41 |
+
ASSISTANT:
|
42 |
+
```
|
43 |
+
|
44 |
## THE FILES IN MAIN BRANCH REQUIRES LATEST LLAMA.CPP (May 19th 2023 - commit 2d5db48)!
|
45 |
|
46 |
llama.cpp recently made another breaking change to its quantisation methods - https://github.com/ggerganov/llama.cpp/pull/1508
|
|
|
64 |
I use the following command line; adjust for your tastes and needs:
|
65 |
|
66 |
```
|
67 |
+
./main -t 10 -ngl 32 -m Karen-The-Editor.ggmlv3.q5_0.bin --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "USER: Edit the following for spelling and grammar mistakes: Hello whats' you're name their frend?\nASSISTANT:"
|
68 |
```
|
69 |
Change `-t 10` to the number of physical CPU cores you have. For example if your system has 8 cores/16 threads, use `-t 8`.
|
70 |
|