Update README.md
Browse files
README.md
CHANGED
@@ -17,9 +17,9 @@ license: other
|
|
17 |
</div>
|
18 |
<!-- header end -->
|
19 |
|
20 |
-
# TII's Falcon 7B
|
21 |
|
22 |
-
These files are GGML format model files for [TII's Falcon 7B
|
23 |
|
24 |
These files will **not** work in llama.cpp, text-generation-webui or KoboldCpp.
|
25 |
|
@@ -56,9 +56,11 @@ Compiling on Windows: developer cmp-nct notes: 'I personally compile it using VS
|
|
56 |
|
57 |
Once compiled you can then use `bin/falcon_main` just like you would use llama.cpp. For example:
|
58 |
```
|
59 |
-
bin/falcon_main -t 8 -ngl 100 -b 1 -m falcon-
|
60 |
```
|
61 |
|
|
|
|
|
62 |
You can specify `-ngl 100` regardles of your VRAM, as it will automatically detect how much VRAM is available to be used.
|
63 |
|
64 |
Adjust `-t 8` (the number of CPU cores to use) according to what performs best on your system. Do not exceed the number of physical CPU cores you have.
|
@@ -108,7 +110,7 @@ Thank you to all my generous patrons and donaters!
|
|
108 |
|
109 |
<!-- footer end -->
|
110 |
|
111 |
-
# Original model card: TII's Falcon 7B
|
112 |
|
113 |
|
114 |
# ✨ Falcon-7B-Instruct
|
|
|
17 |
</div>
|
18 |
<!-- header end -->
|
19 |
|
20 |
+
# TII's Falcon 7B Instruct GGML
|
21 |
|
22 |
+
These files are GGML format model files for [TII's Falcon 7B Instruct](https://huggingface.co/tiiuae/falcon-7b-instruct).
|
23 |
|
24 |
These files will **not** work in llama.cpp, text-generation-webui or KoboldCpp.
|
25 |
|
|
|
56 |
|
57 |
Once compiled you can then use `bin/falcon_main` just like you would use llama.cpp. For example:
|
58 |
```
|
59 |
+
bin/falcon_main -t 8 -ngl 100 -b 1 -m falcon-7b-instruct.ggccv1.q4_0.bin -enc -p "write a story about llamas"
|
60 |
```
|
61 |
|
62 |
+
Parameter `-enc` should automatically use the right prompt template for the model, so you can just enter your desired prompt.
|
63 |
+
|
64 |
You can specify `-ngl 100` regardles of your VRAM, as it will automatically detect how much VRAM is available to be used.
|
65 |
|
66 |
Adjust `-t 8` (the number of CPU cores to use) according to what performs best on your system. Do not exceed the number of physical CPU cores you have.
|
|
|
110 |
|
111 |
<!-- footer end -->
|
112 |
|
113 |
+
# Original model card: TII's Falcon 7B Instruct
|
114 |
|
115 |
|
116 |
# ✨ Falcon-7B-Instruct
|