Transformers
English
falcon
TheBloke commited on
Commit
115f15c
1 Parent(s): 5436273

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -4
README.md CHANGED
@@ -17,9 +17,9 @@ license: other
17
  </div>
18
  <!-- header end -->
19
 
20
- # TII's Falcon 7B GGCC GGML
21
 
22
- These files are GGML format model files for [TII's Falcon 7B GGCC](https://huggingface.co/tiiuae/falcon-7b-instruct).
23
 
24
  These files will **not** work in llama.cpp, text-generation-webui or KoboldCpp.
25
 
@@ -56,9 +56,11 @@ Compiling on Windows: developer cmp-nct notes: 'I personally compile it using VS
56
 
57
  Once compiled you can then use `bin/falcon_main` just like you would use llama.cpp. For example:
58
  ```
59
- bin/falcon_main -t 8 -ngl 100 -b 1 -m falcon-40b-sft-mix-1226.ggccv1.q4_0.bin -enc -p "write a story about llamas"
60
  ```
61
 
 
 
62
  You can specify `-ngl 100` regardles of your VRAM, as it will automatically detect how much VRAM is available to be used.
63
 
64
  Adjust `-t 8` (the number of CPU cores to use) according to what performs best on your system. Do not exceed the number of physical CPU cores you have.
@@ -108,7 +110,7 @@ Thank you to all my generous patrons and donaters!
108
 
109
  <!-- footer end -->
110
 
111
- # Original model card: TII's Falcon 7B GGCC
112
 
113
 
114
  # ✨ Falcon-7B-Instruct
 
17
  </div>
18
  <!-- header end -->
19
 
20
+ # TII's Falcon 7B Instruct GGML
21
 
22
+ These files are GGML format model files for [TII's Falcon 7B Instruct](https://huggingface.co/tiiuae/falcon-7b-instruct).
23
 
24
  These files will **not** work in llama.cpp, text-generation-webui or KoboldCpp.
25
 
 
56
 
57
  Once compiled you can then use `bin/falcon_main` just like you would use llama.cpp. For example:
58
  ```
59
+ bin/falcon_main -t 8 -ngl 100 -b 1 -m falcon-7b-instruct.ggccv1.q4_0.bin -enc -p "write a story about llamas"
60
  ```
61
 
62
+ Parameter `-enc` should automatically use the right prompt template for the model, so you can just enter your desired prompt.
63
+
64
  You can specify `-ngl 100` regardles of your VRAM, as it will automatically detect how much VRAM is available to be used.
65
 
66
  Adjust `-t 8` (the number of CPU cores to use) according to what performs best on your system. Do not exceed the number of physical CPU cores you have.
 
110
 
111
  <!-- footer end -->
112
 
113
+ # Original model card: TII's Falcon 7B Instruct
114
 
115
 
116
  # ✨ Falcon-7B-Instruct