littlebird13 commited on
Commit
879a584
1 Parent(s): 6856bad

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -43,7 +43,7 @@ To run Qwen2, you can use `llama-cli` (the previous `main`) or `llama-server` (t
43
  We recommend using the `llama-server` as it is simple and compatible with OpenAI API. For example:
44
 
45
  ```bash
46
- ./llama-server -m qwen2-0.5b-instruct-q5_k_m.gguf -ngl 24 -fa
47
  ```
48
 
49
  (Note: `-ngl 24` refers to offloading 24 layers to GPUs, and `-fa` refers to the use of flash attention.)
@@ -71,7 +71,7 @@ print(completion.choices[0].message.content)
71
  If you choose to use `llama-cli`, pay attention to the removal of `-cml` for the ChatML template. Instead you should use `--in-prefix` and `--in-suffix` to tackle this problem.
72
 
73
  ```bash
74
- ./llama-cli -m qwen2-0.5b-instruct-q5_k_m.gguf \
75
  -n 512 -co -i -if -f prompts/chat-with-qwen.txt \
76
  --in-prefix "<|im_start|>user\n" \
77
  --in-suffix "<|im_end|>\n<|im_start|>assistant\n" \
 
43
  We recommend using the `llama-server` as it is simple and compatible with OpenAI API. For example:
44
 
45
  ```bash
46
+ ./llama-server -m qwen2-0_5b-instruct-q5_k_m.gguf -ngl 24 -fa
47
  ```
48
 
49
  (Note: `-ngl 24` refers to offloading 24 layers to GPUs, and `-fa` refers to the use of flash attention.)
 
71
  If you choose to use `llama-cli`, pay attention to the removal of `-cml` for the ChatML template. Instead you should use `--in-prefix` and `--in-suffix` to tackle this problem.
72
 
73
  ```bash
74
+ ./llama-cli -m qwen2-0_5b-instruct-q5_k_m.gguf \
75
  -n 512 -co -i -if -f prompts/chat-with-qwen.txt \
76
  --in-prefix "<|im_start|>user\n" \
77
  --in-suffix "<|im_end|>\n<|im_start|>assistant\n" \