Hugging Face's logo

Spaces:
NLPark
/
llama-python-streamingllm
Runtime error

App Files Community
Limour commited on
Commit
537bc2d
•
1 Parent(s): 341b821

Upload rp_sample_config.json

Browse files
Files changed (1) hide show
  1. config/rp_sample_config.json +3 -3
config/rp_sample_config.json CHANGED
@@ -6,10 +6,10 @@
6
  "value": 4294967295
7
  },
8
  "setting_n_gpu_layers": {
9
- "value": 0
10
  },
11
- "setting_ctx": {
12
- "value": 4096
13
  },
14
  "setting_max_tokens": {
15
  "value": 1024,
 
6
  "value": 4294967295
7
  },
8
  "setting_n_gpu_layers": {
9
+ "value": 49
10
  },
11
+ "setting_offload_kqv": {
12
+ "value": true
13
  },
14
  "setting_max_tokens": {
15
  "value": 1024,