Spaces:
Paused
Paused
Daniel Marques
commited on
Commit
•
fbbc3b6
1
Parent(s):
cdef4d4
feat: add websocket
Browse files- constants.py +1 -1
constants.py
CHANGED
|
@@ -38,7 +38,7 @@ MAX_NEW_TOKENS = CONTEXT_WINDOW_SIZE # int(CONTEXT_WINDOW_SIZE/4)
|
|
| 38 |
#### If you get a "not enough space in the buffer" error, you should reduce the values below, start with half of the original values and keep halving the value until the error stops appearing
|
| 39 |
|
| 40 |
N_GPU_LAYERS = 40 # Llama-2-70B has 83 layers
|
| 41 |
-
N_BATCH =
|
| 42 |
|
| 43 |
### From experimenting with the Llama-2-7B-Chat-GGML model on 8GB VRAM, these values work:
|
| 44 |
# N_GPU_LAYERS = 20
|
|
|
|
| 38 |
#### If you get a "not enough space in the buffer" error, you should reduce the values below, start with half of the original values and keep halving the value until the error stops appearing
|
| 39 |
|
| 40 |
N_GPU_LAYERS = 40 # Llama-2-70B has 83 layers
|
| 41 |
+
N_BATCH = 512
|
| 42 |
|
| 43 |
### From experimenting with the Llama-2-7B-Chat-GGML model on 8GB VRAM, these values work:
|
| 44 |
# N_GPU_LAYERS = 20
|