Update README.md
Browse files
README.md
CHANGED
@@ -32,12 +32,12 @@ Zipped Quantization (if you want to download a single file)
|
|
32 |
|
33 |
## Memory Usage
|
34 |
Measured using ExLlamaV2 and 4096 max_seq_len with [Oobabooga's Text Generation WebUI](https://github.com/oobabooga/text-generation-webui/tree/main).
|
35 |
-
| Branch | BPW | VRAM Usage |
|
36 |
-
| ------ | --- | ---------- |
|
37 |
-
[3.0bpw](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/3.0bpw)|3.0|3.7 GB
|
38 |
-
[4.0bpw (main)](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/main)|4.0|4.4 GB
|
39 |
-
[6.0bpw](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/6.0bpw)|6.0|6.1 GB
|
40 |
-
[8.0bpw](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/8.0bpw)|8.0|7.7 GB
|
41 |
|
42 |
## Prompt template: ChatML
|
43 |
|
|
|
32 |
|
33 |
## Memory Usage
|
34 |
Measured using ExLlamaV2 and 4096 max_seq_len with [Oobabooga's Text Generation WebUI](https://github.com/oobabooga/text-generation-webui/tree/main).
|
35 |
+
| Branch | BPW | VRAM Usage | Description |
|
36 |
+
| ------ | --- | ---------- | ----------- |
|
37 |
+
[3.0bpw](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/3.0bpw)|3.0|3.7 GB|For >=6GB VRAM cards
|
38 |
+
[4.0bpw (main)](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/main)|4.0|4.4 GB|For >=6GB VRAM cards
|
39 |
+
[6.0bpw](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/6.0bpw)|6.0|6.1 GB|For >=8GB VRAM cards
|
40 |
+
[8.0bpw](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/8.0bpw)|8.0|7.7 GB|For >=10GB VRAM cards
|
41 |
|
42 |
## Prompt template: ChatML
|
43 |
|