IHaveNoClueAndIMustPost
commited on
Commit
•
127ee91
1
Parent(s):
6dd0163
Update README.md
Browse files
README.md
CHANGED
@@ -9,8 +9,32 @@ tags:
|
|
9 |
This is [Llama2-22b](https://huggingface.co/chargoddard/llama2-22b) in a couple of GGML ormats. I have no idea what I'm doing so if something doesn't work that's likely on me, not the models themselves.
|
10 |
|
11 |
Approximate VRAM requirements
|
12 |
-
|
13 |
-
|
14 |
-
|
15 |
-
|
16 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
9 |
This is [Llama2-22b](https://huggingface.co/chargoddard/llama2-22b) in a couple of GGML ormats. I have no idea what I'm doing so if something doesn't work that's likely on me, not the models themselves.
|
10 |
|
11 |
Approximate VRAM requirements
|
12 |
+
<table style='border: 2px #000000 solid; width: 25%' align='left' border='2'>
|
13 |
+
<tbody>
|
14 |
+
<tr>
|
15 |
+
<td style='text-align: center'>MODEL</td>
|
16 |
+
<td style='text-align: center'>SIZE</td>
|
17 |
+
<td style='text-align: center'>VRAM</td>
|
18 |
+
</tr>
|
19 |
+
<tr>
|
20 |
+
<td style='text-align: center'>q5_1</td>
|
21 |
+
<td style='text-align: center'>16GB</td>
|
22 |
+
<td style='text-align: center'>21.5GB</td>
|
23 |
+
</tr>
|
24 |
+
<tr>
|
25 |
+
<td style='text-align: center'>q4_K_M</td>
|
26 |
+
<td style='text-align: center'>12.8GB</td>
|
27 |
+
<td style='text-align: center'>18.3GB</td>
|
28 |
+
</tr>
|
29 |
+
<tr>
|
30 |
+
<td style='text-align: center'>q3_K_M</td>
|
31 |
+
<td style='text-align: center'>10.0GB</td>
|
32 |
+
<td style='text-align: center'>16.1GB</td>
|
33 |
+
</tr>
|
34 |
+
<tr>
|
35 |
+
<td style='text-align: center'>q2_K</td>
|
36 |
+
<td style='text-align: center'>9.0GB</td>
|
37 |
+
<td style='text-align: center'>14.5GB</td>
|
38 |
+
</tr>
|
39 |
+
</tbody>
|
40 |
+
</table>
|