Initial GGML model commit

README.md

@@ -33,7 +33,7 @@ quantized_by: TheBloke
 
 This repo contains GGML format model files for [Kai Howard's PuddleJumper 13B](https://huggingface.co/totally-not-an-llm/PuddleJumper-13b).
 
-### Important note regarding GGML files
+### Important note regarding GGML files.
 
 The GGML format has now been superseded by GGUF. As of August 21st 2023, [llama.cpp](https://github.com/ggerganov/llama.cpp) no longer supports GGML models. Third party clients and libraries are expected to still support it for a time, but many may also drop support.
 
@@ -95,17 +95,17 @@ Refer to the Provided Files table below to see what files use which methods, and
 | Name | Quant method | Bits | Size | Max RAM required | Use case |
 | ---- | ---- | ---- | ---- | ---- | ----- |
 | [puddlejumper-13b.ggmlv3.Q2_K.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q2_K.bin) | Q2_K | 2 | 5.74 GB| 8.24 GB | New k-quant method. Uses GGML_TYPE_Q4_K for the attention.vw and feed_forward.w2 tensors, GGML_TYPE_Q2_K for the other tensors. |
-| [puddlejumper-13b.ggmlv3.Q3_K_L.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q3_K_L.bin) | Q3_K_L | 3 | 7.14 GB| 9.64 GB | New k-quant method. Uses GGML_TYPE_Q5_K for the attention.wv, attention.wo, and feed_forward.w2 tensors, else GGML_TYPE_Q3_K |
-| [puddlejumper-13b.ggmlv3.Q3_K_M.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q3_K_M.bin) | Q3_K_M | 3 | 6.53 GB| 9.03 GB | New k-quant method. Uses GGML_TYPE_Q4_K for the attention.wv, attention.wo, and feed_forward.w2 tensors, else GGML_TYPE_Q3_K |
 | [puddlejumper-13b.ggmlv3.Q3_K_S.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q3_K_S.bin) | Q3_K_S | 3 | 5.87 GB| 8.37 GB | New k-quant method. Uses GGML_TYPE_Q3_K for all tensors |
+| [puddlejumper-13b.ggmlv3.Q3_K_M.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q3_K_M.bin) | Q3_K_M | 3 | 6.53 GB| 9.03 GB | New k-quant method. Uses GGML_TYPE_Q4_K for the attention.wv, attention.wo, and feed_forward.w2 tensors, else GGML_TYPE_Q3_K |
+| [puddlejumper-13b.ggmlv3.Q3_K_L.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q3_K_L.bin) | Q3_K_L | 3 | 7.14 GB| 9.64 GB | New k-quant method. Uses GGML_TYPE_Q5_K for the attention.wv, attention.wo, and feed_forward.w2 tensors, else GGML_TYPE_Q3_K |
 | [puddlejumper-13b.ggmlv3.Q4_0.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q4_0.bin) | Q4_0 | 4 | 7.32 GB| 9.82 GB | Original quant method, 4-bit. |
-| [puddlejumper-13b.ggmlv3.Q4_1.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q4_1.bin) | Q4_1 | 4 | 8.14 GB| 10.64 GB | Original quant method, 4-bit. Higher accuracy than q4_0 but not as high as q5_0. However has quicker inference than q5 models. |
-| [puddlejumper-13b.ggmlv3.Q4_K_M.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q4_K_M.bin) | Q4_K_M | 4 | 8.06 GB| 10.56 GB | New k-quant method. Uses GGML_TYPE_Q6_K for half of the attention.wv and feed_forward.w2 tensors, else GGML_TYPE_Q4_K |
 | [puddlejumper-13b.ggmlv3.Q4_K_S.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q4_K_S.bin) | Q4_K_S | 4 | 7.56 GB| 10.06 GB | New k-quant method. Uses GGML_TYPE_Q4_K for all tensors |
+| [puddlejumper-13b.ggmlv3.Q4_K_M.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q4_K_M.bin) | Q4_K_M | 4 | 8.06 GB| 10.56 GB | New k-quant method. Uses GGML_TYPE_Q6_K for half of the attention.wv and feed_forward.w2 tensors, else GGML_TYPE_Q4_K |
+| [puddlejumper-13b.ggmlv3.Q4_1.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q4_1.bin) | Q4_1 | 4 | 8.14 GB| 10.64 GB | Original quant method, 4-bit. Higher accuracy than q4_0 but not as high as q5_0. However has quicker inference than q5 models. |
 | [puddlejumper-13b.ggmlv3.Q5_0.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q5_0.bin) | Q5_0 | 5 | 8.95 GB| 11.45 GB | Original quant method, 5-bit. Higher accuracy, higher resource usage and slower inference. |
-| [puddlejumper-13b.ggmlv3.Q5_1.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q5_1.bin) | Q5_1 | 5 | 9.76 GB| 12.26 GB | Original quant method, 5-bit. Even higher accuracy, resource usage and slower inference. |
-| [puddlejumper-13b.ggmlv3.Q5_K_M.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q5_K_M.bin) | Q5_K_M | 5 | 9.40 GB| 11.90 GB | New k-quant method. Uses GGML_TYPE_Q6_K for half of the attention.wv and feed_forward.w2 tensors, else GGML_TYPE_Q5_K |
 | [puddlejumper-13b.ggmlv3.Q5_K_S.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q5_K_S.bin) | Q5_K_S | 5 | 9.14 GB| 11.64 GB | New k-quant method. Uses GGML_TYPE_Q5_K for all tensors |
+| [puddlejumper-13b.ggmlv3.Q5_K_M.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q5_K_M.bin) | Q5_K_M | 5 | 9.40 GB| 11.90 GB | New k-quant method. Uses GGML_TYPE_Q6_K for half of the attention.wv and feed_forward.w2 tensors, else GGML_TYPE_Q5_K |
+| [puddlejumper-13b.ggmlv3.Q5_1.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q5_1.bin) | Q5_1 | 5 | 9.76 GB| 12.26 GB | Original quant method, 5-bit. Even higher accuracy, resource usage and slower inference. |
 | [puddlejumper-13b.ggmlv3.Q6_K.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q6_K.bin) | Q6_K | 6 | 10.83 GB| 13.33 GB | New k-quant method. Uses GGML_TYPE_Q8_K for all tensors - 6-bit quantization |
 | [puddlejumper-13b.ggmlv3.Q8_0.bin](https://huggingface.co/TheBloke/PuddleJumper-13B-GGML/blob/main/puddlejumper-13b.ggmlv3.Q8_0.bin) | Q8_0 | 8 | 13.83 GB| 16.33 GB | Original quant method, 8-bit. Almost indistinguishable from float16. High resource use and slow. Not recommended for most users. |