dranger003
/

dbrx-instruct-iMat.GGUF

Text Generation

Inference Endpoints

Model card Files Files and versions Community

dranger003 commited on Apr 14

Commit

4f67f3f

•

1 Parent(s): 17680aa

Update README.md

Files changed (1) hide show

README.md +4 -4

README.md CHANGED Viewed

@@ -12,11 +12,11 @@ The quants here are meant to test imatrix quantized weights.
 **Added `ggml-dbrx-instruct-16x12b-f16_imatrix-wiki.dat` which is a 2K batches (1M tokens) on FP16 weights using wiki.train.**
-| Precision | Quant/imatrix | Size (GiB) | PPL |
 | -- | -- | -- | -- |
-| IQ4_XS | Q8_0/wiki.train | 65.29 | 5.2260 +/- 0.03558 |
-| IQ4_XS | FP16/wiki.train | 65.29 | 5.2241 +/- 0.03559 |
-| IQ4_XS | None | 66.05 | 5.2546 +/- 0.03570 |
 **2024-04-13**: Support for this model has just being merged - [`PR #6515`](https://github.com/ggerganov/llama.cpp/pull/6515).
 **<u>You will need this llama.cpp commit [`4bd0f93e`](https://github.com/ggerganov/llama.cpp/commit/4bd0f93e4ab4fe6682e7d0241c1bdec1397e954a) to run this model</u>**

 **Added `ggml-dbrx-instruct-16x12b-f16_imatrix-wiki.dat` which is a 2K batches (1M tokens) on FP16 weights using wiki.train.**
+| Quant | IMatrix Quant/Dataset/Chunks | Size (GiB) | PPL (wiki.test) |
 | -- | -- | -- | -- |
+| IQ4_XS | Q8_0/wiki.train/200 | 65.29 | 5.2260 +/- 0.03558 |
+| IQ4_XS | FP16/wiki.train/2000 | 65.29 | 5.2241 +/- 0.03559 |
+| IQ4_XS | - | 66.05 | 5.2546 +/- 0.03570 |
 **2024-04-13**: Support for this model has just being merged - [`PR #6515`](https://github.com/ggerganov/llama.cpp/pull/6515).
 **<u>You will need this llama.cpp commit [`4bd0f93e`](https://github.com/ggerganov/llama.cpp/commit/4bd0f93e4ab4fe6682e7d0241c1bdec1397e954a) to run this model</u>**