dranger003
/

dbrx-instruct-iMat.GGUF

Text Generation

Inference Endpoints

Model card Files Files and versions Community

dranger003 commited on Apr 14

Commit

ac82c46

•

1 Parent(s): d009029

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -10,7 +10,7 @@ base_model: databricks/dbrx-instruct
 The quants here are meant to test imatrix quantized weights.
 <i>If you run metal, you may need this [PR](https://github.com/ggerganov/llama.cpp/pull/6662)</i>
-**I am running a new set of imatrix for `wiki.train.raw` and `groups_merged.txt` and will produce PPL to compare but this will take some time.**
 **2024-04-13**: Support for this model has just being merged - [`PR #6515`](https://github.com/ggerganov/llama.cpp/pull/6515).
 **<u>You will need this llama.cpp commit [`4bd0f93e`](https://github.com/ggerganov/llama.cpp/commit/4bd0f93e4ab4fe6682e7d0241c1bdec1397e954a) to run this model</u>**

 The quants here are meant to test imatrix quantized weights.
 <i>If you run metal, you may need this [PR](https://github.com/ggerganov/llama.cpp/pull/6662)</i>
+**Added `ggml-dbrx-instruct-16x12b-f16_imatrix-wiki.dat` which is a 2K batches (1M tokens) on FP16 weights using wiki.train.**
 **2024-04-13**: Support for this model has just being merged - [`PR #6515`](https://github.com/ggerganov/llama.cpp/pull/6515).
 **<u>You will need this llama.cpp commit [`4bd0f93e`](https://github.com/ggerganov/llama.cpp/commit/4bd0f93e4ab4fe6682e7d0241c1bdec1397e954a) to run this model</u>**