ikawrakow commited on
Commit
a830c8b
·
1 Parent(s): f014ef7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -0
README.md CHANGED
@@ -1,3 +1,9 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+
5
+ This repository contains 2-bit quantized LLaMA-v2 models in GGUF format for use with [llama.cpp](https://github.com/ggerganov/llama.cpp).
6
+ All tensors are quantized with `Q2_K`, except for `output.weight`, which is `Q6_K`, and, in the case of LLaMA-v2-70B, `attn_v`, which is `Q4_K`.
7
+ The quantized models differ from the standard `llama.cpp` 2-bit quantization in two ways:
8
+ * These are actual 2-bit quantized models instead of the mostly 3-bit quantization provided by the standard `llama.cpp` `Q2_K` quantization method
9
+ * The models were prepared a refined (but not yet published) k-quants quantization approach