Create README.md

README.md

@@ -0,0 +1,42 @@
+---
+license: llama2
+language:
+- en
+---
+
+# Euryale-1.4-L2-70B IQ2-GGUF
+
+## Description
+IQ2-GGUF quants of [Sao10K/Euryale-1.4-L2-70B](https://huggingface.co/Sao10K/Euryale-1.4-L2-70B)
+
+Unlike regular GGUF quants this uses important matrix similar to Quip# to keep the quant from degrading too much even at 2bpw allowing you to run larger models on less powerful machines.
+
+***NOTE:*** As of uploading these this llamacpp can run these quants but i am unsure what guis like oobabooga / koboldcpp can run them.
+
+[More info](https://github.com/ggerganov/llama.cpp/pull/4897)
+
+
+# Models
+
+Models: [IQ2-XS](https://huggingface.co/Kooten/Euryale-1.4-L2-70B-IQ2-GGUF/raw/main/Euryale-1.4-L2-70B-IQ2_XS.gguf), [IQ2-XSS](https://huggingface.co/Kooten/Euryale-1.4-L2-70B-IQ2-GGUF/raw/main/Euryale-1.4-L2-70B-IQ2_XXS.gguf)
+
+Regular GGUF Quants: [Here](https://huggingface.co/Sao10K/Euryale-1.4-L2-70B-GGUF)
+
+## Prompt Format
+
+### Alpaca:
+```
+Below is an instruction that describes a task. Write a response that appropriately completes the request.
+
+### Instruction:
+{prompt}
+
+### Input:
+{input}
+
+### Response:
+
+```
+
+## Contact
+Kooten on discord