Kooten's picture
Update README.md
93fc079 verified
metadata
license: llama2
language:
  - en

WinterGoddess-1.4x-70B-L2 IQ2-GGUF

Description

IQ2-GGUF quants of sophosympatheia/Aurora-Nights-70B-v1.0

Unlike regular GGUF quants this uses important matrix similar to Quip# to keep the quant from degrading too much even at 2bpw allowing you to run larger models on less powerful machines.

NOTE: Currently you will need experimental branches of Koboldcpp or Ooba for this to work.

More info about IQ2

Models

Models: IQ2-XS, IQ2-XXS

Regular GGUF Quants: Here

Prompt Format

Unclear

Contact

Kooten on discord