Hugging Face
Models
Datasets
Spaces
Posts
Docs
Solutions
Pricing
Log In
Sign Up
nisten
/
qwenv2-7b-inst-imatrix-gguf
like
3
GGUF
License:
apache-2.0
Model card
Files
Files and versions
Community
Use this model
9cbd6f2
qwenv2-7b-inst-imatrix-gguf
1 contributor
History:
13 commits
nisten
Rename qwen7bf16.gguf to qwen7bv2instruct_bf16.gguf
9cbd6f2
verified
17 days ago
.gitattributes
2.63 kB
Rename qwen7bf16.gguf to qwen7bv2instruct_bf16.gguf
17 days ago
8bitimatrix.dat
4.54 MB
LFS
calculated imatrix in 8bit, was jsut as good as f16 imatrix
17 days ago
README.md
1.55 kB
Update README.md
17 days ago
qwen7bq4kembeddingf16outputf16.gguf
6.11 GB
LFS
Rename qwen7bq4kembeddingbf16outputbf16.gguf to qwen7bq4kembeddingf16outputf16.gguf
17 days ago
qwen7bq4xsembedding8output8.gguf
4.64 GB
LFS
Rename qwen7bq4xsembedding5bitkoutput8bit.gguf to qwen7bq4xsembedding8output8.gguf
17 days ago
qwen7bq4xsoutput6k.gguf
4.22 GB
LFS
Rename qwen7bq4xs.gguf to qwen7bq4xsoutput6k.gguf
17 days ago
qwen7bv2_iq4xs_output8bit.gguf
4.35 GB
LFS
Probably best speed to perplexity ratio of any 7b gguf model so far
17 days ago
qwen7bv2inst_q4km_output8bit.gguf
4.82 GB
LFS
very good quant for speed/perplexity, embedding is at q4k
17 days ago
qwen7bv2instruct_bf16.gguf
15.2 GB
LFS
Rename qwen7bf16.gguf to qwen7bv2instruct_bf16.gguf
17 days ago
qwen7bv2instruct_q5km.gguf
5.58 GB
LFS
standard q5km conversions with 8bit output for reference.
17 days ago
qwenv2instruct7b_q8.gguf
8.1 GB
LFS
Good conversion from bf16 down instead of from f16
17 days ago