File size: 930 Bytes
573d5d5 deee96b 573d5d5 deee96b 573d5d5 ce1822d deee96b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
---
base_model: hpcgroup/hpc-coder-v2-6.7b
library_name: transformers
tags:
- code
- hpc
- parallel
- axonn
- llama-cpp
- gguf-my-repo
---
# HPC-Coder-v2-6.7b-Q4_K_S-GGUF
This is the [HPC-Coder-v2-6.7b](https://huggingface.co/hpcgroup/hpc-coder-v2-6.7b) model with 4 bit quantized weights in the GGUF format that can be used with llama.cpp.
Refer to the [original model card](https://huggingface.co/hpcgroup/hpc-coder-v2-6.7b) for more details on the model.
## Use with llama.cpp
See the [llama.cpp repo](https://github.com/ggerganov/llama.cpp) for installation instructions.
You can then use the model as:
```bash
llama-cli --hf-repo hpcgroup/hpc-coder-v2-6.7b-Q4_K_S-GGUF --hf-file hpc-coder-v2-6.7b-q4_k_s.gguf -r "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:" --in-prefix "\n" --in-suffix "\n### Response:\n" -c 8096 -p "your prompt here"
``` |