README.md

---
library_name: transformers
tags:
- mergekit
- merge
- llama-cpp
- gguf-my-repo
base_model:
- meta-llama/Meta-Llama-3-8B-Instruct
- meta-llama/Meta-Llama-3-8B
---

# AtakanTekparmak/llama-3-20b-instruct-Q8_0-GGUF
This model was converted to GGUF format from [`AtakanTekparmak/llama-3-20b-instruct`](https://huggingface.co/AtakanTekparmak/llama-3-20b-instruct) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
Refer to the [original model card](https://huggingface.co/AtakanTekparmak/llama-3-20b-instruct) for more details on the model.

# DISCLAIMER: THIS MODEL DOES NOT WORK VERY WELL
This model is a part of me learning about and experimenting with model merging, should not be used for ANY use, as you'll see yourself from the outputs if you do decide to do so.
## Use with llama.cpp

Install llama.cpp through brew.

```bash
brew install ggerganov/ggerganov/llama.cpp
```
Invoke the llama.cpp server or the CLI.

CLI:

```bash
llama-cli --hf-repo AtakanTekparmak/llama-3-20b-instruct-Q8_0-GGUF --model llama-3-20b-instruct.Q8_0.gguf -p "The meaning to life and the universe is"
```

Server:

```bash
llama-server --hf-repo AtakanTekparmak/llama-3-20b-instruct-Q8_0-GGUF --model llama-3-20b-instruct.Q8_0.gguf -c 2048
```

Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.

```
git clone https://github.com/ggerganov/llama.cpp &&             cd llama.cpp &&             make &&             ./main -m llama-3-20b-instruct.Q8_0.gguf -n 128
```