metadata

library_name: transformers
tags:
  - mergekit
  - merge
  - llama-cpp
  - gguf-my-repo
base_model:
  - meta-llama/Meta-Llama-3-8B-Instruct
  - meta-llama/Meta-Llama-3-8B

AtakanTekparmak/llama-3-20b-instruct-Q8_0-GGUF

This model was converted to GGUF format from AtakanTekparmak/llama-3-20b-instruct using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

DISCLAIMER: THIS MODEL DOES NOT WORK VERY WELL

This model is a part of me learning about and experimenting with model merging, should not be used for ANY use, as you'll see yourself from the outputs if you do decide to do so.

Use with llama.cpp

Install llama.cpp through brew.

brew install ggerganov/ggerganov/llama.cpp

Invoke the llama.cpp server or the CLI.

CLI:

llama-cli --hf-repo AtakanTekparmak/llama-3-20b-instruct-Q8_0-GGUF --model llama-3-20b-instruct.Q8_0.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo AtakanTekparmak/llama-3-20b-instruct-Q8_0-GGUF --model llama-3-20b-instruct.Q8_0.gguf -c 2048

Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.

git clone https://github.com/ggerganov/llama.cpp &&             cd llama.cpp &&             make &&             ./main -m llama-3-20b-instruct.Q8_0.gguf -n 128