README.md

---
license: other
license_name: llama3
license_link: https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/blob/main/LICENSE
---
GGUF of [Replete-AI Llama 3 11.5B Instruct V2](https://huggingface.co/Replete-AI/Llama-3-11.5B-Instruct-v2)

Quantized with llama.cpp commit <s>[b2710](https://github.com/ggerganov/llama.cpp/releases/tag/b2710)</s> <s>[b2780](https://github.com/ggerganov/llama.cpp/releases/tag/b2780)</s> [b2876](https://github.com/ggerganov/llama.cpp/releases/tag/b2876), verified no warnings in llama.cpp

Simple PPL comparison<br>
<code>
<i>perplexity.exe -[MODEL] -f wiki.test.raw -b 512 -ngl 99</i>

<i>Replete-AI_Llama-3-11.5B-Instruct-V2-Q6_K.gguf</i> - Final estimate: <b>Final estimate: PPL = 8.4438 +/- 0.06271</b><br>
<i>Meta-Llama-3-8B-Instruct-Q6_K</i> - Final estimate: <b>PPL = 8.4727 +/- 0.06308</b>
</code>

Original model description below<hr>

Llama-3-11.5B-Instruct-v2

Thank you to Meta for the weights for Meta-Llama-3-8B-Instruct

![image/png](https://cdn-uploads.huggingface.co/production/uploads/642cc1c253e76b4c2286c58e/aJJxKus1wP5N-euvHEUq7.png)

This is an upscaling of the Meta-Llama-3-8B-Instruct Ai using techniques created for chargoddard/mistral-11b-slimorca. This Ai model has been upscaled from 8b parameters to 11.5b parameters without any continuous pretraining or fine-tuning.

Unlike version 1 this model has no issues at fp16 or any quantizations.

The model that was used to create this one is linked below:

https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct