README.md

---
base_model: meta-llama/Meta-Llama-3-8B-Instruct
inference: false
pipeline_tag: text-generation
language:
- en
license: other
license_name: llama3
license_link: https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/blob/main/LICENSE
model_creator: meta-llama
model_name: Meta-Llama-3-8B-Instruct
model_type: llama
tags:
- facebook
- meta
- pytorch
- llama
- llama-3
quantized_by: brittlewis12

---

# Meta-Llama-3-8B-Instruct GGUF

**Original model**: [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct)

**Model creator**: [Meta](https://huggingface.co/meta-llama)

> Meta developed and released the Meta Llama 3 family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes. The Llama 3 instruction tuned models are optimized for dialogue use cases and outperform many of the available open source chat models on common industry benchmarks. Further, in developing these models, we took great care to optimize helpfulness and safety.

> Llama 3 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.

This repo contains GGUF format model files for Meta’s Llama-3-8B-Instruct,
**updated as of 2024-04-29** to incorporate [tokenization improvements](https://github.com/ggerganov/llama.cpp/pull/6920), as well as previous interventions to handle the `<|eot_id|>` special token as EOS token.

Learn more on Meta’s [Llama 3 page](https://llama.meta.com/llama3).

### What is GGUF?

GGUF is a file format for representing AI models. It is the third version of the format, 
introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp. 
Converted with llama.cpp build 2763 (revision [ffe666](https://github.com/ggerganov/llama.cpp/commits/ffe666572f98a686b17a2cd1dbf4c0a982e5ac0a)),
using [autogguf](https://github.com/brittlewis12/autogguf).

### Prompt template

```
<|start_header_id|>system<|end_header_id|>

{{system_prompt}}<|eot_id|><|start_header_id|>user<|end_header_id|>

{{prompt}}<|eot_id|><|start_header_id|>assistant<|end_header_id|>


```

---

## Download & run with [cnvrs](https://twitter.com/cnvrsai) on iPhone, iPad, and Mac!

![cnvrs.ai](https://pbs.twimg.com/profile_images/1744049151241797632/0mIP-P9e_400x400.jpg)

[cnvrs](https://testflight.apple.com/join/sFWReS7K) is the best app for private, local AI on your device:
- create & save **Characters** with custom system prompts & temperature settings
- download and experiment with any **GGUF model** you can [find on HuggingFace](https://huggingface.co/models?library=gguf)!
- make it your own with custom **Theme colors**
- powered by Metal ⚡️ & [Llama.cpp](https://github.com/ggerganov/llama.cpp), with **haptics** during response streaming!
- **try it out** yourself today, on [Testflight](https://testflight.apple.com/join/sFWReS7K)!
- follow [cnvrs on twitter](https://twitter.com/cnvrsai) to stay up to date

---

## Original Model Evaluation

<table>
  <tr>
   <td><strong>Benchmark</strong>
   </td>
   <td><strong>Llama 3 8B</strong>
   </td>
   <td><strong>Llama 2 7B</strong>
   </td>
   <td><strong>Llama 2 13B</strong>
   </td>
  </tr>
  <tr>
   <td>MMLU (5-shot)
   </td>
   <td><b>68.4</b>
   </td>
   <td>34.1
   </td>
   <td>47.8
   </td>
  </tr>
  <tr>
   <td>GPQA (0-shot)
   </td>
   <td><b>34.2</b>
   </td>
   <td>21.7
   </td>
   <td>22.3
   </td>
  </tr>
  <tr>
   <td>HumanEval (0-shot)
   </td>
   <td><b>62.2</b>
   </td>
   <td>7.9
   </td>
   <td>14.0
   </td>
  </tr>
  <tr>
   <td>GSM-8K (8-shot, CoT)
   </td>
   <td><b>79.6</b>
   </td>
   <td>25.7
   </td>
   <td>77.4
   </td>
  </tr>
  <tr>
   <td>MATH (4-shot, CoT)
   </td>
   <td><b>30.0</b>
   </td>
   <td>3.8
   </td>
   <td>6.7
   </td>
  </tr>
</table>