bartowski

/

Llama-3.2-3B-Instruct-GGUF

like 102

llama 3.2 uses the same base prompt format as 3 and 3.1, It just has extra features, such as the "<|image|>" tag if using the 11B vision. How you choose to load and inject information into the final prompt depends a lot on your method, so I would suggest just to find llama 3.1 (or 3) examples for whatever program/language you use.

The prompt format is basically what your final cleaned up "prompt" should look like. It allows the model to classify the type of inputs/outputs. I'll post an example from SillyTavern below for how a basic conversation could look like.
This is basically saying: START -> "system" said _blank -> "assistant" said _blank -> "user" said _blank -> etc...

SNIPPET:
<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are an example character. Behave like an AI assistant.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Hello!<|eot_id|><|start_header_id|>user<|end_header_id|>

high there<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Hello! How can I assist you today? Is it help with something specific or general chat that you're in the mood for? I'm here to listen and provide information on a wide range of topics.<|eot_id|><|start_header_id|>user<|end_header_id|>

Say hi 5 times.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Hi, hi, hi, hi, Hi!<|eot_id|><|start_header_id|>user<|end_header_id|>

SNIPPET#2 (commandline):
Input: {"prompt": "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nAct as bee....

Perfect thanks!!