--- tags: - text-generation-inference - transformers - unsloth - gguf - reasoning - Qwen2 - Qwen license: apache-2.0 language: - en pipeline_tag: text-generation --- ![BY_PINKSTACK.png](https://cdn-uploads.huggingface.co/production/uploads/6710ba6af1279fe0dfe33afe/2xMulpuSlZ3C1vpGgsAYi.png) # 🧀 Which quant is right for you? - ***Q4:*** This model should be used for super low end devices like older phones or older laptops due to its very compact size, quality is okay but fully usable. - ***Q6:*** This model should be used on most modern devices, good quality and very quick responses. - ***Q8:*** This model should be used on most modern devices Responses are very high quality, but its a little slower than q6 - ***BF16:*** This Lossless model should only be used if maximum quality is needed; it doesn't perform well speed wise, but text results are high quality. ## Things you should be aware of when using PARM models (Pinkstack Accuracy Reasoning Models) 🧀 This PARM is based on Qwen 2.5 0.5B which has gotten extra training parameters so it would have similar outputs to O.1 Mini, We trained with [this](https://huggingface.co/datasets/gghfez/QwQ-LongCoT-130K-cleaned) dataset. it is designed to run on any device, from your phone to high-end PC. that is why we've included a BF16 quant. To use this model, you must use a service which supports the GGUF file format. Additionaly, this is the Prompt Template, it uses the Phi-3 template. ``` {{ if .System }}<|system|> {{ .System }}<|end|> {{ end }}{{ if .Prompt }}<|user|> {{ .Prompt }}<|end|> {{ end }}<|assistant|> {{ .Response }}<|end|> ``` Or if you are using an anti prompt: <|end|><|assistant|> Highly recommended to use with a system prompt. # Extra information - **Developed by:** Pinkstack - **License:** apache-2.0 - **Finetuned from model :** unsloth/qwen2.5-0.5b-instruct-bnb-4bit This model was trained using [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library. Used this model? Don't forget to leave a like :) [](https://github.com/unslothai/unsloth)