base_model: Pinkstack/PARM-V1.5-QwQ-Qwen-2.5-o1-3B-VLLM
tags:
- text-generation-inference
- transformers
- unsloth
- qwen2
- gguf
- Reasoning
- o1
- qwq
license: apache-2.0
language:
- en
- zh
pipeline_tag: text-generation
We are proud to announce, our new high quality flagship model series - PARM2, Very high quality reasoning, math and coding abilities for a small size, that anyone can run on their device for free.
🧀 Which quant is right for you?
- Q4: This model should be used on edge devices like high end phones or laptops due to its very compact size, quality is okay but fully usable.
- Q8: This model should be used on most high end modern devices like rtx 3080, Responses are very high quality, but its slightly slower than Q4. other formats were not included as Q4,Q8 have the best performance, quality.
This Parm v2 is based on Qwen 2.5 3B which has gotten many extra reasoning training parameters so it would have similar outputs to qwen QwQ / O.1 mini (only much, smaller.). We've trained it using the datasets here if you benchmarked this model let me know
This is a pretty lite model which can be run on high end phones pretty quickly using the q4 quant.
Passes "strawberry" test! (Q8 w/ msty & rtx 3080 10gb) ✅
To use this model, you must use a service which supports the GGUF file format. Additionaly, this is the Prompt Template options: efficient & accurate, answers stawberry text correctly 90% of the time.
{{ if .System }}<|system|>
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|user|>
{{ .Prompt }}<|im_end|>
{{ end }}<|assistant|>
{{ .Response }}<|im_end|>
Or if you are using an anti prompt: <|im_end|>
Highly recommended to use with a system prompt. eg; You are a helpful assistant named Parm2 by Pinkstack. think step-by-step for complex stuff, use COT if neeed.
Uploaded model
- Developed by: Pinkstack
- License: apache-2.0
- Finetuned from model : Pinkstack/PARM-V1.5-QwQ-Qwen-2.5-o1-3B-VLLM
This AI model was trained with Unsloth and Huggingface's TRL library.