RichardErkhov/athirdpath_-_Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit - GGUF

• Model creator: https://huggingface.co/athirdpath/
• Original model: https://huggingface.co/athirdpath/Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit/

Name	Quant method	Size
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q2_K.gguf	Q2_K	2.96GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.IQ3_XS.gguf	IQ3_XS	0.01GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.IQ3_S.gguf	IQ3_S	0.01GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q3_K_S.gguf	Q3_K_S	0.01GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.IQ3_M.gguf	IQ3_M	0.01GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q3_K.gguf	Q3_K	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q3_K_M.gguf	Q3_K_M	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q3_K_L.gguf	Q3_K_L	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.IQ4_XS.gguf	IQ4_XS	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q4_0.gguf	Q4_0	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.IQ4_NL.gguf	IQ4_NL	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q4_K_S.gguf	Q4_K_S	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q4_K.gguf	Q4_K	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q4_K_M.gguf	Q4_K_M	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q4_1.gguf	Q4_1	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q5_0.gguf	Q5_0	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q5_K_S.gguf	Q5_K_S	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q5_K.gguf	Q5_K	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q5_K_M.gguf	Q5_K_M	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q5_1.gguf	Q5_1	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q6_K.gguf	Q6_K	0.0GB
Llama-3.1-Instruct_NSFW-pretrained_e1-plus_reddit.Q8_0.gguf	Q8_0	1.17GB

Original model description:

base_model: athirdpath/Llama-3.1-Instruct_NSFW-pretrained_e1 language: - en license: apache-2.0 tags: - text-generation-inference - transformers - unsloth - llama - trl - sft

athirdpath/Llama-3.1-Instruct_NSFW-pretrained_e1 further pretrained on 1 epoch of the dirty stories from nothingiisreal/Reddit-Dirty-And-WritingPrompts, with all scores below 2 dropped.

---

Why do this? I have a niche use case where I cannot increase compute over 8b, and L3/3.1 are the only models in this size category that meet my needs for logic. However, both versions of L3/3.1 have the damn repetition/token overconfidence problem, and this is meant to disrupt that certainty without disrupting the model's ability to function.

By the way, I think it's the lm_head that is causing the looping, but it might be the embeddings being too separated. I'm not going to pay two more times to test them separately, however :p