|
--- |
|
base_model: NeverSleep/Lumimaid-v0.2-8B |
|
quantized_by: Lewdiculous |
|
library_name: transformers |
|
license: cc-by-nc-4.0 |
|
inference: false |
|
language: |
|
- en |
|
tags: |
|
- roleplay |
|
- llama3 |
|
- sillytavern |
|
--- |
|
|
|
# #roleplay #sillytavern #llama3 |
|
|
|
My GGUF-IQ-Imatrix quants for [**NeverSleep/Lumimaid-v0.2-8B**](https://huggingface.co/NeverSleep/Lumimaid-v0.2-8B). |
|
|
|
I recommend checking their page for feedback and support. |
|
|
|
> [!IMPORTANT] |
|
> **Quantization process:** <br> |
|
> Imatrix data was generated from the FP16-GGUF and conversions directly from the BF16-GGUF. <br> |
|
> This is a bit more disk and compute intensive but hopefully avoids any losses during conversion. <br> |
|
> To run this model, please use the [**latest version of KoboldCpp**](https://github.com/LostRuins/koboldcpp/releases/latest). <br> |
|
> If you noticed any issues let me know in the discussions. |
|
|
|
> [!NOTE] |
|
> **Presets:** <br> |
|
> Some compatible SillyTavern presets can be found [**here (Virt's Roleplay Presets - v1.9)**](https://huggingface.co/Virt-io/SillyTavern-Presets). <br> |
|
> Check [**discussions such as this one**](https://huggingface.co/Virt-io/SillyTavern-Presets/discussions/5#664d6fb87c563d4d95151baa) and [**this one**](https://www.reddit.com/r/SillyTavernAI/comments/1dff2tl/my_personal_llama3_stheno_presets/) for other presets and samplers recommendations. <br> |
|
> Lower temperatures are recommended by the authors, so make sure to experiment. <br> |
|
> |
|
> **General usage with KoboldCpp:** <br> |
|
> For **8GB VRAM** GPUs, I recommend the **Q4_K_M-imat** (4.89 BPW) quant for up to 12288 context sizes without the use of `--quantkv`. <br> |
|
> Using `--quantkv 1` (≈Q8) or even `--quantkv 2` (≈Q4) can get you to 32K context sizes with the caveat of not being compatible with Context Shifting, only relevant if you can manage to fill up that much context. <br> |
|
> [**Read more about it in the release here**](https://github.com/LostRuins/koboldcpp/releases/tag/v1.67). |
|
|
|
|
|
<details> |
|
<summary>⇲ Click here to expand/hide information – General chart with relative quant parformances.</summary> |
|
|
|
> [!NOTE] |
|
> **Recommended read:** <br> |
|
> |
|
> [**"Which GGUF is right for me? (Opinionated)" by Artefact2**](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9) |
|
> |
|
> *Click the image to view full size.* |
|
>  |
|
|
|
</details> |
|
|
|
> [!TIP] |
|
> **Personal-support:** <br> |
|
> I apologize for disrupting your experience. <br> |
|
> Eventually I may be able to use a dedicated server for this, but for now hopefully these quants are helpful. <br> |
|
> If you **want** and you are **able to**... <br> |
|
> You can [**spare some change over here (Ko-fi)**](https://ko-fi.com/Lewdiculous). <br> |
|
> |
|
> **Author-support:** <br> |
|
> You can support the authors [**at their pages**](https://ko-fi.com/undiai)/[**here**](https://ikaridevgit.github.io/). |
|
|
|
 |
|
|
|
<details> |
|
<summary>Original model card information.</summary> |
|
|
|
## **Original card:** |
|
|
|
## Lumimaid 0.2 |
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/63ab1241ad514ca8d1430003/TUcHg7LKNjfo0sni88Ps7.png" alt="Image" style="display: block; margin-left: auto; margin-right: auto; width: 65%;"> |
|
<div style="text-align: center; font-size: 30px;"> |
|
<a href="https://huggingface.co/NeverSleep/Lumimaid-v0.2-8B">[8b]</a> - |
|
<a href="https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B">12b</a> - |
|
<a href="https://huggingface.co/NeverSleep/Lumimaid-v0.2-70B">70b</a> - |
|
<a href="https://huggingface.co/NeverSleep/Lumimaid-v0.2-123B">123b</a> |
|
</div> |
|
|
|
### This model is based on: [Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) |
|
Wandb: https://wandb.ai/undis95/Lumi-Llama-3-1-8B?nw=nwuserundis95 |
|
|
|
Lumimaid 0.1 -> 0.2 is a HUGE step up dataset wise. |
|
|
|
As some people have told us our models are sloppy, Ikari decided to say fuck it and literally nuke all chats out with most slop. |
|
|
|
Our dataset stayed the same since day one, we added data over time, cleaned them, and repeat. After not releasing model for a while because we were never satisfied, we think it's time to come back! |
|
|
|
|
|
## Prompt template: Llama-3-Instruct |
|
|
|
``` |
|
<|begin_of_text|><|start_header_id|>system<|end_header_id|> |
|
|
|
{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|> |
|
|
|
{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|> |
|
|
|
{output}<|eot_id|> |
|
``` |
|
|
|
## Credits: |
|
- Undi |
|
- IkariDev |
|
|
|
## Training data we used to make our dataset: |
|
|
|
- [Epiculous/Gnosis](https://huggingface.co/Epiculous/Gnosis) |
|
- [ChaoticNeutrals/Luminous_Opus](https://huggingface.co/datasets/ChaoticNeutrals/Luminous_Opus) |
|
- [ChaoticNeutrals/Synthetic-Dark-RP](https://huggingface.co/datasets/ChaoticNeutrals/Synthetic-Dark-RP) |
|
- [ChaoticNeutrals/Synthetic-RP](https://huggingface.co/datasets/ChaoticNeutrals/Synthetic-RP) |
|
- [Gryphe/Sonnet3.5-SlimOrcaDedupCleaned](https://huggingface.co/datasets/Gryphe/Sonnet3.5-SlimOrcaDedupCleaned) |
|
- [Gryphe/Opus-WritingPrompts](https://huggingface.co/datasets/Gryphe/Opus-WritingPrompts) |
|
- [meseca/writing-opus-6k](https://huggingface.co/datasets/meseca/writing-opus-6k) |
|
- [meseca/opus-instruct-9k](https://huggingface.co/datasets/meseca/opus-instruct-9k) |
|
- [PJMixers/grimulkan_theory-of-mind-ShareGPT](https://huggingface.co/datasets/PJMixers/grimulkan_theory-of-mind-ShareGPT) |
|
- [NobodyExistsOnTheInternet/ToxicQAFinal](https://huggingface.co/datasets/NobodyExistsOnTheInternet/ToxicQAFinal) |
|
- [Undi95/toxic-dpo-v0.1-sharegpt](https://huggingface.co/datasets/Undi95/toxic-dpo-v0.1-sharegpt) |
|
- [cgato/SlimOrcaDedupCleaned](https://huggingface.co/datasets/cgato/SlimOrcaDedupCleaned) |
|
- [kalomaze/Opus_Instruct_25k](https://huggingface.co/datasets/kalomaze/Opus_Instruct_25k) |
|
- [Doctor-Shotgun/no-robots-sharegpt](https://huggingface.co/datasets/Doctor-Shotgun/no-robots-sharegpt) |
|
- [Norquinal/claude_multiround_chat_30k](https://huggingface.co/datasets/Norquinal/claude_multiround_chat_30k) |
|
- [nothingiisreal/Claude-3-Opus-Instruct-15K](https://huggingface.co/datasets/nothingiisreal/Claude-3-Opus-Instruct-15K) |
|
- All the Aesirs dataset, cleaned, unslopped |
|
- All le luminae dataset, cleaned, unslopped |
|
- Small part of Airoboros reduced |
|
|
|
We sadly didn't find the sources of the following, DM us if you recognize your set ! |
|
|
|
- Opus_Instruct-v2-6.5K-Filtered-v2-sharegpt |
|
- claude_sharegpt_trimmed |
|
- CapybaraPure_Decontaminated-ShareGPT_reduced |
|
|
|
## Datasets credits: |
|
- Epiculous |
|
- ChaoticNeutrals |
|
- Gryphe |
|
- meseca |
|
- PJMixers |
|
- NobodyExistsOnTheInternet |
|
- cgato |
|
- kalomaze |
|
- Doctor-Shotgun |
|
- Norquinal |
|
- nothingiisreal |
|
|
|
## Others |
|
|
|
Undi: If you want to support us, you can [here](https://ko-fi.com/undiai). |
|
|
|
IkariDev: Visit my [retro/neocities style website](https://ikaridevgit.github.io/) please kek |
|
|
|
</details> |
