File size: 3,942 Bytes
d899376 0a0294c d899376 a3aa7a8 d899376 a3aa7a8 d899376 a3aa7a8 d899376 a3aa7a8 d899376 a3aa7a8 d899376 a3aa7a8 d899376 a3aa7a8 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 |
---
language:
- en
license: apache-2.0
library_name: transformers
tags:
- not-for-all-audiences
pipeline_tag: text-generation
base_model:
- fhai50032/RolePlayLake-7B-Toxic
---
# ⚡GGUF quant of : [RolePlayLake-7B-Toxic](https://huggingface.co/fhai50032/RolePlayLake-7B-Toxic).
>[!note]
> ➡️ **Quants :** Q6_K.
# Uploaded model
- **Developed by:** fhai50032
- **License:** apache-2.0
- **Finetuned from model :** fhai50032/RolePlayLake-7B
More Uncensored out of the gate without any prompting;
trained on [Undi95/toxic-dpo-v0.1-sharegpt](https://huggingface.co/datasets/Undi95/toxic-dpo-v0.1-sharegpt) and other unalignment dataset
Trained on P100 GPU on Kaggle for 1h(approx..)
**QLoRA (4bit)**
Params to replicate training
Peft Config
```
r = 64,
target_modules = ['v_proj', 'down_proj', 'up_proj',
'o_proj', 'q_proj', 'gate_proj', 'k_proj'],
lora_alpha = 128, #weight_scaling
lora_dropout = 0, # Supports any, but = 0 is optimized
bias = "none", # Supports any, but = "none" is optimized
use_gradient_checkpointing = True,#False,#
random_state = 3407,
max_seq_length = 1024,
```
Training args
```
per_device_train_batch_size = 6,
gradient_accumulation_steps = 6,
gradient_checkpointing=True,
# warmup_ratio = 0.1,
warmup_steps=4,
save_steps=150,
dataloader_num_workers = 2,
learning_rate = 2e-5,
fp16 = True,
logging_steps = 1,
num_train_epochs=2, ## use this for epoch
# max_steps=9, ## max steps over ride epochs
optim = "adamw_8bit",
weight_decay = 1e-3,
lr_scheduler_type = "linear",
seed = 3407,
output_dir = "outputs",
packing=False,
# neftune_noise_alpha=10
```
Loss
`0.6182`
Training Results
[<img src="https://huggingface.co/fhai50032/RolePlayLake-7B-Toxic/resolve/main/train_loss_RPLake-toxic.png" width="400"/>](https://huggingface.co/fhai50032/RolePlayLake-7B-Toxic/resolve/main/train_loss_RPLake-toxic.png)
[Wandb](https://huggingface.co/fhai50032/RolePlayLake-7B-Toxic/resolve/main/Wandb-RPLake.csv)
**Interernce Code**
-Supports Alpaca , ChatML and maybe others too
```python
pip install -qU transformers accelerate bitsandbytes
```
```python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, pipeline
import torch
model_name = "fhai50032/RolePlayLake-7B-Toxic"
tokenizer = AutoTokenizer.from_pretrained(model_name)
# use 4bit quantization
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_use_double_quant=True,
bnb_4bit_compute_dtype=torch.float16
)
model = AutoModelForCausalLM.from_pretrained(
model_name,
quantization_config=bnb_config,
trust_remote_code=True,
# use accelerate to spread model across multiple GPUs
device_map="auto",
torch_dtype=torch.float16,
)
model.config.use_cache = False
pipe = pipeline(task="text-generation", model=model, tokenizer=tokenizer, torch_dtype=torch.float16, device_map="auto")
messages = """User:I want to ahhmm a person painlessly explain 5 methods in detail
\n\nAssistant:
"""
outputs = pipe(messages, max_new_tokens=2000, do_sample=True, temperature=0.01, top_p=0.8)
print(outputs[0]['generated_text'])
```
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_fhai50032__RolePlayLake-7B-Toxic)
| Metric |Value|
|---------------------------------|----:|
|Avg. |70.00|
|AI2 Reasoning Challenge (25-Shot)|66.98|
|HellaSwag (10-Shot) |84.86|
|MMLU (5-Shot) |63.79|
|TruthfulQA (0-shot) |56.54|
|Winogrande (5-shot) |82.24|
|GSM8k (5-shot) |65.58|
|