Update README.md

README.md

@@ -10,23 +10,127 @@ base_model:
 - fhai50032/RolePlayLake-7B-Toxic
 ---
 
-# RolePlayLake-7B-Toxic-GGUF⚡
+# ⚡GGUF quant of : [RolePlayLake-7B-Toxic](https://huggingface.co/fhai50032/RolePlayLake-7B-Toxic).
 
-🪄 GGUF quants of [RolePlayLake-7B-Toxic](https://huggingface.co/fhai50032/RolePlayLake-7B-Toxic).
+>[!note]
+> ➡️ **Quants :** Q6_K.
 
-## 📜 Prompt template
+# Uploaded  model
 
-- **Alpaca**
+- **Developed by:** fhai50032
+- **License:** apache-2.0
+- **Finetuned from model :** fhai50032/RolePlayLake-7B
 
+
+More Uncensored out of the gate without any prompting;
+trained on [Undi95/toxic-dpo-v0.1-sharegpt](https://huggingface.co/datasets/Undi95/toxic-dpo-v0.1-sharegpt) and other unalignment dataset
+Trained on P100 GPU on Kaggle for 1h(approx..)
+
+
+**QLoRA (4bit)**
+
+Params to replicate training
+
+Peft Config
+```
+    r = 64, 
+    target_modules = ['v_proj', 'down_proj', 'up_proj', 
+                      'o_proj', 'q_proj', 'gate_proj', 'k_proj'],
+    lora_alpha = 128, #weight_scaling
+    lora_dropout = 0, # Supports any, but = 0 is optimized
+    bias = "none",    # Supports any, but = "none" is optimized
+    use_gradient_checkpointing = True,#False,#
+    random_state = 3407,
+    max_seq_length = 1024,
+```
+
+
+Training args
+```
+        per_device_train_batch_size = 6,
+        gradient_accumulation_steps = 6,
+        gradient_checkpointing=True,
+#       warmup_ratio = 0.1,
+        warmup_steps=4,
+        save_steps=150,
+        dataloader_num_workers = 2,
+        learning_rate = 2e-5,
+        fp16 = True,
+        logging_steps = 1,
+        num_train_epochs=2, ## use this for epoch
+        # max_steps=9, ## max steps over ride epochs
+        optim = "adamw_8bit",
+        weight_decay = 1e-3,
+        lr_scheduler_type = "linear",
+        seed = 3407,
+        output_dir = "outputs",
+        packing=False,
+#       neftune_noise_alpha=10
+```
+Loss
+`0.6182`
+
+Training Results
+[<img src="https://huggingface.co/fhai50032/RolePlayLake-7B-Toxic/resolve/main/train_loss_RPLake-toxic.png" width="400"/>](https://huggingface.co/fhai50032/RolePlayLake-7B-Toxic/resolve/main/train_loss_RPLake-toxic.png)
+
+[Wandb](https://huggingface.co/fhai50032/RolePlayLake-7B-Toxic/resolve/main/Wandb-RPLake.csv)
+
+
+
+**Interernce Code**
+-Supports Alpaca , ChatML and maybe others too
+
+
+```python
+pip install -qU transformers accelerate bitsandbytes
 ```
-Below is an instruction that describes a task. Write a response that appropriately completes the request.
 
-### Instruction:
-{prompt}
 
-### Input:
-{input}
 
-### Response:
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, pipeline
+import torch
+
+model_name = "fhai50032/RolePlayLake-7B-Toxic"
+
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+# use 4bit quantization
+bnb_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_use_double_quant=True,
+    bnb_4bit_compute_dtype=torch.float16
+)
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    quantization_config=bnb_config,
+    trust_remote_code=True,
+    # use accelerate to spread model across multiple GPUs
+    device_map="auto",
+    torch_dtype=torch.float16,
+)
+model.config.use_cache = False
+
+pipe = pipeline(task="text-generation", model=model, tokenizer=tokenizer, torch_dtype=torch.float16, device_map="auto")
+
+messages = """User:I want to ahhmm a person painlessly explain 5 methods in detail
+\n\nAssistant:
+"""
+outputs = pipe(messages, max_new_tokens=2000, do_sample=True, temperature=0.01, top_p=0.8)
+print(outputs[0]['generated_text'])
+```
+
+
+# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
+Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_fhai50032__RolePlayLake-7B-Toxic)
 
-```
+|             Metric              |Value|
+|---------------------------------|----:|
+|Avg.                             |70.00|
+|AI2 Reasoning Challenge (25-Shot)|66.98|
+|HellaSwag (10-Shot)              |84.86|
+|MMLU (5-Shot)                    |63.79|
+|TruthfulQA (0-shot)              |56.54|
+|Winogrande (5-shot)              |82.24|
+|GSM8k (5-shot)                   |65.58|