File size: 1,700 Bytes
40a29de |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
---
license: apache-2.0
---
# Model Card for decruz07/kellemar-DPO-7B-e
<!-- Provide a quick summary of what the model is/does. -->
Learning Rate: 5e-5, steps 300
## Model Details
Created with beta = 0.05
### Model Description
<!-- Provide a longer summary of what this model is. -->
- **Developed by:** @decruz
- **Funded by [optional]:** my full-time job
- **Finetuned from model [optional]:** teknium/OpenHermes-2.5-Mistral-7B
## Uses
You can use this for basic inference. You could probably finetune with this if you want to.
## How to Get Started with the Model
You can create a space out of this, or use basic python code to call the model directly and make inferences to it.
[More Information Needed]
## Training Details
The following was used:
`training_args = TrainingArguments(
per_device_train_batch_size=4,
gradient_accumulation_steps=4,
gradient_checkpointing=True,
learning_rate=5e-5,
lr_scheduler_type="cosine",
max_steps=200,
save_strategy="no",
logging_steps=1,
output_dir=new_model,
optim="paged_adamw_32bit",
warmup_steps=100,
bf16=True,
report_to="wandb",
)
# Create DPO trainer
dpo_trainer = DPOTrainer(
model,
ref_model,
args=training_args,
train_dataset=dataset,
tokenizer=tokenizer,
peft_config=peft_config,
beta=0.1,
max_prompt_length=1024,
max_length=1536,
)`
### Training Data
This was trained with https://huggingface.co/datasets/argilla/distilabel-intel-orca-dpo-pairs
### Training Procedure
Trained with Labonne's Google Colab Notebook on Finetuning Mistral 7B with DPO.
## Model Card Authors [optional]
@decruz
## Model Card Contact
@decruz on X/Twitter |