|
--- |
|
library_name: transformers |
|
tags: [] |
|
--- |
|
# Mistral 7b Self-Alignment DPO Model |
|
|
|
The Mistral 7b Self-Alignment DPO Model is an adapter fine-tuned for self-alignment and harmlessness using the Direct Preference Optimization (DPO) technique. It has been trained utilizing the Mistral Self-Alignment Preference Dataset, accessible [here](https://huggingface.co/datasets/August4293/Preference-Dataset). |
|
|
|
Detailed information about the DPO fine-tuning process and its application for self-alignment can be found on the corresponding [GitHub page](https://github.com/August-murr/Lab/tree/main/Mistral%20Self%20Alignment). |
|
|
|
## Model Details: |
|
- **Base Model:** Mistral 7b |
|
- **Fine-Tuning Purpose:** Self-Alignment and Harmlessness |
|
- **Fine-Tuning Method:** Direct Preference Optimization (DPO) |
|
- **Fine-Tuning Dataset:** [Mistral Self-Alignment Preference Dataset](https://huggingface.co/datasets/August4293/Preference-Dataset) |
|
|
|
|
|
|
