README.md

metadata

library_name: transformers
tags: []

Mistral 7b Self-Alignment DPO Model

The Mistral 7b Self-Alignment DPO Model is an adapter fine-tuned for self-alignment and harmlessness using the Direct Preference Optimization (DPO) technique. It has been trained utilizing the Mistral Self-Alignment Preference Dataset, accessible here.

Detailed information about the DPO fine-tuning process and its application for self-alignment can be found on the corresponding GitHub page.

Model Details:

• Base Model: Mistral 7b
• Fine-Tuning Purpose: Self-Alignment and Harmlessness
• Fine-Tuning Method: Direct Preference Optimization (DPO)
• Fine-Tuning Dataset: Mistral Self-Alignment Preference Dataset