File size: 2,296 Bytes
3f49b2a 5a5d35c 2aa575c 3f49b2a 6a29385 3f49b2a 6a29385 51b0541 6a29385 3f49b2a 6a29385 3f49b2a 6c61771 6a29385 3f49b2a 6a29385 3f49b2a 6a29385 6c61771 6a29385 3f49b2a 6a29385 3f49b2a 6a29385 3f49b2a 6a29385 3f49b2a 6a29385 3f49b2a 8f21a09 097e2d8 8f21a09 6a29385 3f49b2a 6a29385 3f49b2a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 |
---
library_name: peft
base_model: microsoft/Phi-3-mini-4k-instruct
datasets:
- argilla/ultrafeedback-binarized-preferences-cleaned
- >-
flax-sentence-embeddings/stackexchange_titlebody_best_and_down_voted_answer_jsonl
language:
- en
---
# Model Card for Phi-3-mini-4k-instruct DPO
## Model Details
- **Model Name:** Phi-3-mini-4k-instruct DPO
- **Publisher:** Team chatterbox, EPFL
- **Model Type:** Language Model, Fine-tuned with direct preference optimization (DPO)
- **Training Environment:** Trained on the EPFL SCITAS cluster using a 32GB GPU.
## Intended Use
- **Primary Applications:** This model is designed as part of an AI-Tutor system.
- **Intended Audience:** Educators, students, and developers creating educational AI applications.
## Model/Data Description
### Training Data
- **Datasets Used:**
- **Milestone 1 Dataset:** Includes 1522 unique questions with preference pairs based on the 'overall' rating, totaling 20k+ usable entries after processing.
- **Stack Exchange Dataset:** Filters content from specific domains within the Stack Exchange network, using upvoted and downvoted answers to form preference pairs. Total entries after preprocessing: 54458.
- **Ultra Feedback:** Utilizes responses rated on criteria like truthfulness and helpfulness to form preference pairs, with a total of 60917 entries after preprocessing.
- **Preprocessing Details:** Entries with identical chosen and rejected answers were removed. Datasets were formatted as JSONL where each line represents a JSON object with a "prompt", "chosen", and "rejected" response.
## Training Procedure
- **Configurations:** (Refer to the provided `training_args` and `trainer` configuration)
- **Evaluation Metrics:** The primary metric for model performance is `eval_loss`, with the aim to minimize this value.
## Evaluation Results
- **Accuracies:** eval/rewards/accuracies - 0.83
- **Loss:** eval/loss - 0.47
- **Margins:** eval/margins - 4.31
### MT-Bench
- **Single Grading Score, Overall Avg.** - 8.2
- **STEM Score** - 9.8 (higher than GPT-4)
## References
- **[Include references and citations for datasets, tools, and methodologies used.]**
- PEFT 0.11.1 |