--- license: mit library_name: peft tags: - generated_from_trainer base_model: microsoft/phi-2 model-index: - name: hate-phi results: [] --- # hate-phi This model is a fine-tuned version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 0.3268 - Classification Report: precision recall f1-score support 0 0.57 0.08 0.14 438 1 0.91 0.97 0.93 5755 2 0.80 0.79 0.80 1242 accuracy 0.89 7435 macro avg 0.76 0.61 0.62 7435 weighted avg 0.87 0.89 0.87 7435 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0002 - train_batch_size: 64 - eval_batch_size: 16 - seed: 42 - gradient_accumulation_steps: 4 - total_train_batch_size: 256 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 1 - num_epochs: 1 ### Training results | Training Loss | Epoch | Step | Validation Loss | Classification Report | |:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:| | 0.8106 | 0.37 | 25 | 0.4551 | precision recall f1-score support 0 0.18 0.03 0.04 438 1 0.85 0.97 0.91 5755 2 0.75 0.46 0.57 1242 accuracy 0.83 7435 macro avg 0.59 0.49 0.51 7435 weighted avg 0.79 0.83 0.80 7435 | | 0.3677 | 0.74 | 50 | 0.3374 | precision recall f1-score support 0 0.51 0.09 0.16 438 1 0.91 0.95 0.93 5755 2 0.77 0.83 0.80 1242 accuracy 0.88 7435 macro avg 0.73 0.63 0.63 7435 weighted avg 0.87 0.88 0.87 7435 | ### Framework versions - PEFT 0.11.1 - Transformers 4.39.3 - Pytorch 2.1.2 - Datasets 2.18.0 - Tokenizers 0.15.2