|
--- |
|
license: mit |
|
datasets: |
|
- ethical-spectacle/biased-corpus |
|
language: |
|
- en |
|
metrics: |
|
- f1(0.8998) |
|
- precision |
|
- recall() |
|
library_name: transformers |
|
co2_eq_emissions: |
|
emissions: 10 |
|
source: Code Carbon |
|
training_type: fine-tuning |
|
geographical_location: Albany, New York |
|
hardware_used: T4 |
|
base_model: |
|
- google-bert/bert-base-uncased |
|
pipeline_tag: text-classification |
|
tags: |
|
- Social Bias |
|
--- |
|
|
|
## How to Use |
|
``` |
|
classifier = pipeline("text-classification", model="maximuspowers/bias-type-classifier") // pass in return_all_scores=True for multi-label |
|
result = classifier("Tall people are so clumsy") |
|
|
|
// Example Result |
|
// [ |
|
// { |
|
// "label": "physical", |
|
// "score": 0.9972801208496094 |
|
// } |
|
// ] |
|
``` |
|
|
|
This model was trained on a [synthetic dataset](https://huggingface.co/datasets/ethical-spectacle/biased-corpus) of biased statements and questions, generated by Mistal 7B as part of the [GUS-Net paper](https://arxiv.org/abs/2410.08388). |
|
|
|
### Model Performance: |
|
| Label | F1 Score | Precision | Recall | |
|
|-----------------|----------|-----------|--------| |
|
| **Macro Average** | **0.8998** | **0.9213** | **0.8807** | |
|
| racial | 0.8613 | 0.9262 | 0.8049 | |
|
| religious | 0.9655 | 0.9716 | 0.9595 | |
|
| gender | 0.9160 | 0.9099 | 0.9223 | |
|
| age | 0.9185 | 0.9683 | 0.8737 | |
|
| nationality | 0.9083 | 0.9053 | 0.9113 | |
|
| sexuality | 0.9304 | 0.9484 | 0.9131 | |
|
| socioeconomic | 0.8273 | 0.8727 | 0.7864 | |
|
| educational | 0.8791 | 0.9091 | 0.8511 | |
|
| disability | 0.8713 | 0.8762 | 0.8665 | |
|
| political | 0.9127 | 0.8914 | 0.9351 | |
|
| physical | 0.9069 | 0.9547 | 0.8635 | |
|
|
|
### Training Params: |
|
**Learning Rate:** 5e-5 |
|
**Batch Size:** 16 |
|
**Epochs:** 3 |
