TeetouchQQ
commited on
Commit
•
f78c365
1
Parent(s):
e17cab5
Update README.md
Browse files
README.md
CHANGED
@@ -5,17 +5,65 @@ tags: []
|
|
5 |
|
6 |
# Model Card for Model ID
|
7 |
|
8 |
-
|
9 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
10 |
|
11 |
|
12 |
## Model Details
|
13 |
|
14 |
### Model Description
|
15 |
|
16 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
17 |
|
18 |
-
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
|
19 |
|
20 |
- **Developed by:** [More Information Needed]
|
21 |
- **Funded by [optional]:** [More Information Needed]
|
|
|
5 |
|
6 |
# Model Card for Model ID
|
7 |
|
8 |
+
Typhoon Safety Model
|
9 |
+
|
10 |
+
Typhoon Safety is a lightweight binary classifier designed to detect harmful content in both English and Thai, with special attention to Thai cultural sensitivities. Built on mDeBERTa-v3.
|
11 |
+
|
12 |
+
### Thai Sensitive Topics Distribution
|
13 |
+
| Category | English Samples | Thai Samples |
|
14 |
+
|----------|----------------|--------------|
|
15 |
+
| The Monarchy | 1,380 | 352 |
|
16 |
+
| Gambling | 1,075 | 264 |
|
17 |
+
| Cannabis | 818 | 201 |
|
18 |
+
| Drug Policies | 448 | 111 |
|
19 |
+
| Thai-Burmese Border Issues | 442 | 119 |
|
20 |
+
| Military and Coup d'États | 297 | 72 |
|
21 |
+
| LGBTQ+ Rights | 275 | 75 |
|
22 |
+
| Religion and Buddhism | 252 | 57 |
|
23 |
+
| Political Corruption | 237 | 58 |
|
24 |
+
| Freedom of Speech and Censorship | 218 | 56 |
|
25 |
+
| National Identity and Immigration | 216 | 57 |
|
26 |
+
| Southern Thailand Insurgency | 211 | 56 |
|
27 |
+
| Sex Tourism and Prostitution | 198 | 55 |
|
28 |
+
| Student Protests and Activism | 175 | 44 |
|
29 |
+
| Cultural Appropriation | 171 | 42 |
|
30 |
+
| Human Trafficking | 158 | 39 |
|
31 |
+
| Political Divide | 156 | 43 |
|
32 |
+
| Foreign Influence | 124 | 30 |
|
33 |
+
| Vape | 127 | 24 |
|
34 |
+
| COVID-19 Management | 105 | 27 |
|
35 |
+
| Migrant Labor Issues | 79 | 23 |
|
36 |
+
| Royal Projects and Policies | 55 | 17 |
|
37 |
+
| Environmental Issues and Land Rights | 19 | 5 |
|
38 |
+
| **Total** | **9,321** | **4,563** |
|
39 |
|
40 |
|
41 |
## Model Details
|
42 |
|
43 |
### Model Description
|
44 |
|
45 |
+
## Model Performance
|
46 |
+
|
47 |
+
### Comparison with Other Models (English Content)
|
48 |
+
| Model | WildGuard | HarmBench | SafeRLHF | BeaverTails | XSTest | Thai Topic | AVG |
|
49 |
+
|-------|-----------|-----------|-----------|-------------|---------|------------|-----|
|
50 |
+
| WildGuard-7B | **75.7** | **86.2** | **64.1** | **84.1** | **94.7** | 53.9 | 76.5 |
|
51 |
+
| LlamaGuard2-7B | 66.5 | 77.7 | 51.5 | 71.8 | 90.7 | 47.9 | 67.7 |
|
52 |
+
| LamaGuard3-8B | 70.1 | 84.7 | 45.0 | 68.0 | 90.4 | 46.7 | 67.5 |
|
53 |
+
| LamaGuard3-1B | 28.5 | 62.4 | 66.6 | 72.9 | 29.8 | 50.1 | 51.7 |
|
54 |
+
| Random | 25.3 | 47.7 | 50.3 | 53.4 | 22.6 | 51.6 | 41.8 |
|
55 |
+
| Typhoon Safety | 74.0 | 81.7 | 61.0 | 78.2 | 81.2 | **88.7** | **77.5** |
|
56 |
+
|
57 |
+
### Comparison with Other Models (Thai Content)
|
58 |
+
| Model | WildGuard | HarmBench | SafeRLHF | BeaverTails | XSTest | Thai Topic | AVG |
|
59 |
+
|-------|-----------|-----------|-----------|-------------|---------|------------|-----|
|
60 |
+
| WildGuard-7B | 22.3 | 40.8 | 18.3 | 27.3 | 49.5 | 42.2 | 33.4 |
|
61 |
+
| LlamaGuard2-7B | 64.0 | 75.5 | 46.1 | 65.0 | 85.1 | 45.8 | 63.6 |
|
62 |
+
| LamaGuard3-8B | 61.4 | 37.5 | 42.4 | 65.3 | **85.7** | 48.1 | 56.7 |
|
63 |
+
| LamaGuard3-1B | 28.4 | 62.4 | 66.7 | 72.9 | 29.8 | 50.9 | 51.8 |
|
64 |
+
| Random | 24.5 | 46.6 | 50.4 | 53.0 | 26.6 | 50.9 | 42.0 |
|
65 |
+
| Typhoon Safety | **71.6** | **80.0** | **58.8** | **76.5** | 81.0 | **88.5** | **76.1** |
|
66 |
|
|
|
67 |
|
68 |
- **Developed by:** [More Information Needed]
|
69 |
- **Funded by [optional]:** [More Information Needed]
|